Unraveling What Thinking Machines’ Inkling Signals About AI

Thinking Machines has publicly released the full weights of its latest foundation model, Inkling, under the Apache 2.0 license, making it available for download on Hugging Face. This marks a notable departure from typical proprietary model releases, emphasizing transparency and open access in AI development.

Inkling is a 975-billion-parameter mixture-of-experts transformer supporting multimodal input—text, images, and audio—with a 1-million-token context window. It was trained on 45 trillion tokens across various media types, using a hybrid optimizer and over 30 million reinforcement learning rollouts. The model’s weights are available openly on Hugging Face, under Apache 2.0, allowing for modification, deployment, and commercial use.

However, the release includes important caveats: the weights are not open source, as the training data and pipeline are not published. Additionally, reports suggest that Thinking Machines enforces a separate Model Acceptable Use Policy (AUP), which restricts surveillance, deception, and automated decision-making affecting individuals, raising questions about the scope of openness and enforceability.

Despite these restrictions, the release is a significant step toward transparency, as it allows inspection and fine-tuning of the model’s weights, unlike typical proprietary models that are only accessible via APIs.

Implications of Open Release for AI Transparency

The open release of Inkling’s weights under a permissive license represents a shift toward greater transparency in AI development. It allows researchers and developers to inspect, modify, and deploy the model independently, reducing reliance on API-based access and potential vendor lock-in.

However, the accompanying restrictions via the AUP and the lack of open training data introduce questions about the true openness of the model. This development could influence industry standards, encouraging more open practices, but also highlights ongoing tensions between openness and control in AI.

Background on Open-Weight Model Releases

Until now, most large foundation models have been released as closed APIs or with limited access to weights, citing concerns over misuse and safety. Few companies have openly published full model weights, and those that do often impose restrictions through licenses or use policies. The recent release of Inkling’s weights by Thinking Machines marks a notable exception, emphasizing transparency but also raising questions about the scope of openness in practice.

Previously, the industry has seen debates over open sourcing models versus proprietary control, especially after incidents where models were turned off or restricted by governments or companies. Inkling’s release comes amid ongoing discussions about balancing openness, safety, and commercial interests.

“We believe in open access to our models, but responsible use policies are essential to prevent misuse.”

— Thinking Machines spokesperson

Unclear Aspects of Inkling’s Open Model Policy

It remains unclear how enforceable the separate Model Acceptable Use Policy (AUP) is, and whether it effectively limits the ways the model can be used despite the open weights. The specifics of how the restrictions apply to modified versions or derivative works are also not fully confirmed.

Additionally, the extent to which the training data and pipeline will be disclosed in the future is unknown, which impacts the overall transparency of the model’s development process.

Next Steps in Model Evaluation and Industry Impact

Independent researchers and industry observers will likely scrutinize Inkling’s performance across benchmarks and real-world applications. Further transparency on the training data and use policies will be critical to assess the model’s openness fully.

Expect ongoing debates about the balance between open access and safety restrictions, as well as potential adoption of similar open-weight releases by other organizations.

Key Questions

What makes Inkling different from other foundation models?

Inkling is notable for its full weights being released openly under Apache 2.0, allowing modification and deployment, unlike most models which are only accessible via APIs.

Are there restrictions on how Inkling can be used?

Yes, according to reports, Thinking Machines enforces a separate Model Acceptable Use Policy that restricts surveillance, deception, and automated decision-making affecting individuals, despite the open weights.

Will the training data for Inkling be released?

No, the training data and full pipeline have not been published, which limits full transparency about the model’s origins.

Why is this release significant for the AI industry?

It signals a shift toward more open practices in large model development, potentially influencing industry standards and encouraging more transparency, even amidst restrictions.

Source: ThorstenMeyerAI.com