The New American AI Model Designed to be Customized

Thinking Machines: Inkling supports a context window of one million tokens, and a trillion comparisons per layer across 66 layers is far beyond what any reasonable amount of hardware can deliver. This is where a sliding-window layer restricts each token to a fixed number of recent tokens rather than everything before it. Inkling alternates between sliding-window layers and full-attention layers at a ratio of 5:1. Integration notes published by the vLLM project put concrete numbers on the split, describing the 66 layers as 55 sliding-window layers and 11 full-attention layers. Information from far away travels through those eleven full layers. For example, picture a fact at token 200 that matters for a prediction at token 900,000. In layer 6, the first full-attention layer, the representation at position 900,000 can reach back and pick up that fact directly. From there, it rides forward within the local representations, is refreshed at layer 12, again at layer 18, and so on. The long-range path exists on roughly one layer in six, and the other five handle nearby context at a fraction of the price.

Thinking Machines: Everything so far concerns text moving through the model. Images and audio have to get in first, and Inkling accepts both without a separately trained encoder standing in front of it. Sound arrives as a mel spectrogram, which is a standard way of representing audio as a grid of numbers. The dMel method then rounds each of those loudness values to one of a fixed set of levels, in the same way you might round 0.73 to 0.7. That is the entire conversion. No separate audio model needs training beforehand, because rounding numbers requires no training. In the case of images, an image is cut into square patches measuring 40 by 40 pixels. A 400 by 400 pixel image therefore becomes 100 patches. Each patch passes through a small four-stage network called an hMLP stem, which combines the pixels within that patch and processes each patch independently of every other one. Both then pass through a lightweight conversion layer and join the text tokens in a single sequence, processed by the same 66 layers we have been talking about. Thinking Machines states that these multimodal components were trained from scratch on general-domain data, meaning they learned alongside the rest of the model rather than arriving pretrained.

Thinking Machines: Effort is a number between 0 and 1. The documented presets run from 0 for none, through 0.1 for minimal and 0.2 for low, then 0.7 for medium, 0.9 as the default, and 0.99 at the top. The spacing between those values is uneven, which is a hint that the number represents a learned response rather than a token allowance. Thinking Machines trained Inkling's response to the effort setting during reinforcement learning, a training stage where the model produces complete attempts at tasks and receives a score for each attempt, then adjusts toward whatever scored well. During that stage, Thinking Machines varied the effort message across attempts while also adjusting the cost charged per token generated: An attempt labelled high effort could produce lengthy working-out without much penalty. An attempt labelled low effort was charged heavily for every token, so short answers scored better. Across many attempts, the connection between the message and the profitable length was learned. Higher effort encourages more reasoning without guaranteeing a longer response or a better one on any single sample. Sweeping the effort setting across its range produces a curve of score against tokens generated. On the coding benchmark Terminal Bench 2.1, Inkling reaches the same score as NVIDIA's Nemotron 3 Ultra while producing roughly a third as many tokens.

Thinking Machines: Thinking Machines picked an older technique over the current standard. This is due to the lengths the model never saw during training. Almost every recent open model uses Rotary Position Embedding, shortened to RoPE. Each token's query and key are treated as points that get rotated by an angle proportional to that token's position in the sequence. However, those rotation angles were only ever encountered at positions the model actually trained on. If training used sequences up to 32,000 tokens, then every angle the model learned to interpret came from that range. If we ask it about position 900,000, the angle involved falls outside anything it has experience with. Inkling uses a relative scheme in the style of Shaw and colleagues. Rather than encoding where each token sits, this approach learns a value for each distance between two tokens and adds that value directly to the comparison score. For example, tokens at positions 5 and 9 are 4 apart. Tokens at positions 500,005 and 500,009 are also 4 apart. A relative scheme treats both pairs identically, because 4 is 4 wherever it occurs. Distances beyond some cutoff, say anything more than 128 apart, all share the same learned value, so a pair 900,000 tokens apart uses a value the model has seen countless times during training. Nothing has to be extrapolated. Thinking Machines states that this performed better and extrapolated better to longer sequences than RoPE in their testing.