A Million Tokens of Context: What It's Actually Good For

GPT-6 Astra's 1.05 million token context window sounds like an unqualified win. In practice, long context is a specialized tool with real failure modes — here's what it actually delivers.

A Million Tokens of Context: What It's Actually Good For

GPT-6 Astra's 1.05 million token context window has dominated the specification conversation since September 3. But long context is one of the most oversold capabilities in modern AI, and the truth — as usual — is more interesting than the number.

What a million tokens holds. Roughly 750,000 words: an entire mid-sized codebase, a complete regulatory filing set, a full novel with annotations, or several hours of meeting transcripts. The unit of work changes from "passage" to "corpus," and that genuinely enables new applications — whole-repository engineering, longitudinal document analysis, litigation-scale review.

What it does not do. Long context is not retrieval, and models degrade non-uniformly across it. Information in the middle of a huge window gets less attention than content at the edges — the well-documented "lost in the middle" effect persists in current generations. Precision questions over needle-in-haystack tests score well; synthesized reasoning across fifty documents at once remains where errors concentrate.

The economics. A full context window at Astra's $10 per million input tokens means a fully-loaded request costs real money before generating a single output token — and repeating that context across a conversation multiplies the bill. This is why caching matters so much: Anthropic's $0.20 cache reads exist precisely because stable prefixes dominate long-context workloads, and routing strategies that cache aggressively can cut effective costs dramatically.

The practical pattern. Teams getting value from million-token windows overwhelmingly use them hybrid-style: retrieval to select a relevant subset, long context to hold that subset plus its surroundings, and a premium reasoning pass over the assembly. The window is the container for thoroughness, not a replacement for selectivity.

The honest summary: long context is a real capability unlock for a specific class of corpus-scale work, and a expensive illusion if you expected it to make retrieval architecture obsolete. It doesn't.