Nvidia researchers say huge AI context windows are mostly unnecessary

Started by SignalMage, Today at 04:00 PM

Previous topic - Next topic

SignalMage and 1 Guest are viewing this topic.

Topic: Nvidia researchers say huge AI context windows are mostly unnecessary   Views(Read 18 times)
Active members in this topic:
SignalMage(1)

SignalMage

Bigger context windows may not be the best path for AI agents. The Decoder reports on an Nvidia Research paper titled You don't need long context, which argues that agents can work from periodic summaries and targeted keyword search instead of keeping their full history

With context compacted after every action, Claude Opus 4.5 scored 70.2 percent on SWE-bench Verified against 70.0 percent with full history. Its largest context per task fell from 200,000 to 16,000 tokens

GPT 5.6 Sol scored 82.4 percent with the same method, two points below its baseline. Generating each token at 256,000 tokens of context costs between 3.6 and 7 times as much as at 16,000, depending on the model

The team estimates savings on long tasks of up to 8.8 times. It proposes a new class of models designed to work from short notes, and the paper itself was first posted on 3 November

Keeping costs down matters as agents run for hours or days, and short notes are how people work too. If accuracy barely drops, many developers will rethink their designs

Do you think long context windows are overrated? And would you trust an agent that works from summaries?

COYB — you know who you are

Browser ID: smf (possibly_robot, is_webkit)
Templates: 6: index (default), RelatedTopics (default), Block (default), Optimus (default), Display (default), GenericControls (default).
Sub templates: 12: init, html_above, Blockheaders_above, body_above, Blockindex_above, footer_counters_above, main, footer_counters_below, Blockindex_below, body_below, Blockheaders_below, html_below.
Language files: 11: index+Modifications.english (default), Turnstile.english (default), RelatedTopics.english (default), Block.english (default), Optimus/Optimus.english (default), Post.english (default), Editor.english (default), Drafts.english (default), giphy.english (default), QRCode.english (default), klipy.english (default).
Style sheets: 7: index.css, attachments.css, turnstile.css, ajaxsearchlite.css, jquery.sceditor.css, show_password.css, responsive.css.
Hooks called: 147 (show)
Instances created: 14 (show)
Files included: 138 - 1847KB. (show)
Memory used: 11075KB.
Tokens: post-login.
Cache hits: 22: 0.00367s for 44,505 bytes (show)
Cache misses: 3: (show)
Queries used: 26.

[Show Queries]