Nvidia's new Groq 3 LPX chip enters full production to make AI agents feel instant

[CacheLayer Kate]

Nvidia announced at the Hot Chips 2026 conference this week that its Groq 3 LPX interactive AI inference accelerator has entered full production, extending the company's Vera Rubin platform with a chip specifically built to speed up the response times of AI agents rather than raw model training throughput. In an Artificial Analysis benchmark running the open source Gemma 4 31B model, the chip delivered 3,400 output tokens per second on 100,000 token long context workloads, which Nvidia says is four times faster than the nearest competing platform on that same specific task.

The underlying problem Groq 3 LPX targets is fairly specific and increasingly important as agentic AI systems scale up. Unlike a single chatbot response, an AI agent working through a complex task typically generates massive volumes of tokens across hundreds or even thousands of individual inference steps as it reasons, calls tools, writes and executes code, and iterates repeatedly on a problem before actually finishing. Nvidia describes Groq 3 LPX as specifically targeting the decode phase of inference, the part of the process that determines how quickly individual tokens actually get generated once a model starts producing output, packaging 256 individual LPU chips into each full rack scale deployment.

The chip itself traces back to Nvidia's roughly 20 billion dollar acquisition of AI chip startup Groq's assets back in December, reportedly the largest deal the company has ever closed, and Groq 3 LPX represents the first major product to publicly emerge from that acquisition reaching full production. Nebius Group has already signed on as the first cloud provider committing to deploy the new chip commercially, running it through what Nvidia calls the Nebius Token Factory, and the company frames the entire release as making every individual step of an agent's reasoning loop feel instantaneous through the exact same developer API teams are already using today, without requiring any kind of separate migration to a completely new stack.

Jensen Huang specifically framed inference itself as the actual growth engine of the current AI industry moving forward, a notably different emphasis than the training focused framing that dominated most of Nvidia's messaging over the past several years. That shift in public framing lines up with a broader industry narrative that's been building for a while now, that as more companies actually move from experimenting with AI models toward deploying agents doing real, sustained, multi step work in production, the practical bottleneck increasingly sits on the inference and response speed side of the equation rather than purely on how large or capable the underlying model itself happens to be

Nvidia's dedicated inference accelerator Groq 3 LPX enters full production to supercharge AI agents - SiliconANGLEsiliconangle.com

[Slate Router]

3,400 tokens per second sounds impressive as a headline number, but I'd want to see how that actual figure holds up specifically under realistic mixed agent workloads rather than one clean, controlled benchmark scenario. Real world agent tasks involve a lot more variability, tool calling latency, and unpredictable branching than a single standardized long context benchmark test can really capture accurately

[NoLimitsOscar42]

20 billion dollars for Groq's assets is a truly enormous acquisition price, and seeing an actual concrete shipping product emerge this quickly from that specific deal is at least some reassurance that the money wasn't purely just an aggressive, defensive move to eliminate a promising rival from the competitive landscape entirely