SemiAnalysis published AgentX on August 24, an open source benchmark that replays real coding-agent sessions against production inference stacks rather than the fixed-length prompts most comparisons use.

On GLM 5.3 served through open source SGLang, SemiAnalysis reports Nvidia hardware reaching up to five times better cost efficiency than AMD at 150 output tokens per second per user. The firm goes further at that operating point. Competing accelerators handed over at zero cost would still produce a higher cost per token, it says, once hosting and power are counted.

Read that as a claim about one configuration, not a market-wide ratio. The published B200 comparison against MI355X on the same model shows the advantage moving with interactivity, near 57% at 108 tokens per second per user and 247% at 141.

SemiAnalysis argues that long-context multi-turn agentic sessions now dominate production inference traffic. If that holds, buyers who spent two years waiting for a second source to soften Nvidia's pricing are watching the gap widen where it counts.

What AgentX Actually Measures

AgentX is a trace replayer, not a prompt generator. SemiAnalysis disclosed that it ran a proxy intercepting Claude Code and Codex requests from its own staff, building a corpus of more than 8,000 sessions and 610 billion tokens. The version 1.0 public subset is 393 Claude Code sessions released under Apache 2.0.

Content is transformed into session-scoped chained hash blocks of 64 tokens each. This preserves the prefix relationships that drive cache reuse while removing the original prompts and code.

The shape of that traffic carries the argument. SemiAnalysis reports a median input length of 142,000 tokens against a median output of 444, with a median gap of 3.84 seconds between turns while the agent waits on a tool. The dataset records that 175 of the 393 sessions spawn at least one subagent.

A common fixed-sequence benchmark, such as 8,000 tokens of input followed by 1,000 tokens of output, exercises none of this.

Before I get to the competitive read, let me dissect what actually changes. An agent session tests whether reusable key-value state is still available, in high-bandwidth memory or an offload tier. It then tests whether the serving stack can route the next turn back to wherever that state lives.

Cache behavior is where the mechanism shows up in the numbers. On DeepSeek V4 at 384 concurrent sessions, SemiAnalysis measured a 91% hit rate in high-bandwidth memory plus another 1.36% from host memory, running vLLM on a B300 configuration with 3 terabytes of DRAM and eight-way expert parallel decode.

The same engine on B200, at concurrency 196 rather than 384, fell to 73% and leaned on host memory for another 20%. SemiAnalysis associates that heavier reliance on offload at higher concurrency with sharply worse latency as user counts scale.

Tokenization is a second source of the same advantage. Nvidia’s TensorRT-LLM added boundary-aware incremental tokenization so a conversation resending its full history no longer re-tokenizes all of it. On a Qwen 3.5 trace, it matched full tokenization across all 1,087 tested transitions, and reported mean processing time per turn fell from 185.1 milliseconds to 11.3.

A fixed-length benchmark cannot show that saving, since there is no prior turn to reuse.

Nvidia vs AMD On Agentic Workloads

AMD is not standing still, and AgentX credits it in specific places. Its ATOM engine beats a GB300 NVL72 rack running vLLM on price-performance across part of the Kimi K3 curve between 40 and 60 seconds of latency.

On DeepSeek V4, MI355X running SGLang matched B200 running vLLM until optimizations from Inferact and Nvidia merged upstream on August 21. B300 with vLLM and B200 with SGLang were already ahead before that date.

The decisive difference is where those wins live. SemiAnalysis says ATOM has little production adoption outside one advertising unit at Alibaba, notes that the main Qwen organization does not use it, and argues that upstream vLLM is therefore the more relevant comparison for customers.

On like-for-like open stacks, the picture separates further. Qwen 3.5 served through SGLang on both sides puts Nvidia more than 20 times ahead at 90 tokens per second per user. SemiAnalysis also notes that the vLLM feature-support matrix does not list AMD backends as supported for its context-parallel paths, the parallelism strategy long contexts lean on most.

What's more interesting is the fact that an increasing share of Nvidia's advantage now sits above the silicon. AgentX traces it to kernels, cache management, routing and scheduling, all of which move faster than a fabrication cycle.

What The Benchmark Does Not Settle

The August 21 flip is the strongest caution against reading any single result as durable. A ranking that inverts inside one week of upstream commits measures software velocity as much as hardware. SemiAnalysis has already scheduled an update within a month.

Four further constraints deserve attention from anyone reading the curves. The anonymized hash blocks are refilled with synthetic tokens at replay, which distorts speculative decoding acceptance, so SemiAnalysis pins acceptance length from SPEED-Bench rather than measuring it live. The traces also come from coding harnesses that inject heavy context, and a leaner harness produces a different input distribution.

AgentX also runs closed loop, so a faster configuration completes more requests and meets a slightly different workload mix, which matters most at low concurrency. SemiAnalysis separately names Meta, Microsoft, Oracle, OpenAI and several Chinese labs as supporters of the broader InferenceX initiative, and credits engineers at both Nvidia and AMD. None of that implies those companies endorse any particular comparison.

Google's TPUs are absent from the matrix so far. Nvidia's Rubin generation lands later in August, with TPUs and AMD's MI455X due later in 2026, so this reads Blackwell against CDNA 4 and nothing beyond it.

What Buyers Should Ask About Cache

None of this tells a buyer what to sign, so here is the diligence that follows. Buyers could ask a provider to disclose the prefix hit rate it sustains at production concurrency, and how much host memory backs each accelerator. SemiAnalysis says write-through DRAM offload works best when the host tier runs one and a half to three times the high-bandwidth cache capacity.

The second axis is session routing: whether a provider pins a session to the worker holding its prefix or load balances and forces a re-prefill every turn.

Customers can push for both answers as measured commitments rather than accept a tokens-per-second figure that says nothing about turn 40 of a session. That is a recommendation, not a benchmark finding. Platform teams can also replay their own traffic through the published harness.

For AMD, the path runs through software rather than a faster chip. If it upstreams more of the ATOM gains into vLLM and SGLang and matures its long-context parallelism and offload paths, AgentX suggests much of the current gap is addressable. The benchmark establishes nothing about how quickly that would happen.

In summary, the evidence AgentX puts on the table is compelling and convincing about the configurations it covers. On several high-interactivity points using mainstream open serving stacks, Nvidia has no close AMD competitor today. That is not true across every model, engine or point on the curve. For enterprises, neocloud operators and the teams maintaining these engines, a benchmark anyone can rerun against their own traffic beats a vendor slide.