NVIDIA says its Vera Rubin platform raises agentic throughput per dollar by up to 30x over Blackwell Ultra while HBM capacity per GPU rises only about 3%. The company published the claim on November 5, 2026, arguing that fitting more memory on a chip is not the answer to tight supply and rising prices.

Instead, NVIDIA describes what it calls full-stack codesign: GPUs, CPUs, networking, storage and software designed together so each byte of memory does more work. The figures come from NVIDIA’s own blog and a benchmark it cites, SemiAnalysis AgentX, so treat them as the maker’s claims.

Why is memory the bottleneck for AI servers?

Larger models, longer contexts and agent workloads are putting more pressure on GPU memory, according to NVIDIA. HBM, the high-bandwidth memory next to the GPU, is the fastest and most expensive tier, and model weights and KV cache compete for it.

KV cache holds the context a model needs for each active request. NVIDIA says agentic sessions carry earlier context across repeated model and tool calls, in some cases up to 3 million tokens, so working memory keeps growing. Retrieval-augmented generation adds yet more context.

NVIDIA also points to a limit it calls the memory wall. More HBM can hold more users’ context, but all those users share the same compute and bandwidth. Once bandwidth cannot feed the compute quickly enough, extra capacity sits idle while still adding cost and power draw.

What is NVIDIA doing about it?

NVIDIA lists three techniques, each aimed at a different source of waste:

  • Smaller weights. The NVFP4 format needs less space than FP8, the lowest precision the Hopper platform supports. Inside an NVL72 system, weights can be spread across up to 72 GPUs, leaving each GPU more HBM for KV cache. NVIDIA names several recent models released with NVFP4 checkpoints, including its own Nemotron 3.5 Super VL.
  • Smaller, movable KV cache. NVFP4 applied to KV cache shrinks it versus FP8. The Vera CPU shortens tool-call idle time, and its large LPDDR memory, linked to Rubin GPUs over NVLink-C2C, gives paused context a nearby place to wait. Dynamo software can also move idle context to CMX context memory storage, a tier NVIDIA says costs far less per bit than HBM.
  • Leaner retrieval data. Nemotron Retriever models produce smaller embeddings, and the cuVS library lets the search index live on CPU and storage, so the knowledge base never occupies HBM.

For context, a separate report on NVIDIA’s rack plans describes the Vera Rubin NVL72 as combining 72 Rubin GPUs and 36 Vera CPUs.

What do the numbers say?

NVIDIA’s results come from the SemiAnalysis AgentX DeepSeek V4 Pro workload:

  • GB300 NVL72 versus H200: up to 10x higher throughput per dollar at 100 tokens per second per user, with HBM capacity per GPU roughly doubling.
  • Vera Rubin NVL72 versus GB300 NVL72: up to 30x higher throughput per dollar at 125 tokens per second per user, with HBM capacity per GPU up about 3%.

NVIDIA says the metric operators care about is useful tokens delivered within a power budget while meeting response-time targets. It also says it was first to adopt HBM3e and HBM4 in an accelerated computing stack.

Prices

NVIDIA lists no prices in this post, and no shipping dates for Vera Rubin systems. The post does say memory supply is tightening and prices are rising, a trend it expects to continue in the near term. Check NVIDIA and your server supplier for quotes.

Who it suits

Data center and cloud buyers. The claim matters most if you pay for inference by the token. Throughput per dollar and per watt are the figures to ask vendors to confirm on your own workload.

AI developers. Models released with NVFP4 checkpoints and tools like Dynamo and cuVS are the practical parts you can look at today.

Everyday PC and phone buyers. This is rack-scale server hardware, so there is no direct product for consumers. The memory squeeze it describes is the same supply pressure the post says is pushing prices up.

Who should wait. Anyone planning a purchase on these numbers should wait for independent tests, since the figures are NVIDIA’s and rest on one benchmark workload.

What is still unknown?

NVIDIA does not say how the gains split between the memory techniques and the raw compute improvements, nor how results change on other workloads. Independent benchmarks and pricing will show how much of this reaches customers.

Next step: read NVIDIA’s technical deep dive and video explainer linked from the original post, and compare the AgentX results with your own model and context lengths.