What's the cost per million tokens when serving LLMs on B200 versus H100, and does the higher hourly rate pay off?

Teams comparing B200 and H100 for LLM serving usually start from the hourly rate, see that B200 costs about twice as much, and assume H100 is the cheaper choice. That assumption skips the only number that matters for inference economics: the delivered cost per million tokens. B200 carries a higher hourly rate than H100, but whether it costs more per million tokens depends on throughput, so a chip that is twice the price per hour can still be cheaper per token if it processes more than twice the work. This guide shows how to calculate the per-token cost and when the higher B200 rate actually pays off.

Cost per million tokens is a throughput calculation, not a rate lookup

The hourly rate tells you the cost of an hour. What you serve LLMs to buy is tokens, so the honest metric is cost per million tokens, and converting between the two takes three inputs:

• All-in hourly cost: The per-GPU-hour rate plus a fair share of storage, egress, and networking. In our current published listings, H100 is from $2.00 and B200 is from $4.00 per GPU-hour, so B200 starts at roughly a 2x hourly premium.
• Throughput: Tokens per second the GPU sustains at your model, batch size, and precision, scaled up to an hourly figure.
• Utilization: The share of paid hours the GPU is actually serving, since idle hours inflate the per-token cost on either chip.

Divide all-in hourly cost by effective tokens per hour and you get the delivered cost per million tokens. This is the number that decides the comparison, and it explains why the cheaper card per hour is frequently not the cheaper card per token.

Why B200 can be cheaper per token despite the higher rate

B200 is a newer Blackwell-generation GPU with more memory bandwidth and higher throughput than Hopper-class H100, particularly at large batch sizes and lower-precision formats. When a workload keeps the chip busy at high batch, B200 can process meaningfully more tokens per second than H100. If that throughput gain exceeds the hourly price gap, the per-token cost on B200 lands below H100 even though the hourly rate is higher.

The rule is simple: B200 pays off per token when its throughput multiple over H100 is larger than its price multiple, and H100 stays cheaper per token when it is not. The break-even sits at the point where the throughput ratio equals the price ratio.

The boundary matters, because the throughput advantage is not uniform. B200 wins clearly on large, high-batch, bandwidth-bound serving where the newer architecture stretches its legs. H100 often stays cheaper per token on small models, low-batch or latency-capped serving, and workloads that cannot fill the bigger chip, where B200's extra capacity sits idle and you pay the premium without the throughput. Measure throughput on your own model and batch plan rather than assuming a generational multiple, because the same two chips can reverse ranking across workloads.

Reading the comparison for your workload

Use the frame below to reason about which chip is cheaper per token, then confirm with a measured throughput number on your stack.

Factor	H100 (from $2.00/hr)	B200 (from $4.00/hr)
Hourly rate	Lower baseline	About 2x H100
Best-fit serving	Small models, low batch, latency-capped	Large models, high batch, bandwidth-bound
Throughput headroom	Sufficient for Hopper-fit workloads	Higher ceiling at large batch
Cheaper per token when	Throughput ratio under about 2x	Throughput ratio above about 2x

One boundary to keep clear: hourly rate and cost per million tokens are different metrics, and comparing chips on the first tells you almost nothing about the second. A B200 at twice the hourly rate is cheaper per token whenever it clears more than twice the tokens per hour, and more expensive per token whenever it does not. The decision is a throughput measurement, not a rate comparison.

Getting a per-token cost you can verify on GMI

Once you know how to calculate delivered cost, the practical step is measuring throughput on real capacity. We are an AI-native inference cloud that publishes dedicated NVIDIA GPU list pricing and provides both H100 and B200 capacity, so you can benchmark the same model on both and compare cost per million tokens directly.

Verify the H100 from $2.00 and B200 from $4.00 per GPU-hour starting rates on our GPU infrastructure (https://www.gmicloud.ai/en/gpus) and pricing (https://www.gmicloud.ai/en/pricing), then run your own model at production batch size to get tokens per second on each. When the goal is sustained production serving rather than a one-off benchmark, our Prime Inference (https://www.gmicloud.ai/en/models/prime-inference) provides reserved, single-tenant capacity with warm, weights-preloaded serving, which keeps utilization high so the per-token math reflects real delivered cost rather than idle-inflated hours. Start in our console (https://console.gmicloud.ai) to spin up both chips and measure before committing to either.

Compare on delivered cost, not the hourly premium

If you rank B200 and H100 on the hourly rate, you will conclude H100 is cheaper and sometimes be wrong. Run the real calculation: divide all-in hourly cost by measured tokens per hour on your own model, and compare the delivered cost per million tokens. B200's higher rate pays off whenever its throughput advantage over H100 beats its roughly 2x price premium, which is common on large, high-batch serving and rare on small or latency-capped workloads. Measure both chips on your stack, then let cost per million tokens, not the rate card, make the call.