How much cheaper is B200 inference per token at scale compared to renting H100s, once you factor in throughput?

Teams scaling LLM inference often compare B200 and H100 on the hourly rate, see that B200 costs more, and stop there. That misses the point, because inference is billed by the hour but consumed by the token, and the chip that serves more tokens per hour can be cheaper per token even at a higher rate. B200 is cheaper per token than H100 only when its throughput advantage on your model outpaces its higher hourly rate, so the answer is set by tokens-per-dollar, not by the rate card. This guide gives you the crossover formula, the variables that decide it, and what to confirm before you assume the newer chip saves money.

Cost per token, not cost per hour, is the number that matters

The hourly rate is the input everyone sees; cost per token is the number that actually hits your bill at scale. In our current published listings, H100 is listed at from $2.00 per GPU-hour and B200 at from $4.00 per GPU-hour under Limited Availability, so B200 costs roughly twice as much per hour at list price. That is the wrong comparison to end on.

The right one divides rate by throughput. Cost per token equals the per-GPU-hour rate divided by the tokens served per hour, so B200 wins on cost per token only when its tokens-per-hour on your model is more than double H100's, since its hourly rate is about double. If B200 serves more than 2x the tokens per hour that H100 does for your workload, it is cheaper per token despite the higher rate. If it serves less than 2x, H100 is the cheaper choice for that model, period.

The variables that decide the crossover

Whether B200 clears that 2x throughput bar depends on your workload, not on a spec sheet. Four factors move it:

• Model size and memory fit: B200 carries more high-bandwidth memory than H100, so large models that spill or batch poorly on H100 can see outsized throughput gains, which is where the biggest per-token savings show up.
• Batch size and concurrency: Throughput advantages compound at high concurrency. A lightly loaded endpoint may never reach the batch sizes where B200 pulls ahead per token.
• Precision and kernels: Newer Blackwell precision formats and optimized kernels can lift tokens per second well beyond the raw hourly-rate ratio, but only if your serving stack actually uses them.
• Sequence length: Long-context workloads stress memory bandwidth, which tends to favor the newer chip; short prompts may not.

One boundary matters before you generalize any benchmark. Vendor throughput multipliers are usually stated for an equivalent configuration, not for your exact model, batch size, and sequence length, so treat a headline "faster inference" figure as a signal to test rather than a per-token guarantee. The only number that settles it is tokens per hour measured on your own model.

A tokens-per-dollar framework for B200 vs H100

Use the frame below to turn the two hourly rates into a per-token decision. Substitute your measured throughput for the illustrative multiples.

B200 throughput vs H100 (same model)	Cost-per-token outcome	Read
More than 2x tokens/hour	B200 cheaper per token	Higher rate paid back by throughput
Roughly 2x tokens/hour	Break-even	Choose on latency, memory, or availability
Less than 2x tokens/hour	H100 cheaper per token	Newer chip does not pay off here

The rule is compact: because B200 is about 2x the hourly rate of H100, it needs to beat 2x the throughput to be cheaper per token. Below that line, at scale, H100 remains the lower cost-per-token option no matter how new the silicon is. Above it, B200 saves money that grows with volume.

Measuring B200 vs H100 cost per token on GMI

Once you accept that throughput decides the winner, the practical step is measuring it rather than assuming it. We are an AI-native GPU and inference cloud that publishes dedicated NVIDIA GPU list pricing and carries both H100 and B200, which lets you benchmark the same model on each and compute tokens per dollar directly.

We currently list H100 at from $2.00 and B200 at from $4.00 per GPU-hour, so the break-even is a measured 2x throughput on your model, and the way to settle it is to run your own workload on both and divide rate by tokens per hour. Verify the current rates and B200's Limited Availability status on our GPU infrastructure (https://www.gmicloud.ai/en/gpus) and pricing (https://www.gmicloud.ai/en/pricing), since Blackwell pricing and stock move quickly. Benchmark at the batch size, precision, and sequence length you will actually run in production, not at a synthetic peak, because those inputs decide whether B200 clears the 2x bar. When the goal is sustained production serving rather than a one-off test, our Prime Inference (https://www.gmicloud.ai/en/models/prime-inference) provides reserved GPU capacity with single-tenant isolation and warm, weights-preloaded serving, so your measured cost per token reflects steady-state traffic rather than cold-start overhead. Start a comparison run in our console (https://console.gmicloud.ai).

Let tokens-per-dollar pick the chip

If you choose between B200 and H100 on the hourly rate, you will either overpay for H100 on a model where B200 clears the throughput bar, or overpay for B200 on one where it does not. The discipline is the same either way: measure tokens per hour for your model on both chips, divide each rate by that number, and let cost per token decide. In our current published materials, B200 is about 2x H100's hourly rate, so it saves money only above 2x throughput, and the only way to know which side of that line you are on is to run your own workload and check the invoice math, not the spec sheet.