What's the performance-per-dollar of B200 versus H200 for production LLM serving, not just raw speed?

Teams choosing between B200 and H200 for production serving often compare raw tokens per second and pick the faster chip. That skips the number that actually matters at scale, which is how much useful throughput each dollar buys under real load, not at a synthetic peak. B200 lists at roughly 1.5 times the hourly rate of H200, so it wins on performance-per-dollar only when its sustained production throughput exceeds that ratio, and the honest comparison uses throughput measured under your real concurrency, batch size, and latency targets rather than a peak benchmark. This guide shows how to define performance-per-dollar for production serving, why steady-state load changes the answer, and how to measure it for your workload.

Performance-per-dollar is throughput under load divided by rate

Raw speed is a peak number; performance-per-dollar is an economics number. For production serving it is the sustained useful throughput a chip delivers under your actual traffic, divided by what that chip costs per hour. A GPU that is faster at peak but costs more per hour can still lose on performance-per-dollar if its advantage does not hold under load.

The rate ratio sets the bar the faster chip has to clear. In our current published listings, H200 lists at from $2.60 and B200 at from $4.00 per GPU-hour, so B200 costs about 1.54 times as much per hour, which means it must deliver more than roughly 1.54 times H200's sustained throughput on your model to win on performance-per-dollar. Below that margin, H200 buys more useful work per dollar; above it, B200 does. This is a smaller bar than B200 faces against H100, because H200 is already a strong memory-bandwidth chip, so the two are often closer on price-performance than a raw-speed comparison suggests.

Why production load changes the answer

Peak benchmarks and production serving are different regimes, and the gap between them is where price-performance decisions go wrong. Three production realities move the number:

• Sustained concurrency, not batch 1: Production serving runs many simultaneous requests. B200's larger memory can hold bigger batches and larger KV caches, which can widen its throughput lead exactly where H200 hits memory pressure.
• Latency targets: Performance-per-dollar only counts throughput delivered within your latency budget. A chip that posts high tokens per second but misses your p99 latency is not delivering usable throughput.
• Utilization: A more expensive chip that sits underutilized wastes its rate advantage. Price-performance assumes you can keep the GPU busy at production load.

The takeaway is that a peak tokens-per-second ratio can mislead in both directions. Under heavy concurrency on a memory-hungry model, B200's advantage may exceed its 1.54x rate premium and win on price-performance. On a light or latency-capped workload, H200 may deliver the same usable throughput for less, and buy more performance per dollar.

Reading B200 versus H200 price-performance

Use the frame below to judge which chip wins per dollar for your production profile. The outcome column is directional until you measure it.

Production condition	Favors on performance-per-dollar	Why
Sustained throughput above ~1.54x H200	B200	Clears the hourly rate premium
Sustained throughput below ~1.54x H200	H200	Cheaper useful work per dollar
High concurrency, large KV cache	B200	Memory headroom lifts batch throughput
Latency-capped or light load	H200	Premium unused; same usable throughput for less

The rule is compact: divide each chip's sustained, within-latency throughput by its hourly rate, and the higher number wins. Because the rate gap is about 1.54x, the decision is genuinely close for many workloads, which is exactly why raw speed alone is the wrong tiebreaker and measured price-performance is the right one.

Measuring B200 versus H200 price-performance on GMI

Since performance-per-dollar depends on your production load, the reliable step is measuring both chips under that load rather than trusting a peak figure. We are an AI-native GPU and inference cloud that publishes dedicated NVIDIA GPU list pricing and carries both H200 and B200, so you can compute price-performance directly.

We currently list H200 at from $2.60 and B200 at from $4.00 per GPU-hour under Limited Availability, so you can serve your model on each under production concurrency and latency targets, then divide sustained throughput by the hourly rate to see which wins on performance-per-dollar for your workload. Verify current rates and B200 availability on our GPU infrastructure (https://www.gmicloud.ai/en/gpus) and pricing (https://www.gmicloud.ai/en/pricing), since Blackwell pricing and stock move quickly. Benchmark at your real concurrency and within your latency budget, not at peak, because that is what production performance-per-dollar actually measures. When the workload is sustained production serving, our Prime Inference (https://www.gmicloud.ai/en/models/prime-inference) provides reserved GPU capacity with single-tenant isolation and warm serving, so measured throughput reflects steady traffic and high utilization rather than cold starts. Start a comparison in our console (https://console.gmicloud.ai).

Divide throughput by rate, under real load

If you choose between B200 and H200 on raw speed, you can overpay for a premium your production traffic never uses, or miss real savings on a memory-bound model at high concurrency. Measure it properly: run both chips at your production concurrency within your latency budget, divide sustained throughput by the hourly rate, and let performance-per-dollar decide. B200 must clear roughly 1.54x H200's throughput to win on cost, a margin it often exceeds under heavy concurrency and misses on light load, so the answer is your workload's to measure, not the spec sheet's to declare.