NVIDIA GPU Cloud Pricing by Model: What H100, H200, B200, and GB200 Actually Cost

If you're comparing NVIDIA gpu cloud pricing, the first thing to accept is that there's no single "GPU price." An H100 and a GB200 NVL72 can differ by 4x per hour, and the gap between them isn't arbitrary: it tracks memory capacity, memory bandwidth, and interconnect. The right question isn't which model is cheapest per hour, it's which model gives you the lowest cost for your specific workload once memory and bandwidth are counted. This guide breaks NVIDIA gpu cloud pricing down model by model, so you can match the card to the job instead of defaulting to the biggest number or the smallest one.

The four models you'll actually see priced

Most production AI runs today land on one of four NVIDIA data center GPUs, plus the next-generation Blackwell rack systems. Each sits at a different price point for a reason tied to hardware, not marketing.

• H100 (Hopper): The workhorse. 80GB of HBM3 and roughly 3.35 TB/s of memory bandwidth. It's the default for most training and inference today, and it's the reference point everything else gets compared against.
• H200 (Hopper, upgraded memory): Same compute architecture as H100, but 141GB of HBM3e and about 4.8 TB/s of bandwidth. That extra memory is the whole point: it fits larger models and longer context on a single card.
• B200 (Blackwell): 192GB of HBM3e and roughly 8 TB/s of bandwidth, with a large step up in raw compute over Hopper. Built for the largest single-GPU workloads and high-throughput inference.
• GB200 NVL72 (Grace Blackwell): Not a single card but a rack-scale system linking 72 Blackwell GPUs over NVLink into one coherent domain. Priced per GPU-hour, it targets frontier-scale training and very large model serving.

The pattern is consistent: as you move up the stack, you're mostly paying for more memory and faster bandwidth, not just more FLOPs. That matters because memory and bandwidth, more than peak compute, decide whether a big model even fits and how fast tokens come out.

NVIDIA gpu cloud pricing by model, side by side

Here's how the four models compare on price and the specs that drive it. Rates shown are GMI Cloud's published starting prices; always confirm current numbers on the provider's pricing page before you budget.

NVIDIA GPU	Price (from)	Memory	Bandwidth	Best-fit workload	Availability
H100	$2.00/GPU-hour	80GB HBM3	~3.35 TB/s	Mainstream training, steady inference	Available now
H200	$2.60/GPU-hour	141GB HBM3e	~4.8 TB/s	Large models, long-context inference, fewer GPUs per job	鈿狅笍 Limited availability
B200	$4.00/GPU-hour	192GB HBM3e	~8 TB/s	High-throughput inference, large single-GPU models	Available now
GB200 NVL72	$8.00/GPU-hour	192GB per GPU (72-GPU NVLink domain)	~8 TB/s per GPU	Frontier-scale training, very large model serving	Available now

Read this table as a starting point for a shortlist, not a ranking. The H100 line is the cheapest per hour, but for a model that spills out of 80GB, the H100 isn't in the running at all: you'd need multiple cards plus the networking to link them, and that combined cost can exceed a single H200 or B200 doing the same job.

How memory and bandwidth change cost per dollar

The reason NVIDIA gpu cloud pricing rises with each model is that the expensive parts, HBM memory and bandwidth, are exactly what remove bottlenecks for large models. Two effects decide your real cost per unit of work.

First, memory capacity decides how many GPUs a job needs. A 70B-parameter model in half precision needs roughly 140GB just for weights, before activations and KV cache. On an 80GB H100 you're forced to shard across two or more cards and pay for the interconnect between them. On a 141GB H200, the same model can fit on a single GPU, which cuts GPU count and removes cross-GPU communication overhead. So even though the H200 costs about 30 percent more per hour than the H100, a job that drops from two H100s to one H200 can end up cheaper overall.

Second, bandwidth decides token throughput. Inference on large language models is usually memory-bandwidth bound, not compute bound. The B200's roughly 8 TB/s moves weights and KV cache to the compute units far faster than the H100's 3.35 TB/s, so it can generate more tokens per second on the same model. At $4.00 per hour versus $2.00, the B200 costs 2x per hour, but if it delivers more than 2x the tokens on your workload, its cost per million tokens is lower. That reversal is the core insight of NVIDIA gpu cloud pricing: the cheapest card per hour is often not the cheapest per token.

To make it concrete, here's how to decide which model to price out:

1. Size the model in memory. Weights plus KV cache plus activation overhead. If it exceeds 80GB, the H100 needs multiple cards and the H200 or B200 becomes the honest comparison.
2. Classify the workload. Bandwidth-bound inference favors B200. Fitting a large model on one card favors H200. Steady, mid-size training or inference stays fine on H100.
3. Estimate throughput per card. Tokens per second at your batch size, scaled to an hour.
4. Divide all-in hourly cost by hourly throughput. That delivered cost per million tokens, not the sticker rate, is what you compare across models.

Which model fits which job

A quick mapping from workload to model, based on where each one earns its price:

• H100: Fine-tuning and training of models up to about 70B when sharded, plus production inference for models that fit comfortably in 80GB. Best cost per hour when the card stays busy.
• H200: The pick when a model or its context window won't fit in 80GB but does fit in 141GB. Consolidating a two-H100 job onto one H200 often lowers total cost and simplifies deployment. Note that H200 sits at Limited availability, so plan capacity ahead rather than assuming instant scale.
• B200: High-throughput inference serving where tokens per second per dollar decides your unit economics, and single-GPU jobs on very large models.
• GB200 NVL72: Training runs and serving that span many GPUs and depend on the NVLink domain to behave like one large accelerator. This is frontier territory, not a default choice for a first deployment.

Where the bandwidth actually reaches your job

There's a catch that changes real-world NVIDIA gpu cloud pricing: the bandwidth figures above are only available if the platform gives them to you. A virtualization layer (hypervisor) between your job and the GPU can quietly skim throughput, so you pay the B200 rate but receive less than the B200's advertised bandwidth. GMI Cloud is an AI-native inference cloud built for production AI, and its bare metal GPU option runs with root access and no hypervisor, so you receive 100 percent of the card's advertised bandwidth. On a bandwidth-bound workload, that's the difference between the cost-per-token math working out and quietly slipping.

GMI Cloud publishes transparent per-GPU-hour rates for all four models, and the same stack scales from serverless inference to bare metal clusters on NVIDIA hardware without a rebuild. That means you can prototype a model on serverless, then move the production version to a dedicated H200 or B200 once you know its memory and throughput profile, using the same platform. Pricing also flexes with maturity: start on demand, shift to dedicated capacity as traffic stabilizes, and apply commitment-based savings for sustained runs, without locking in early. You can review current per-model rates on the GMI Cloud pricing page or compare the hardware directly on the GPUs page, and start deploying from the console.

Price the model against the job, not the rate card

NVIDIA gpu cloud pricing looks like a ladder from H100 up to GB200, but the cheapest rung isn't the cheapest outcome. Size your model in memory first, classify whether it's bandwidth-bound or capacity-bound, then estimate throughput per card and divide the all-in hourly cost by it. Read that way, the H200 can undercut two H100s, the B200 can undercut the H100 per token, and the model you pick follows the workload instead of the sticker price.