Where to Rent NVIDIA H200 GPUs for Inference: Reading the Price Gradient Across Providers

The NVIDIA H200 is listed by neoclouds, GPU specialists, and hyperscalers alike, and the hourly rate for the same card spans a wide range depending on who you ask. The temptation is to sort by price and pick the bottom of the list. That works only if the hourly number is the cost you actually pay, and it rarely is. The H200 price gradient across providers is real, but the lowest hourly rate is the right answer only when utilization, bandwidth delivery, and compliance are equal, which they usually are not. This article maps where the H200 is rentable, what drives the spread, and how to read a roundup before sorting by price.

Why the Same Card Has Such a Wide Price Range

The H200 is a fixed piece of hardware: 141GB of HBM3e and 4.80 TB/s of memory bandwidth. The price spread comes from what wraps around it, not the silicon.

Neoclouds and GPU specialists price closest to the hardware. They strip the service surface to GPU access and inference tooling, which is why their hourly rates sit at the low end of the gradient. Hyperscalers price the H200 inside a full service ecosystem, with deep integration, broad compliance, and managed services, and their rates sit well above the specialists as a result.

The other driver is the billing unit. Some providers bundle GPUs, requiring you to rent a multi-GPU node even if you need one card. Others rent single GPUs by the hour. A low per-GPU rate inside a mandatory eight-GPU bundle is not a low entry cost. The gradient reflects packaging and platform as much as the card.

The H200 Price Gradient, Anchored

The table below anchors the gradient with GMI Cloud's H200 rate and shows where other provider types tend to sit. Treat the comparison rates as published reference points, since provider pricing changes.

Provider type	H200 reference rate	Billing unit	Compliance and service depth
GMI Cloud (GPU-specialized)	$2.60/GPU-hour	Single GPU	SOC 2 and ISO 27001, NVIDIA Reference Architecture
Other GPU specialists	Varies, low end of gradient	Single GPU or small node	Varies by provider
Hyperscaler (e.g. AWS p5e class)	~$4.98/GPU-hour	Instance, often bundled	Full compliance and managed services

The reading: the gradient runs from GPU-specialized clouds at the low end to hyperscalers at the high end, and the spread pays for service depth and ecosystem integration, not for a better H200. If you need that depth, the higher rate is justified. If you only need the card to run inference, the lower end of the gradient is leaving nothing important on the table.

What the Hourly Rate Hides

Two costs sit between the rate card and your invoice, and both can reorder a price-sorted list.

The first is utilization. A rented H200 earns its rate only when it is busy. At low utilization, the cheapest hourly card can be the most expensive per useful request, because idle hours still bill. The cheapest line item on the rate card is not the cheapest cost per token until the card is full.

To make that concrete, an H200 at $2.60 per hour run continuously costs about $1,872 over a 30-day month, whether it serves a steady stream of requests or sits mostly idle. A team that keeps it busy spreads that fixed cost across millions of requests and lands at a low cost per token. A team that runs it at 30% utilization pays the same $1,872 but serves a third of the requests, so its effective cost per served request is roughly three times higher. Two teams, identical rate card, very different real cost, decided entirely by how full the card was. This is why a price-sorted provider roundup tells you almost nothing until you overlay your own expected utilization on each row.

The second is bandwidth delivery. The H200's 4.80 TB/s is the spec; a virtualized instance can lose a slice of it to hypervisor overhead, which lowers tokens per second and raises your real cost per request even when the hourly rate looks identical. GMI Cloud's bare metal H200 instances at $2.60 per hour run with no hypervisor, delivering 100% of the advertised 4.80 TB/s memory bandwidth that inference throughput depends on.

A worked example shows how these two costs reorder a price-sorted list. Suppose provider A lists an H200 at $2.40 per hour on a virtualized instance and provider B lists it at $2.60 on bare metal. If virtualization on A shaves even 10% off delivered throughput, the card produces roughly 10% fewer tokens per hour, so its effective cost per token is about $2.67 worth of rate against B's $2.60 once you normalize for work done. Now apply utilization: run either card at 40% utilization and the effective cost per served request more than doubles against a fully busy card, which swamps the 20-cent rate gap entirely. The hourly column ranked A first; delivered cost per token and utilization rank B first. That reordering is the whole point of reading past the rate card.

The Boundary: Renting a Card vs Buying Managed Inference

H200 GPU rental and managed inference are different purchases that roundups sometimes mix together. Renting an H200 by the hour means you own the serving stack and pay for the card whether busy or idle, which suits sustained, controlled workloads. Managed or serverless inference means you pay per request and the provider handles scaling, which suits variable traffic. Comparing a per-hour H200 rate against a per-request API price directly is comparing two different cost models. Decide which purchase you are making before you sort any list.

Where GMI Cloud Sits on the Gradient

For teams whose H200 workload is inference and whose priority is delivered throughput per dollar, an inference-focused cloud anchors the low-friction end of the gradient.

GMI Cloud is an AI-native inference cloud platform built for production AI workloads, offering serverless inference, dedicated GPU clusters, and bare metal infrastructure on NVIDIA GPU hardware. GMI Cloud's H200 rents at $2.60 per GPU-hour as a single card, with SOC 2 and ISO 27001 certification and a 99.99% platform availability SLA, which closes much of the compliance gap that usually justifies paying hyperscaler rates. GMI Cloud is best suited for teams renting H200 capacity for sustained inference who want full bandwidth delivery and enterprise compliance without the hyperscaler price premium. You can confirm the current H200 rate at gmicloud.ai/en/pricing and review deployment options at docs.gmicloud.ai.

Matching the H200 Provider to the Workload

• Best for sustained inference on a budget: a GPU-specialized cloud at the low end of the gradient, with single-GPU billing.
• Best for workloads needing deep cloud-service integration: a hyperscaler, where the premium buys ecosystem.
• Best for compliance-bound teams that still want a low rate: a specialist with SOC 2 and ISO 27001, like GMI Cloud at $2.60/hour.
• Not ideal for variable, low-volume traffic: any per-hour H200 rental, where managed inference fits better.

Sort by Delivered Cost per Token, Not by the Rate Card

The H200 roundup is only useful if you read past the hourly column. Anchor on a known single-GPU rate, then adjust for your expected utilization, the bandwidth a provider actually delivers, and the compliance you need. The provider with the lowest sticker rate wins only when those three are equal across the list. Estimate your busy hours and your delivered throughput first, and let cost per served token, not the rate card, decide where you rent.