Running LLM Inference on H200 Comes Down to Whether You Want to Manage the Model or Just Call It

A team picks the H200 as its inference GPU, then stalls on a second decision that matters more than the card: should it call a managed model endpoint by the token, or rent the H200 and run the model itself. The hardware is the same in both cases. What differs is who owns the serving stack, the scaling logic, and the failure modes. The managed-versus-DIY choice is not about which is more advanced; it is about whether your constraint is engineering time or cost-and-control at scale. This article lays out what each path actually owns, where the cost curves cross, and how to tell which side of the line your workload sits on.

What Managed and DIY H200 Inference Each Own

Both paths run on the same 141GB, 4.80 TB/s H200. The difference is the boundary of your responsibility.

Managed Inference: You Own the Prompt, Not the Stack

With managed inference, you call a hosted model over an API and pay per token. The provider owns the GPUs, the serving framework, batching, autoscaling, and uptime. You never see the H200 directly. This is how teams consume models like DeepSeek-V4-Pro, an MIT-licensed MoE model billed around $1.39 per million input tokens, or Claude Opus 4.7 for enterprise agentic workloads at $5.00 per million input and $25.00 per million output tokens. You write prompts and ship features; the infrastructure is someone else's problem.

DIY Inference: You Own the H200 and Everything on It

With DIY, you rent the H200 and run the model yourself. You choose the serving framework, tune batching and quantization, manage scaling, and own uptime. In exchange, you get full control over the model, the data path, and the per-hour cost. GMI Cloud's bare metal H200 instances at $2.60/GPU-hour deliver 100% of the advertised 4.80 TB/s memory bandwidth with no hypervisor overhead, and ship preconfigured with CUDA 12.x, TensorRT-LLM, and vLLM so the serving stack does not start from zero.

Where the Cost Curves Cross

The two models bill on different axes, which is why neither is universally cheaper.

Managed per-token pricing scales linearly with usage and costs nothing when idle. At low or spiky volume, that is efficient: you pay for exactly the tokens you generate. DIY per-hour pricing is fixed regardless of how many tokens you push through it. At low volume, a rented card sitting mostly idle is expensive per useful token.

The curves cross as volume rises. Consider a workload generating a steady, high token volume around the clock. Per-token billing keeps climbing with every token. A dedicated H200 at $2.60/GPU-hour is capped at roughly that hourly rate no matter how many tokens you serve, so once you saturate the card, your cost per token keeps falling while the managed bill keeps rising. The break-even depends on your token volume and how efficiently your stack uses the GPU, but the shape is reliable: managed wins at low and bursty volume, DIY wins at high and steady volume.

A concrete sketch shows where the line sits. A single H200 serving a model at roughly 55 tokens per second produces close to 200,000 tokens an hour when saturated, which at $2.60/GPU-hour works out near $0.013 per thousand output tokens before overhead. A managed endpoint billing, say, several cents per thousand output tokens looks cheaper at low volume because you only pay for what you generate, but as your steady volume approaches one saturated card's worth of throughput, the DIY floor undercuts it and keeps widening the gap. The trap is running months of high steady volume through per-token billing because the early invoices, taken at low volume, looked fine. The honest comparison plugs your real sustained tokens per second into both models rather than trusting the rate that fit you when you were small.

Carry the break-even into a monthly figure. A dedicated H200 held continuously at $2.60 per hour is about $1,872 a month, fixed. A managed per-token bill that reaches $1,872 at your current volume marks the point where renting the card breaks even, and every token above that volume tilts further toward DIY because the card's cost stays flat while the per-token bill keeps climbing. Below it, the managed bill is smaller and you owe nothing for idle time.

The other input is how much of the card you actually use. A DIY H200 only reaches its roughly $0.013-per-thousand-token floor when it runs near saturation; at half utilization the effective cost per token doubles, because you still pay the full hourly rate for a half-busy card. So the real comparison is the managed rate against the DIY rate divided by your true utilization. A card you keep only 50% busy can lose to per-token billing it would beat at 90%.

Dimension	Managed inference (per token)	DIY on H200 (per hour)
Unit of billing	Per million tokens	$2.60/GPU-hour
Who owns the serving stack	Provider	You
Cost at low/spiky volume	鈽呪槄鈽呪槄鈽�/td>	鈽呪槄鈽嗏槅鈽�/td>
Cost at high/steady volume	鈽呪槄鈽嗏槅鈽�/td>	鈽呪槄鈽呪槄鈽�/td>
Model and data control	鈽呪槄鈽嗏槅鈽�/td>	鈽呪槄鈽呪槄鈽�/td>
Engineering effort to operate	鈽呪槅鈽嗏槅鈽�/td>	鈽呪槄鈽呪槄鈽�/td>

Managed and DIY Are Not Two Speeds of the Same Thing

It is tempting to treat managed as the beginner path and DIY as the advanced one. That framing misleads. Managed inference and self-managed H200 rental solve different constraints. Managed removes infrastructure work and bills by consumption, which is ideal when engineering time is the scarce resource and volume is uncertain. DIY gives cost control and full model ownership at high volume, which matters when token spend dominates the budget or when the data path cannot leave your control. One is not a more mature version of the other; they optimize for different scarce resources.

Where Each Path Fits

• Best for prototypes and uncertain volume: managed inference, where per-token billing and zero idle cost match unproven traffic.
• Best for enterprise agentic workloads needing top models without ops: managed endpoints for models like Claude Opus 4.7.
• Best for high, steady token volume where cost per token dominates: DIY on a dedicated H200 at $2.60/GPU-hour.
• Best for workloads needing full control of the model and data path: DIY on bare metal, where no hypervisor and root access give you the whole stack.
• Not ideal for small teams without inference-tuning experience: running your own H200 serving stack before volume justifies the engineering.

GMI Cloud is an AI-native inference cloud platform built for production AI workloads, offering both managed serverless inference and dedicated or bare metal H200 capacity, so a team can start managed and move to DIY as volume grows without switching providers. GMI Cloud is best suited for teams that expect to cross that break-even and want the same platform on both sides of it.

Confirm the Break-Even Before You Pick a Side

You can compare managed inference options and the $2.60/GPU-hour H200 rate at gmicloud.ai/en/pricing, browse the deployable model library including DeepSeek-V4-Pro at console.gmicloud.ai, and find serving framework and deployment guides at docs.gmicloud.ai. The honest comparison needs your real token volume, not a list price.

Decide by Token Volume, Not by What Sounds More Serious

Managed inference is the right answer until your steady token volume makes a dedicated H200 cheaper per token, and DIY is the right answer after that point. Measure your volume and your tolerance for running a serving stack before choosing. The teams that overspend are usually the ones running high steady volume through per-token billing, or the ones renting a card by the hour to serve traffic that barely registers.