Cloud GPU Pricing Models Explained: On-Demand vs Reserved vs Serverless

Most teams pick a cloud GPU pricing model by accident. They start on whatever the provider's default is, the workload changes shape, and the billing model never catches up. The result is a bill that has almost nothing to do with the advertised hourly rate. The fix isn't hunting for a lower number. It's matching the pricing model to how your workload actually behaves. Cloud GPU pricing splits into three models: on-demand, reserved, and serverless. Each one is the cheapest option for a specific traffic pattern and the most expensive for the wrong one. This guide explains what each model charges for, when it wins, and how to decide which one your workload needs.

The three cloud GPU pricing models at a glance

Before choosing, it helps to see the three models against the same set of dimensions. The differences that matter are what you commit to, when you pay, and what happens when the GPU sits idle.

Model	You commit to	Idle cost	Best-fit traffic	Typical rate posture
On-demand	Nothing; start and stop anytime	You pay for every hour reserved	Spiky, unpredictable, short jobs	Highest per-hour
Reserved	A fixed term (weeks to a year)	You pay for the whole term	Steady, sustained, 24/7	Lowest per-hour
Serverless	Nothing; capacity follows traffic	Scales to zero, no idle charge	Intermittent, bursty inference	Per-request or per-token

The single question that decides among them is not "what's the lowest rate," it's "how full will the GPU be." A model that looks expensive per hour can be the cheapest per unit of work once idle time is counted, and a model that looks cheap per hour can be the most wasteful if your traffic can't keep the GPU busy.

On-demand: flexibility you pay a premium for

On-demand pricing lets you spin up a GPU, use it, and release it whenever you want, with no term commitment. You're billed by the hour (sometimes by the minute or second) for the time the instance is running. Because the provider carries the risk of that capacity sitting unsold, on-demand carries the highest per-hour rate of the three models.

On-demand is the right choice when your usage is genuinely unpredictable:

• Early development and experimentation, where you don't yet know how much compute you'll need.
• Short, one-off training runs or evaluations that finish in hours or days.
• Spiky workloads where you need capacity now and can't wait on a procurement cycle.
• Proof-of-concept work before a workload's shape is known.

The trap with on-demand is leaving instances running. Because there's no commitment, it's easy to forget a GPU that bills 24 hours a day while your job only ran for three. If you're using on-demand, treat instance lifecycle as a discipline: start, run, stop. When you find yourself running the same on-demand GPU continuously for weeks, that's the signal you've outgrown the model and should price out a reservation.

Reserved: the discount you earn by committing

Reserved pricing (also called committed-use or commitment-based) trades flexibility for a lower rate. You agree to pay for a set amount of capacity over a set term, and in exchange the per-hour rate drops well below on-demand. The provider gets predictable revenue; you get predictable cost.

The economics only work if you actually use what you reserve. Reserved capacity bills for the full term whether the GPU is busy or idle, so the model rewards high, steady utilization and punishes low utilization hard.

Here's the math that decides it. An H100 reserved full time over a 720-hour month runs a fixed cost regardless of use. At 90 percent utilization, that cost spreads across a lot of useful work and the per-job cost is low. At 25 percent utilization, three quarters of the spend buys idle hours, and you'd have been better off on-demand or serverless. Reserved wins when:

1. Your workload runs continuously or on a predictable schedule.
2. You can forecast capacity needs for the term with reasonable confidence.
3. Utilization will stay high enough that the discount beats the flexibility you give up.

Production training pipelines, batch systems that run every night, and inference services with steady baseline traffic are classic reserved workloads. If your load has a stable floor plus occasional spikes, a common pattern is to reserve the baseline and cover the spikes with on-demand or serverless.

Serverless: pay only for the work you run

Serverless GPU pricing bills per request or per token instead of per hour. There's no instance to manage and no idle charge, because capacity scales up with traffic and scales to zero when nobody is calling your endpoint. You pay for inference actually served, not for time a GPU was reserved.

This model is built for inference with uneven demand. Serverless is usually the cheapest option when:

• Traffic is bursty or intermittent, with quiet stretches between spikes.
• You're serving a product with unpredictable or growing usage.
• You want to avoid capacity planning entirely at the start.
• Cold-start latency is acceptable for your use case, or your traffic keeps endpoints warm.

The reason serverless wins on intermittent workloads is straightforward: a reserved or on-demand GPU idling overnight still bills, while a serverless endpoint that served no requests overnight bills nothing. Where serverless gets expensive is sustained, high-volume inference. Once an endpoint is busy nearly all the time, per-request pricing can exceed what a busy reserved GPU would cost for the same throughput. That crossover point is exactly where you'd migrate a maturing workload off serverless and onto dedicated capacity.

How to choose: start from the workload shape

The decision follows from one attribute of your workload: how consistently the GPU will be busy. Sketch your traffic over a typical week, then match it.

• Flat, high, and continuous: reserved. You're paying for hours you'll fill, so take the discount.
• Occasional and short: on-demand. You need flexibility more than a discount.
• Bursty inference with idle gaps: serverless. Let capacity and cost follow the traffic.
• Steady baseline plus spikes: reserve the baseline, burst on on-demand or serverless.

The reason teams overpay is that a workload rarely stays in one box. It starts spiky (on-demand), grows into steady inference (serverless), and eventually hits volume that justifies reservation. Every transition is a chance to overpay if your pricing model doesn't move with it. So the practical goal isn't picking one model forever, it's being able to shift models as the workload matures without re-architecting your stack.

Where GMI Cloud fits the three models

GMI Cloud is an AI-native inference cloud built for production AI, and its two-engine design maps directly onto these three cloud GPU pricing models. The Cluster Engine covers on-demand and reserved capacity through container service, bare metal with full root access and no hypervisor overhead, and managed multi-node clusters, so a reserved GPU delivers 100 percent of its advertised bandwidth. The Inference Engine covers serverless through Model-as-a-Service, running 100-plus models with scale-to-zero billing so idle inference costs nothing.

The part that addresses the overpay-on-transition problem is Usage-Adaptive Pricing: you can start serverless, move to dedicated endpoints as traffic stabilizes, and apply commitment-based savings for sustained load, without being locked in before you know your workload's shape. Region-Aware Pricing keeps the rate consistent and transparent across GMI's NA, Europe, and Asia-Pacific regions. Published per-GPU-hour rates make the reserved and on-demand math easy to run:

NVIDIA GPU	GMI Cloud rate	Availability
H100	from $2.00/GPU-hour	Available now
H200	from $2.60/GPU-hour	Limited availability
B200	from $4.00/GPU-hour	Available now
GB200 NVL72	from $8.00/GPU-hour	Available now

You can compare current rates on the GMI Cloud pricing page and start deploying from the console. Cloud GPU pricing is easiest to reason about when the on-demand, reserved, and serverless options live on one platform, because you can switch models as the workload changes instead of migrating providers.

Match the model before you compare the rate

Don't start a cloud GPU pricing decision by ranking hourly rates. Start by describing your workload: continuous or spiky, inference or training, predictable or growing. Continuous and predictable points to reserved, short and occasional points to on-demand, and bursty inference points to serverless. Pick the model that fits the shape, then compare rates within that model. Chosen that way, the pricing model does most of the cost optimization for you, and the invoice stops surprising you.