GPU Cloud Pricing Explained: On-Demand, Reserved, and Bare Metal

GPU cloud pricing looks simple on a rate card, but the hourly number hides two separate decisions: How you commit to paying for capacity, and how much of the server you actually get. Understanding both is what separates a GPU bill that tracks useful work from one that pays for idle hardware.

• On-demand buys flexibility: You pay per GPU-hour with no commitment and can stop at any time, which suits experiments, bursty jobs, and workloads whose utilization you have not measured yet.

• Reserved pricing buys a lower rate: Committing to a term lowers the effective hourly price, but you pay for every hour of the term, so it only saves money when utilization stays above the break-even point set by the discount.

• Bare metal is a deployment model, not a billing term: It gives you the whole physical server with root access and no hypervisor, which can be bought on demand or reserved, and NVIDIA has measured virtualized training running 12% slower than bare metal.

• Published rates are only a starting point: On GMI Cloud, on-demand rates start at $2.60 per GPU-hour for H100, $3.20 for H200, $5.00 for B200, and $8.00 for GB200 as of October 2026, while committed plans are discounted and storage and networking are priced separately.

• Utilization decides which model wins: Cast AI’s 2026 analysis of production clusters found average enterprise GPU utilization of roughly 5%, which shows how easily reserved capacity turns into paid idle time. Matching the pricing model to measured usage is the single biggest lever on GPU cost.

Most teams compare GPU clouds by the hourly price of an H100 or H200. That number matters, but it rarely decides the final bill. Two teams paying the same hourly rate can end up with very different costs per training run or per million tokens, depending on whether they reserved capacity they did not use, paid for a hypervisor they did not need, or left storage and IP addresses running after a job ended.

The gap between paid and useful GPU time is large. Cast AI’s 2026 State of Kubernetes Optimization Report, which analyzed operational data from tens of thousands of production clusters, put average enterprise GPU utilization at roughly 5%, and described about 50% as a reasonable benchmark for well-optimized environments. The report excludes AI labs running dedicated training, but the lesson applies broadly: The pricing model you choose has to match how busy your GPUs really are.

This guide explains how on-demand, reserved, and bare metal GPU pricing work, when each one saves money, which costs sit outside the GPU rate, and how to compare quotes using current GMI Cloud pricing.

Two Questions Behind Every GPU Price

Every GPU cloud quote answers two separate questions. The first is the billing term: Do you pay as you go, or do you commit to a period in exchange for a lower rate? The second is the deployment model: Do you get a container on shared infrastructure, a virtual machine, a whole physical server, or a managed cluster? On-demand and reserved are answers to the first question. Bare metal is an answer to the second. That is why the same bare metal server can be rented on demand for a test and reserved for a year of training.

Keeping the two questions apart prevents a common mistake, which is comparing a reserved bare metal price from one provider against an on-demand virtual machine price from another. Hold the GPU, region, and term fixed, then compare.

On-Demand GPU Pricing

How On-Demand Billing Works

On-demand pricing charges a rate per GPU-hour while your instance is running, with no minimum term. You can start, scale, and stop whenever you like, and the bill follows usage, provided you actually release capacity when it is idle. Metering granularity matters for short jobs. Some providers bill by the second or minute, others round up to the hour, and that difference adds up quickly for bursty work.

On GMI Cloud, all GPU resources are dedicated, single-tenant instances priced per GPU-hour, with the rate depending on GPU type and deployment model. Pay-as-you-go containers have no expiration date and can be terminated at any time, and pay-as-you-go managed clusters are charged by the minute with no upfront cost, as GMI explains in its guide to comparing container, bare metal, and managed cluster quotes.

When On-Demand Is the Right Choice

On-demand pricing wins whenever the workload is short, bursty, or not yet measured. That covers prototyping, benchmarking a new model, one-off fine-tuning runs, evaluation jobs, and inference traffic whose daily pattern you have not seen yet. It is also the right way to start almost any new project, because a few weeks of on-demand usage turns a utilization guess into a measured number before you commit to anything.

On-Demand GPU Rates on GMI Cloud

The table below shows GMI Cloud’s published starting rates, taken from the GMI Cloud pricing page in October 2026. Rates vary by region and deployment model and can change with promotions, so treat them as starting points and confirm the live rate in the GMI Cloud console before you budget.

Reserved and Committed GPU Pricing

How Reservations Work

Reserved pricing, also called committed-use pricing, trades flexibility for a lower effective rate. You commit to a GPU type, quantity, and term, and the provider prices that capacity below its on-demand rate. In exchange, you pay for the full term whether the GPUs are busy or idle, and you usually cannot cancel early.

On GMI Cloud, containers can run on prepaid plans with renewal periods from one day to three years, and a prepaid container cannot be terminated before its rental period ends. Committed usage plans are discounted relative to on-demand, with volume and custom pricing handled by the sales team. Before you confirm an order, the console shows the list price, the discount, and the estimated monthly total, which makes it easy to compare a committed plan with the on-demand equivalent.

Short-Term Reservations Are Now Common

Reservations no longer have to mean multi-year contracts. Many providers now sell short reservations sized for a single training run. AWS, for example, lets customers reserve GPU capacity through EC2 Capacity Blocks for ML with a start date up to eight weeks ahead and durations from one day up to 182 days. GMI Cloud’s prepaid terms starting at one day follow the same logic: Reserve exactly the window you need, then release it.

The Utilization Break-Even

A reservation saves money only when you would otherwise have used the capacity for most of the term. The break-even point follows directly from the discount.

reserved is cheaper when expected utilization > (reserved rate ÷ on-demand rate)

With a 30% discount, the reserved rate is 70% of on-demand, so the reservation pays off only if you would have kept the GPUs running more than 70% of the time on demand. With a 50% discount, the break-even falls to 50%. A quick example makes the stakes concrete. Eight H100s running around the clock for a 730-hour month cost about $15,184 at GMI Cloud’s October 2026 on-demand rate of $2.60 per GPU-hour. If a committed plan priced that capacity 30% lower, it would cost about $10,629, which beats on-demand only if you would otherwise have run those eight GPUs for more than about 511 hours of the month.

The short Python helper below runs the same comparison for any GPU count, rate, discount, and utilization. The discount is a placeholder, because committed rates are quoted per customer, so replace it with the figure from your own quote.

# GMI Cloud on-demand "from" rate per GPU-hour, as of October 2026
ON_DEMAND_RATE = {"H100": 2.60, "H200": 3.20, "B200": 5.00, "GB200": 8.00}
HOURS_PER_MONTH = 730

def monthly_cost(gpu: str, num_gpus: int, utilization: float, discount: float) -> dict:
    """Compare on-demand (pay only for busy hours) with a reserved commitment."""
    full_month = ON_DEMAND_RATE[gpu] num_gpus HOURS_PER_MONTH
    on_demand = full_month utilization
    reserved = full_month (1 - discount)
    return {"on_demand": round(on_demand), "reserved": round(reserved),
            "break_even_utilization": 1 - discount}

# Placeholder discount; use the committed rate from your own quote
print(monthly_cost("H100", num_gpus=8, utilization=0.60, discount=0.30))

With eight H100s at 60% utilization and a 30% placeholder discount, the script shows about $9,110 on demand against about $10,629 reserved, so on-demand is still cheaper. At 85% utilization, the same reservation saves about $2,277 a month.

The Utilization Trap

The biggest risk with reserved capacity is buying it before you know your usage. Teams often reserve for peak demand out of fear of shortages, then run far below that peak most of the time, which is how fleet-wide utilization ends up near the 5% figure in Cast AI’s data. The fix is sequencing. Measure on demand first, reserve only the baseline you are confident you will use, and cover peaks with on-demand capacity on top.

Bare Metal GPU Pricing

What Bare Metal Includes

A bare metal GPU server gives you the entire physical machine, typically eight GPUs with their CPUs, memory, local storage, and network adapters, with full root and operating system access and no hypervisor in between. On GMI Cloud, Bare Metal GPU includes custom drivers, persistent local NVMe, and hardware-level control to stop, start, reboot, or power-cycle the machine. It runs in an isolated VPC on RDMA-ready infrastructure, and GMI’s H200 servers are built with eight 400G NDR InfiniBand compute adapters for multi-node training.

Why Bare Metal Can Cost Less per Unit of Work

Bare metal is priced per GPU-hour like other options, so its value shows up in throughput rather than in the rate itself. Removing the virtualization layer gives the workload direct access to GPUs, NVLink, and InfiniBand. In its Exemplar Cloud lessons, NVIDIA reported a case in which GB200 NVL72 FP8 pretraining ran 12% slower in a virtual machine than on bare metal. On a long training run, a 12% gap is effectively a 12% price difference, even at identical hourly rates. GMI Cloud has been recognized as an NVIDIA Exemplar Cloud for training on GB300 NVL72 systems, which means its infrastructure met NVIDIA’s performance thresholds against reference architecture.

What You Take On with Bare Metal

Bare metal shifts operational work to your team. You own the operating system, drivers, CUDA and framework versions, scheduling, scaling, and patching, and that engineering time is a real cost even though it never appears on an invoice. Bare metal also tends to come with longer terms and an approval step. On GMI Cloud, bare metal is enabled per organization and provisioned on demand once enabled, so check availability for your account early. GMI recommends asking for the billing model, minimum term, and metering granularity in writing so a bare metal quote can sit on the same row as container and cluster quotes.

Bare Metal, Containers, or Managed Clusters

Bare metal is not the only dedicated option. On GMI Cloud, Container Service offers Kubernetes-based GPU environments with fast startup for prototyping and elastic inference, Bare Metal GPU fits large-scale training and long-running, high-utilization workloads, and Managed GPU Cluster, currently in early access, provides multi-node Kubernetes clusters that GMI provisions and operates. All three run from the same console and are priced per GPU-hour, which keeps comparisons clean. The Cluster Engine documentation covers setup for each option.

Spot and Serverless: Two Other Pricing Models

Two more models complete the picture. Spot or preemptible pricing sells unused capacity at a deep discount, up to 90% off on-demand on AWS, in exchange for the risk of interruption, which makes it suitable only for workloads that checkpoint and restart cleanly. Serverless inference removes GPU-hours from the bill altogether and charges per token or per request instead. GMI Cloud’s inference engine offers both per-token serverless APIs and dedicated endpoints, and per-token prices carry their own promotions. In October 2026, for example, the console listed DeepSeek V4.1 Flash at 40% off, and a dedicated endpoint ran on one H200 at $4.50 per hour per replica.

Costs That Sit Outside the GPU Rate

The GPU-hour rate usually covers compute only. On GMI Cloud, storage and networking are priced separately when you allocate them, and several of these line items can outweigh a small difference in GPU rates. Persistent storage is one, since container-local data does not survive reconfiguration and workloads that need durable data need a volume. Public IP addresses are another. GMI Cloud bills Elastic IPs by the minute on pay as you go, and they keep billing after they are disassociated until you release them, even when the container they served has been terminated.

Service levels belong in the comparison too. GMI Cloud’s service level agreement sets monthly availability for container instances at no less than 99% in a single region and 99.9% across multiple regions, with service credits based on affected GPUs and downtime hours. Bare metal availability commitments and credits are set in each order. When you compare providers, record any line a quote leaves out as not quoted rather than zero, then divide the full total by the units of work you care about, such as tokens served or training steps completed.

How to Choose a GPU Pricing Model

A practical sequence works for most teams. Start on demand and measure real GPU utilization for a few weeks. Commit only the baseline you are sure to use, and keep peaks on demand. Move long-running, network-heavy training to bare metal once the workload is stable. Revisit the mix every quarter, because both your usage and the market rates will change.

Conclusion

GPU cloud pricing comes down to two choices made together: The billing term and the deployment model. On-demand pricing buys flexibility and suits anything you have not measured yet. Reserved pricing buys a lower rate, but only pays off when utilization stays above the break-even point set by the discount. Bare metal is a deployment choice that can be bought either way, and it often delivers the lowest cost per unit of work for steady, performance-sensitive training, at the price of more operational responsibility. Look past the hourly rate to storage, networking, service levels, and real utilization, and compare quotes on cost per finished job. GMI Cloud offers on-demand, committed, container, bare metal, and serverless options on one platform, so you can start small, measure, and move each workload to the pricing model that fits it.