Google Cloud GPU Cost Explained: Per-Hour Rates, Discounts, and Line Items

When you estimate google cloud gpu cost for an AI workload, the pricing page shows you a per-hour rate for an accelerator-attached VM. That rate is real, but it's one line on a multi-line invoice. Google Cloud bills the GPU, the host VM's vCPUs and memory, the disk the VM boots from, the snapshots you take, the network traffic in and out, and the load balancer in front of it, each on its own meter. Discounts like sustained-use and committed-use apply to the compute portion only, so the share of your bill made up of non-discountable items grows as the compute discount deepens. Understanding google cloud gpu cost means reading the full pricing model, not just the GPU hourly rate, because the line items outside compute often decide whether the bill matches your budget.

How GCP prices GPU instances: A2, A3, and C3

Google Cloud attaches GPUs to fixed machine families rather than selling standalone accelerators. Each family pins you to a specific NVIDIA GPU generation, host CPU and memory configuration, and price tier. The family you pick is the single biggest determinant of your per-hour cost.

• A2 machine family: Built around the NVIDIA A100 (40GB and 80GB variants). A2 is the established workhorse for mid-generation training and high-throughput inference, widely available across GCP regions and typically the most cost-effective option for teams that don't need H100-class throughput.
• A3 machine family: Built around the NVIDIA H100 80GB. A3 instances target large-scale training and high-density inference, with GPU-to-GPU interconnects designed for multi-node work. A3 carries a meaningfully higher per-hour rate than A2, reflecting the H100's performance leap over the A100.
• C3 machine family: GCP's compute-optimized family built on Intel Sapphire Rapids processors. C3 instances can be configured with NVIDIA L4 GPUs and are positioned for workloads where host CPU performance and memory bandwidth matter alongside GPU acceleration, such as real-time inference with heavy preprocessing.

GCP doesn't publish a single flat GPU price across these families. Each family has sub-variants that change the GPU count per VM (from 1 to 8), the host vCPU count, and the host memory. An A2 instance with one A100 has a different hourly rate than the same family configured with eight A100s, and the per-GPU effective rate can shift based on how host resources scale with accelerator count.

Instance family	NVIDIA GPU	Typical use case	GPU count options	Price range vs A2 baseline
A2	A100 40GB / 80GB	Training, batch inference	1 to 8	Baseline
A3	H100 80GB	Large training, high-throughput inference	1 to 8	Significantly higher
C3	L4	Cost-sensitive inference, video, light ML	1 to 4	Lower to comparable

GCP publishes per-hour rates for each configuration on its pricing page, and those rates vary by region. When you compare google cloud gpu cost across regions, you are comparing the same hardware at different local price points, not different capability.

Sustained-use discounts: what they covered and what replaced them

Google Cloud previously offered a Sustained Use Discount (SUD) that automatically lowered your effective compute rate the longer you ran a VM in a given month, up to a cap. SUDs required no commitment: you simply ran the VM and the discount applied automatically.

GCP has discontinued the automatic SUD program for GPU-attached instances. This is critical for anyone budgeting google cloud gpu cost based on older documentation. You can't count on automatic volume discounts just because your VM runs continuously. The discount structure now requires explicit action through commitment programs.

If you are planning based on a cost estimate that assumes sustained-use discounts will reduce your monthly GPU rate, that estimate is likely too low. The current model shifts the discount burden from automatic to commitment-based, meaning you trade flexibility for a lower rate only by signing up for a term.

Committed-use discounts: the current discount path

The primary discount mechanism on GCP today is the Committed Use Discount (CUD). A CUD is a contract: you agree to pay for a specific amount of GPU compute over a one-year or three-year term, and GCP applies a discount to the hourly rate in return.

1. Resource-based CUDs: You commit to a specific machine type or GPU configuration, such as a specific A3 H100 instance. The committed rate applies to that resource for the full term. This suits workloads with stable, predictable hardware needs.
2. Spend-based CUDs: You commit to a dollar amount of usage per hour across a broader category of resources, giving you flexibility to shift between instance types within the commitment. This suits teams whose GPU needs might change over the term but who know their overall spend level.
3. CUDs don't stack with Spot VMs: Spot VMs use spare capacity at a steep discount but carry preemption risk. You can't apply a CUD to spot capacity. You pick one discount path per instance.

The discount from a CUD applies to the compute line, which includes the GPU and the host VM resources. It doesn't apply to egress, persistent disk storage, snapshots, or networking add-ons. As your compute discount deepens, the non-discountable portion of your bill grows as a percentage of the total, and those line items become the main driver of cost variance.

Discount type	Mechanism	Applies to	Stacks with Spot?
Sustained Use (discontinued)	Automatic, volume-based	Compute only	N/A, no longer offered
Resource-based CUD	1-year or 3-year commitment to specific machine type	Compute only	No
Spend-based CUD	1-year or 3-year commitment to dollar amount	Compute across resource category	No
Spot VM	Spare capacity, preemptible	Compute only	N/A, standalone discount

Hidden line items: where the real cost lives

The GPU compute rate is the number everyone quotes, but the invoice line items that surprise teams live outside compute.

Network egress

Egress is the fee GCP charges for data leaving its network. For GPU workloads, egress hits when you move training datasets into a VM from another region or provider, sync checkpoints between regions, or serve inference outputs to users. GCP tiers egress pricing by destination: traffic to the internet, traffic between GCP regions, and traffic leaving GCP entirely for another provider each carry different per-gigabyte rates. The cross-provider tier is the most expensive.

For a workload that regularly moves large model weights or training datasets, egress can add 10 to 30 percent to the total bill. Because egress is billed per gigabyte, a single large data transfer can add a lump sum that dwarfs a full day of compute. Budgeting GPU cost without modeling data movement is where most teams get caught.

Persistent disk storage

Every GPU VM needs a boot disk, and most AI workloads add a data disk for datasets, model weights, or checkpoints. Persistent disks bill per gigabyte per month, continuously, whether the GPU is running or idle. A 2TB data disk bills every hour of every month, even if the GPU VM is shut down. Teams that provision large disks for convenience and forget to resize accumulate storage costs that persist long after the compute is gone.

Snapshots

Snapshots bill per gigabyte per month for storage. Teams that snapshot GPU VMs for backup, cloning, or disaster recovery accumulate snapshot storage costs that grow over time, especially if retention policies aren't enforced. Old snapshots from deleted VMs are a common source of creeping storage bills, because they persist independently of the compute resource that created them.

Load balancers and Cloud NAT

Production inference setups routing traffic to GPU VMs through Cloud Load Balancing add a fixed hourly fee per forwarding rule plus a per-gigabyte data processing charge. Cloud NAT adds per-GB processing on top of the egress fee when GPU VMs need outbound internet access without public IPs. Together they add a meaningful percentage to the monthly total.

How to read your GCP GPU bill without surprises

A few practices consistently close the gap between the rate card and the invoice.

1. Separate compute from non-compute on the billing export. Filter by the GPU VM's project label, then split GPU compute line items from network, disk, and load balancer charges. Sum the non-compute lines and divide by the compute total to get your hidden-cost ratio. Track it month over month.
2. Model egress before you model compute. Estimate your data movement volume (training dataset ingestion, checkpoint sync, inference output) and multiply by the relevant egress tier rate. If egress exceeds 20 percent of compute, caching, region consolidation, or a provider with simpler egress pricing may be worth evaluating.
3. Size persistent disks to actual usage, not to comfort. A disk that's 3x larger than needed bills 3x more than necessary every month. Resize down when data is pruned, and delete data disks when the VM is decommissioned.
4. Enforce snapshot retention. Set automatic deletion policies on snapshot schedules. Snapshots from deleted VMs should be pruned on a regular cadence, not left to accumulate indefinitely.
5. Match the CUD term to your real confidence level. A one-year CUD is the safer choice when traffic is growing. A three-year CUD gives a deeper discount but only makes sense for infrastructure you are confident will stay in production through the term.

Where a specialized AI cloud changes the pricing model

GCP's pricing structure is built for a broad customer base running everything from web servers to data pipelines to ML training, so GPU pricing carries overhead designed for the average workload, not for AI inference specifically.

GMI Cloud is an AI-native inference cloud built for production AI, and its pricing model is structured so the rate you see is the rate you plan against. An H100 starts at $2.00 per GPU-hour on bare metal with no hypervisor overhead, an H200 starts at $2.60 per GPU-hour, and a B200 starts at $4.00 per GPU-hour. There are no separate host VM charges to reconcile and no load balancer fees stacking on top of compute. Usage-Adaptive Pricing lets you move from on-demand to dedicated to committed capacity as your workload matures, without locking into a long-term commitment before you know your traffic pattern. It's an NVIDIA Reference Architecture provider with 30,000-plus GPUs deployed, 99.99 percent platform availability, and GPU regions across North America, Europe, and Asia-Pacific. You can review current rates on the GMI Cloud pricing page.

Compare the full bill, not the GPU rate

The biggest mistake teams make with google cloud gpu cost is treating the GPU hourly rate as the total cost. The lines outside compute, egress, persistent disk, snapshots, and networking add-ons, are where budgets break. Model the non-compute line items, pick the discount path that fits your confidence level, then compare providers on delivered cost per token or per job.