Cloud Run GPU Pricing Explained: How Serverless GPU Billing Actually Works

If you're trying to understand cloud run gpu pricing, the first thing to know is that it doesn't behave like a rate card for a rented GPU. Cloud Run is Google Cloud's serverless container platform, and its GPU option attaches an NVIDIA accelerator (currently L4-class hardware in supported regions) to a container that spins up on request and shuts down when traffic stops. That changes the billing math completely. Instead of paying a fixed hourly rate for a GPU you hold around the clock, you pay for the fraction of time your container is actually running, measured in fine-grained increments. This guide explains how serverless GPU pricing works across platforms like Cloud Run, which billing dimensions actually move your bill, and where serverless is the right model versus where it quietly costs more than a dedicated GPU.

What serverless GPU pricing charges you for

The defining trait of a serverless GPU platform is that billing tracks activity, not ownership. On a per-hour rental you pay for the GPU whether it's busy or idle. On a serverless model like Cloud Run, the meter runs only while an instance is provisioned to handle a request. When traffic drops to zero and the platform scales your service down, the GPU charge stops.

That sounds simple, but the real cost comes from several dimensions layered together. Google publishes Cloud Run GPU pricing per instance-second, and the exact rate depends on region and configuration, so treat any single number you see online as a snapshot and confirm on the official pricing page before you budget. What matters more than the headline figure is understanding which dimensions you're being charged on:

• Instance time: Billed per second (with a small minimum billable duration per request), covering the time an instance is active and serving.
• GPU attachment: The accelerator carries its own per-second charge on top of the CPU and memory allocated to the container.
• CPU and memory: Sized alongside the GPU, these bill on the same per-second basis while the instance runs.
• Requests and networking: Request counts and outbound data transfer can carry separate fees depending on the platform.
• Idle-but-warm capacity: If you configure a minimum number of instances to stay warm, you pay for that reserved capacity even when no requests arrive.

The last point is where teams get surprised. Scale to zero is the headline benefit, but the moment you set a minimum instance count above zero to avoid cold starts, you've reintroduced a fixed cost that behaves a lot like renting.

Per-second billing, cold starts, and scale to zero

Three mechanics define how serverless gpu pricing plays out in practice. Understanding all three together is the only way to predict your bill.

Per-second billing means your effective cost is proportional to how long each request keeps an instance alive. A model that responds in 300 milliseconds costs a tiny fraction of one that streams a long generation for 20 seconds. On a per-hour rental this distinction disappears because you've already paid for the hour. On serverless it's the whole game.

Scale to zero means that during quiet periods your GPU cost genuinely drops to nothing. For a service that sees traffic during business hours and goes silent overnight, this can remove more than half the compute cost compared to a GPU reserved full time. This is the single strongest argument for serverless.

Cold starts are the tax you pay for scaling to zero. When a request arrives and no warm instance exists, the platform has to schedule a container, attach a GPU, and often load model weights into GPU memory before it can respond. For large models that load step can take seconds. You're not billed for the wait in the same way you're billed for active serving, but your users feel the latency, and the workaround (keeping instances warm) costs money. So there's a direct trade between cold-start latency and idle cost, and where you land on that trade determines your real cloud run gpu pricing outcome.

A billing-dimension table for serverless GPU platforms

To compare serverless GPU offerings on equal footing, look past the per-second rate and map every dimension that can appear on the invoice. The table below is a generic framework you can apply to Cloud Run or any comparable serverless GPU service.

Billing dimension	How it's charged	When it dominates your bill
Active instance time	Per second while serving	Long-running or streaming requests
GPU attachment	Per second, on top of CPU/memory	Any GPU workload
CPU + memory	Per second, sized with the container	Heavy pre/post-processing
Request count	Per request or per million	Very high request volume, short calls
Data egress	Per GB out of the network	Large model outputs or media
Warm/min instances	Fixed per reserved instance	Latency-sensitive, always-on services
Cold start (indirect)	Latency, not a line item	Bursty traffic after idle gaps

The value of this framing is that it stops you from comparing two platforms on a single number. A service with a lower per-second GPU rate but higher egress fees can end up more expensive for a media-heavy workload, while a platform with pay-per-token billing sidesteps the instance-time question entirely.

What serverless GPU pricing fits, and what it doesn't

Serverless GPU billing rewards a specific traffic shape. It's the cheaper model when your load is variable, bursty, or intermittent, because you only pay during active seconds and nothing during the gaps. It fits early-stage products still finding demand, internal tools used a few hours a day, event-driven pipelines, and inference endpoints with uneven traffic.

It works against you in two situations. First, sustained high-utilization workloads: if a GPU would stay busy most of the day anyway, the per-second premium of serverless usually exceeds the flat rate of a dedicated GPU, and a reserved instance delivers a lower cost per unit of work. Second, latency-critical services that can't tolerate cold starts: keeping instances warm to avoid the cold-start penalty erodes the scale-to-zero savings that made serverless attractive in the first place.

Here's the practical decision path:

1. Map your traffic: Is it steady, bursty, or somewhere between? Measure the share of hours a GPU would actually be working.
2. Set your latency budget: Can your users tolerate a multi-second cold start, or do you need warm capacity?
3. Estimate active seconds: Multiply request volume by average serving duration to get billable GPU-seconds per day.
4. Compare against a reserved GPU: Convert both to delivered cost per thousand requests or per million tokens, not to the raw per-second or per-hour rate.

That last step is the one most comparisons skip. The honest metric for any inference workload is delivered cost per unit of work, and serverless only wins when your utilization is low enough that idle-hour savings outweigh the per-second premium.

Where GMI Cloud fits in the serverless picture

GMI Cloud is an AI-native inference cloud built for production AI, and its Model-as-a-Service (MaaS) is the serverless piece of that stack. MaaS runs 100+ models behind a serverless API, scales to zero so idle time costs nothing, and bills per token rather than per instance-second. Pay-per-token billing removes the cold-start-versus-warm-capacity trade from your cost model, because you're charged for tokens generated, not for how long a container stayed alive. GMI Cloud is a one-stop platform that lets you start on serverless MaaS and grow into dedicated endpoints, GPU containers, or bare metal on the same stack without rebuilding your application.

That matters because the serverless-versus-dedicated decision isn't permanent. GMI Cloud's Usage-Adaptive Pricing lets a workload start on demand, move to dedicated capacity as traffic stabilizes, and adopt commitment-based savings for sustained load, without forcing an early lock-in. When your inference outgrows the serverless sweet spot, you can shift to per-hour GPU rental on NVIDIA hardware (H100 from $2.00/GPU-hour, with H200, B200, and GB200 also available) rather than paying a per-second premium on capacity that now runs most of the day. You can review current rates on the GMI Cloud pricing page and deploy from the console.

The core takeaway for anyone evaluating serverless gpu billing: transparent, activity-based pricing only pays off if you match it to a workload that spends real time idle. GMI Cloud publishes both its per-token serverless rates and its per-GPU-hour dedicated rates so you can run that comparison honestly instead of guessing.

Match the billing model to your traffic

Cloud run gpu pricing, and serverless GPU pricing in general, isn't cheaper or more expensive by default. It's cheaper when your GPU would otherwise sit idle and more expensive when it wouldn't. Start by measuring how much of the day your workload actually needs a GPU, set your tolerance for cold-start latency, then convert every option to delivered cost per request or per token. Read serverless pricing that way and the per-second meter stops being a mystery and becomes a number you can plan around.