H100
Hopper · baseline
- Memory
- 80 GB HBM3
- Inference perf
- 1.0× (baseline)
The workhorse. General LLM and multimodal inference. Where most production workloads start.
Ship faster. Scale further.One Prime Inference endpoint.
Prime Inference gives you reserved GPU capacity, tuned per model, with the engineering partnership to turn a working prototype into a production system.
H100 · H200 · Blackwell
NVIDIA-validated hardware
99.9% uptime
Production SLA
Three reasons to run inference here.
Performance, reach, and elasticity — built for real production traffic, sized for any scale.
2×
Top performance
Higher throughput on the models that matter
Per-model runtime tuning — kernel, scheduling, and routing — delivers up to 2× the sustained throughput of a generic stack on leading open-source models.
Directional benchmark. Actual performance varies by model and workload.
3
Global reach · low latency
Capacity where your users live
Single-tenant capacity across APAC, North America, and Europe. Region-pin for low TTFT, region-lock for residency — we fit the deployment to your market.
Industry-leading
Elastic by design
Scale with your traffic, not your forecast
Burst absorbs spikes, quiet hours drain to save. We've already closed the gap most platforms can't — and we're pushing provisioning faster still.
Lease the GPU. Own the throughput.
Reserved capacity rewards real production traffic — and per-model runtime tuning compounds the advantage over time.
Tuned runtimes
Per-model kernel, scheduling, and routing optimization — not a generic stack. Pick your model, we handle the engine.
Warm by default
Reserved GPUs stay warm with weights pre-loaded. Every call lands hot — no cold-start delay, no first-token jitter.
Single-tenant isolation
GPUs reserved only for your workload. No noisy neighbors, no contention under load, no shared-tier surprises.
Bring your own model
Any open-source, fine-tuned, or proprietary weights. Load from Hugging Face, S3, or your own storage — onto a runtime built to serve it well.
Optimized for the models you use
Our inference engineers continuously tune the runtimes behind the most-deployed open-source models — so when you pick one, the kernel work is already done.
Production-grade engines
vLLM, TensorRT-LLM, and SGLang pre-tuned per GPU class. Quantization configurable. Multi-GPU orchestration handled.
Deploy close to your users.
Region-pin endpoints for first-token latency, or region-lock them for data residency.
Asia-Pacific
Tokyo · Singapore · Taiwan — serving the fastest-growing AI markets.
North America
U.S. West, East, Central, and South — high-throughput production traffic.
Europe
EU partner data centers — residency and compliance-sensitive workloads.
Scale with your traffic.
Reserved capacity when you need guaranteed performance. Burst capacity when demand spikes. Drain when it doesn't. Pay only for what you actually use.
Burstable capacity
Spikes get absorbed automatically. No queueing, no manual scaling, no failed requests during demos or launches.
Drain to save
Quiet hours cost less. Capacity scales down gracefully without dropping in-flight calls.
One global pool
When your home region hits capacity, traffic borrows from the next-closest region to keep latency low and service continuous.
From idea to live endpoint, in four steps.
Pick a model, pick the hardware, deploy. The platform handles model loading, resource orchestration, and routing — so you go from selection to a live API in minutes.
1
Pick a model
Any open-source model, anything from Hugging Face, or upload your own weights.
2
Choose your setup
GPU type, GPU count per replica, replica count, and target region.
3
Deploy
Launch from console, CLI, or API. Endpoint is live in minutes, not days.
4
Operate & scale
Monitor latency and throughput. Burst when traffic spikes, drain when it doesn't.
Access the model you want.
One-click deployment for the leading open-source models — DeepSeek, Kimi, GLM, Llama, NVIDIA, and more. From frontier LLMs to vision, voice, and multimodal — pick a model, get a production endpoint.
DeepSeek V4
deepseek-ai
Kimi K2.6
moonshot-ai
GLM 5.1
zhipu-ai
Llama 4
meta-llama
Nemotron Omni
nvidia
Workloads where shared inference falls short.
Production traffic patterns where predictability, throughput, and engineering partnership turn a working prototype into a reliable product.
Agents & copilots
Coding agents & developer tools
Many short calls per task. First-call latency dominates user perception. Tool-use needs to be reliable, not just fast.
Stable endpoint per agent fleet · warm capacity · no cold-start during demos or launches.
Real-time voice
TTS, transcription, conversation
Voice doesn't tolerate variability. Persistent WebSocket sessions on warm capacity. Region-pinned for short round-trips.
Sub-second first-byte TTS · streaming endpoints · no shared-tier jitter.
High-throughput
RAG & chat at scale
Sustain millions of daily queries with hardware-bounded throughput. Consistent tail latency on long-context workloads.
Optimized KV-cache · bounded P95/P99 · no shared-pool contention.
Regulated
Private & compliant deployments
Isolated runtime, audit logs, zero-retention serving. Region-locked for finance, healthcare, public sector.
EU residency available · single-tenant isolation · enterprise SLAs.
Pick the right GPU for the job.
Hopper, Hopper-refresh, and Blackwell — choose by memory footprint, context length, or frontier performance need.
H200
Hopper refresh
- Memory
- 141 GB HBM3e
- Inference perf
- ~1.4× memory & bandwidth
Memory-heavy workloads — long context, large KV-cache, big batch sizes.
B200
Blackwell · frontier
- Memory
- 192 GB HBM3e
- Inference perf
- Up to ~2.5× on FP4
Frontier models, FP4 inference, max throughput. For performance-critical workloads.
FAQ
Frequently asked questions
Ready when you are.
Spin up a Prime Inference endpoint from the console — or contact sales about reserved capacity, custom tuning, and trial credits.