What GPU setup do I need to serve Kimi K2 on Blackwell (B200/GB200), and how does it compare to running it on H200?

Teams planning to serve Kimi K2 want a concrete GPU setup and a sense of whether Blackwell is worth it over H200. Because Kimi K2 is a large mixture-of-experts model, the setup is driven by how much memory its expert pool demands, and that is exactly where Blackwell's larger memory changes the picture against H200. Serving Kimi K2 requires enough GPU memory to hold its full expert pool plus the KV cache, so a Blackwell setup on B200 or GB200 can fit it on fewer or better-connected GPUs than H200, whose smaller per-GPU memory may force more sharding and more inter-GPU communication for the same model. This guide shows how to size the setup and how Blackwell compares to H200 for this specific model class.

Kimi K2's expert pool sets the memory demand

Kimi K2 is a large MoE model, which means it holds a big pool of expert weights that must all be resident in memory even though only a subset activates per token. As with any MoE model, the memory you provision is driven by the total parameter count across all experts, not the smaller active count, plus the KV cache for concurrency and context.

That total footprint is what your GPU setup has to hold. The GPU setup for Kimi K2 is sized by its full expert-pool footprint at your serving precision plus the KV cache, divided by the memory each GPU provides, so a chip with more memory per GPU holds more of the expert pool per card and reduces how far the model must be sharded. This is the point where B200 and GB200 diverge from H200: more memory per GPU, or a pooled rack domain, means fewer cards and less communication overhead to hold the same large expert pool. Size the footprint first, then the memory per GPU tells you how many cards each option needs.

How Blackwell compares to H200 for Kimi K2

The comparison turns on memory per GPU and interconnect, both of which favor Blackwell for a large MoE model. H200 is a capable memory-bandwidth chip, but its per-GPU memory is smaller than B200's, so a large expert pool that fits on fewer B200s may need more H200s, and more GPUs means more routing traffic crossing between them.

The three options serve different scales of the same model. B200 as an individual GPU holds more of the expert pool per card than H200, cutting the shard count for a given deployment. GB200 NVL72 goes further by pooling 72 GPUs into one NVLink domain, which suits a Kimi K2 deployment large enough that even a B200 cluster would span nodes, since the rack keeps the expert routing inside one high-bandwidth domain. H200 can serve Kimi K2, but for a large expert pool it tends to require more sharding and pays more inter-GPU communication cost, which is the overhead Blackwell reduces. Treat the exact GPU counts as a calculation from K2's footprint at your precision rather than a fixed spec, and confirm by measurement.

Matching the setup to your Kimi K2 deployment

Use the frame below to pick the setup by your scale and constraint.

Your Kimi K2 situation	Setup that fits	Why
Expert pool fits few B200s	B200 cluster	More memory per GPU, fewer shards
Pool spans many GPUs across nodes	GB200 NVL72	Pooled domain avoids inter-node cost
Serving on H200 today, hitting memory limits	Move to B200	Larger per-GPU memory cuts sharding
Moderate scale, fits H200 comfortably	H200 may suffice	No memory pressure to relieve

The pattern is consistent: Blackwell's advantage for Kimi K2 grows with how much the expert pool strains per-GPU memory. When K2 fits H200 comfortably, H200 can serve it; when the expert pool pushes memory limits or forces cross-node sharding, B200 or GB200 reduces both the card count and the communication overhead.

Serving Kimi K2 on GMI

Since the setup depends on K2's footprint at your precision, the practical step is using our platform, which carries H200, B200, and GB200 so you can size and compare directly. We are an AI-native GPU and inference cloud that publishes dedicated NVIDIA GPU list pricing and lists all three.

We currently list H200 at from $2.60, B200 at from $4.00 per GPU-hour under Limited Availability, and GB200 NVL72 at from $8.00 per GPU-hour Available Now, so you can measure Kimi K2's real footprint, size the GPU count on each option, and compare Blackwell against H200 for your deployment rather than assuming a setup. Verify current rates and availability on our GPU infrastructure (https://www.gmicloud.ai/en/gpus) and pricing (https://www.gmicloud.ai/en/pricing), since Blackwell pricing and stock move quickly. Benchmark at your production precision, concurrency, and context length, because those decide the expert-pool footprint and the KV cache that set your GPU count. When the workload is sustained Kimi K2 serving, our Prime Inference (https://www.gmicloud.ai/en/models/prime-inference) provides reserved GPU capacity with single-tenant isolation and warm, weights-preloaded serving, which keeps K2's expert weights resident for lower routing latency. Start sizing in our console (https://console.gmicloud.ai).

Size on the expert pool, compare on the memory

If you pick a GPU setup for Kimi K2 without sizing its expert pool, you may over-shard on H200 or provision Blackwell you do not need. Do it in order: measure K2's full expert-pool footprint at your precision, add the KV cache for your concurrency, divide by memory per GPU on each option, and compare the resulting counts and communication overhead. Blackwell's B200 and GB200 pull ahead of H200 exactly when the expert pool strains per-GPU memory or forces cross-node sharding, so size on the pool, compare on the memory, and let the model's real footprint pick the setup.