How much GPU memory does a B200 have, and what model sizes can it hold without sharding across GPUs?

Teams sizing single-GPU deployments want two numbers: how much memory a B200 has, and how large a model that memory holds before sharding is required. The first is a spec; the second is a calculation from that spec at your precision. An NVIDIA B200 carries roughly 180GB of high-bandwidth memory, which at two bytes per parameter holds a dense model up to around 90 billion parameters at FP16, or roughly 180 billion at FP8, before the KV cache, so the single-GPU model size depends on precision as much as on the memory itself. This guide turns the B200's memory into practical model-size limits and shows where sharding across GPUs begins.

The B200's memory sets the single-GPU ceiling

A B200 provides on the order of 180GB of high-bandwidth memory, a large capacity that is what lets it hold sizable models on a single card. Confirm the exact figure against NVIDIA's official specification, since memory is a hard number that decides everything downstream, but roughly 180GB is the working figure for sizing.

That memory has to hold more than the weights. The model size a single B200 holds is its memory divided by the bytes per parameter at your precision, minus the room the KV cache needs, so the weight ceiling is a starting point that shrinks once you account for concurrency and context. At FP16, two bytes per parameter, roughly 180GB supports a dense model near 90 billion parameters for the weights alone. At FP8, one byte per parameter, that roughly doubles to near 180 billion. Lower precision extends it further. But the KV cache for active requests consumes memory on top of the weights, so the practical single-GPU model size is somewhat below the weight-only ceiling, and more so at high concurrency or long context.

Precision moves the limit as much as the model does

Because the model-size ceiling is memory divided by bytes per parameter, precision is a lever as powerful as the parameter count. Halving the precision roughly doubles the parameter count a B200 holds, which can move a model from needing multiple GPUs to fitting on one.

• FP16 or BF16: Two bytes per parameter. A B200 holds a dense model up to roughly 90 billion parameters in weights, before the KV cache.
• FP8: One byte per parameter. The weight ceiling roughly doubles to near 180 billion, if calibrated quantization holds output quality.
• FP4 or lower: Further reduces the footprint, extending the single-GPU ceiling more, when quality validation allows it.

The takeaway is that "what fits on one B200" is not a single number but a function of precision. A model that must shard at FP16 may fit one GPU at FP8, so the precision decision comes before the sharding decision. Always subtract realistic KV cache from the weight ceiling to get the model size you can actually serve.

Where sharding across GPUs begins

Use the frame below to see when a model stays on one B200 and when it must span several.

Model and precision	Fits one B200	Read
Up to ~90B dense at FP16	Yes, minus KV cache	Weights fit within ~180GB
Up to ~180B dense at FP8	Yes, if quality holds	Lower precision doubles the ceiling
Larger than the weight ceiling at your precision	No, shard across GPUs	Weights exceed one card's memory
Large MoE with big expert pool	Often no	Resident experts push past one GPU

The pattern is consistent: a model fits one B200 when its weights at your precision plus the KV cache stay within roughly 180GB, and it must shard once they exceed it. Dense models past the precision-dependent ceiling and large MoE models with big expert pools are the common cases that cross into multi-GPU territory, where a B200 cluster or a GB200 rack takes over.

Sizing single-GPU models on GMI

Since the single-GPU limit is a calculation from memory and precision, the practical step is using our platform, which lets you confirm the fit on real B200 hardware. We are an AI-native GPU and inference cloud that publishes dedicated NVIDIA GPU list pricing and carries B200.

We currently list B200 at from $4.00 per GPU-hour under Limited Availability, so you can deploy a model at your chosen precision, measure its real footprint including KV cache against the B200's memory, and confirm whether it fits one GPU or needs sharding. Verify the current rate and the exact B200 memory specification on our GPU infrastructure (https://www.gmicloud.ai/en/gpus) and pricing (https://www.gmicloud.ai/en/pricing), since Blackwell pricing and stock move quickly, and confirm memory against NVIDIA's official figure. Calculate the weight footprint at your precision, subtract the KV cache for your concurrency, and compare to the B200's memory before provisioning. When a model exceeds one GPU, GB200 NVL72 (https://www.gmicloud.ai/en/gpus) provides a pooled domain, and our Prime Inference (https://www.gmicloud.ai/en/models/prime-inference) offers reserved single-tenant serving on whichever setup your model size calls for. Start in our console (https://console.gmicloud.ai).

Size from memory and precision, then confirm

If you judge what fits a B200 by parameter count alone, you will miss that precision moves the limit by a factor of two or more and that the KV cache eats into the weight ceiling. Do the math: take the B200's roughly 180GB, divide by the bytes per parameter at your precision, subtract the KV cache for your concurrency, and compare to your model. Around 90 billion parameters fit at FP16 and roughly double at FP8 for dense weights, but the practical limit is lower once the cache is counted, so size from memory and precision together, confirm the exact memory against NVIDIA's spec, and shard only when the model genuinely exceeds one card.