Which Providers Can Size GB300 NVL72 for Your Workload Before You Reserve It?

The provider to pick is the one that will put the sizing in writing and show the math. A GB300 NVL72 sizing you can take into a reservation should state its input assumptions, a memory budget per GPU, whether your parallel layout fits inside one 72-GPU NVLink domain, the resulting GPU and rack count, how that count moves when the inputs change, and the power and delivery assumptions behind it. The second test is whether the provider can give you GPUs today to measure those inputs, because a sizing built on your own measured numbers holds up far better than one built on spec sheets. At Nebius, GB300 NVL72 is available "through the sales team only," and Together AI rents it as reserved capacity "from a single node up to a full rack or multiple racks," so the sizing conversation happens before the quote.

At GMI Cloud, our NVIDIA Blackwell lineup, including GB300 NVL72, is on pre-order and coming soon, and NVIDIA H100 and H200 are available now. That combination is the practical path: measure the inputs on H100 or H200 now, then bring them to our sales team as part of the GB300 NVL72 pre-order conversation. A GB300 NVL72 sizing answers two questions: how many GPUs and racks, and how to lay them out.

What a provider needs from you before it can size GB300 NVL72

A sizing is only as specific as its inputs. The resulting rack count should trace back to these inputs.

• Model architecture: parameter count, dense or mixture-of-experts, number of layers, attention layout (KV heads and head dimension), and expert count if MoE.
• Precision: training precision and optimizer, inference weight precision, and KV-cache precision.
• Sequence lengths: training sequence length, and for inference the typical and maximum context plus expected output length.
• Training targets: total tokens to train on and the calendar window to finish in.
• Inference targets: peak concurrent sequences or requests per second, and latency targets such as time to first token and per-token latency.
• Checkpoint and data plan: checkpoint interval, how many checkpoints to retain, and dataset size.
• Split between workloads: whether training and inference share the reservation or run as separate pools.

What a workload-based sizing calculates

NVIDIA lists GB300 NVL72 as 72 Blackwell Ultra GPUs and 36 Grace CPUs in one liquid-cooled rack, with 20 TB of GPU memory and 130 TB/s of NVLink bandwidth. NVIDIA's enterprise reference architecture breaks the rack into 18 compute trays, each with four GPUs and 1,152 GB of HBM, which works out to 288 GB per GPU. Those are the fixed numbers. A sizing is the work of fitting your workload against them.

Memory per GPU

For training, a common planning figure for mixed-precision training with the Adam optimizer is about 16 bytes of model and optimizer state per parameter, before activations. A 400B-parameter dense model needs roughly 6.4 TB for that state. Fully sharded across one rack's 72 GPUs, that is about 89 GB per GPU, leaving room under the 288 GB ceiling for activations and communication buffers. A 1T-parameter model needs about 16 TB, or roughly 222 GB per GPU on one rack, which leaves about 66 GB for everything else. At long sequence lengths that usually means activation recomputation, spreading optimizer state across more than one rack, or both. The sizing should show which of these it assumes.

For inference, the KV cache usually decides the count. Per token, it takes 2 × layers × KV heads × head dimension × bytes per element. As an illustration, a 70B-parameter dense model with 80 layers, 8 KV heads, and a head dimension of 128, with an FP8 KV cache, uses 163,840 bytes (160 KiB) per token, or about 21.5 GB for a single 128K-token (131,072-token) sequence. On a four-GPU replica with 1,152 GB of HBM and a 70 GB FP8 copy of the weights, that leaves room for roughly 50 such sequences before runtime overhead. Cut the context to 32K and the same replica holds about four times as many. This is why the context assumption belongs at the top of the sizing document.

Whether the parallel layout fits one 72-GPU NVLink domain

Inside a GB300 NVL72 rack, NVIDIA's reference architecture connects all 72 GPUs through nine NVLink switch trays, so every GPU reaches every other GPU over NVLink. Between racks, traffic runs over the scale-out network, which the same architecture builds on ConnectX-8 adapters rated at 800 Gb/s, four per compute tray. The sizing has to place each parallel group on the right side of that boundary. Tensor-parallel and expert-parallel groups, which communicate on every layer, should sit inside one NVLink domain. Pipeline stages and data-parallel replicas tolerate the slower path and can span racks. When a proposed layout puts a tensor-parallel or expert-parallel group across two racks, the document should say so and explain the expected cost. This is also where the sizing answers whether one NVLink domain is enough for your largest model instance.

GPU count and rack count from throughput and deadline

Memory sets the minimum GPUs per model instance. Throughput and time set the total. For training a dense Transformer, a standard first-order approximation of total compute is 6 × parameters × training tokens; for a mixture-of-experts model, the sizing should state which parameter count it plugged in. Divide that by the sustained FLOPS per GPU the provider assumes, then by the length of your training window in seconds, and you have a GPU count. The input that matters most is the sustained FLOPS figure: it should be a stated utilization applied to a dense peak, not the headline number. NVIDIA notes that the Tensor Core figures on its GB300 NVL72 specification are with sparsity unless otherwise noted. For inference, the count is peak concurrent sequences divided by sequences per replica, times GPUs per replica, plus headroom for failover. Convert GPUs to racks last, and show the remainder: a result of 80 GPUs is a decision about a second rack, not an automatic one.

Training and inference as separate pools

When one reservation serves both, the sizing should size each pool on its own terms and then state whether they share racks, run at different times, or stay fixed. A training job that wants every GPU in the domain and an inference fleet that must stay up cannot both assume the same 72 GPUs.

Storage and network

At GMI Cloud, GPU pricing covers compute, and networking and storage are provisioned separately based on your workload requirements. The sizing therefore has to include them: dataset and checkpoint capacity, the write bandwidth needed to save a checkpoint within an acceptable pause, and cross-rack bandwidth for the parallel layout above. NVIDIA's reference architecture gives each compute tray four E1.S NVMe drives typically used as a fast local cache, which helps with staging but does not replace shared storage for checkpoints.

Power and delivery as stated assumptions

NVIDIA's reference architecture puts a full GB300 NVL72 rack at up to 142 kW, liquid cooled. In a hosted reservation that load sits in the provider's facility, but a two-rack answer and a four-rack answer are different facility requests, and the sizing should state the power envelope it assumes. The delivery window is one more assumption the document should name, since a later date can change how much capacity the timeline requires.

The sizing document to ask for

Deliverable	What it should contain	What its absence tells you
Input assumptions	Model, precision, sequence lengths, token target, concurrency, latency targets, all as numbers	The count was not derived from your workload
Memory budget per GPU	Weights, optimizer state, activations, KV cache, and runtime overhead against 288 GB	The model may not fit as proposed
Parallel layout	Tensor, expert, pipeline, and data-parallel degrees, and which groups stay inside one NVLink domain	Cross-rack communication has not been checked
GPU and rack count	The formula, sustained-throughput assumption, and resulting count, with the remainder shown	The number may be rounded to inventory
Sensitivity analysis	Count at shorter and longer context, faster and slower throughput, and a tighter deadline	The plan has no margin for error
Storage and network	Capacity, checkpoint write bandwidth, and cross-rack bandwidth	Costs and bottlenecks will surface after signing
Power and delivery	Assumed rack power envelope and delivery window	The schedule behind the count is unknown

How to tell a workload sizing from an inventory sizing

A sizing calculated from the workload moves when the workload moves. The quickest check is to change one input and ask for the new answer. Double the context length, shorten the training window by a month, or switch the KV cache from FP8 to BF16. A workload-based sizing returns an updated memory budget and throughput calculation with the arithmetic shown. The rack count may still come out the same after rounding to whole racks, so judge the answer by whether the calculation moved, not by the final number alone.

A few other signals are worth checking. A result that lands exactly on a whole number of racks with no remainder discussed, a memory budget that is missing or lists only weights, a throughput figure with no stated utilization, and a document with no storage or network lines can mean the answer started from available capacity rather than from your model. Use those signals to inspect the calculation, rather than treating a round rack count alone as evidence of poor sizing.

What to measure on H100 or H200 while GB300 NVL72 is on pre-order

The inputs above are easier to trust when they come from your own runs. On H100 or H200 today, measure the quantities that feed the sizing rather than trying to predict Blackwell performance:

• Activation memory at your real training sequence length and micro-batch size.
• KV-cache bytes per token with your model, runtime, and cache precision, which confirms the formula against what the runtime actually allocates.
• Tokens per second per GPU for training and inference, so the provider can show how its GB300 throughput assumption relates to it.
• Checkpoint size and write time for your model and framework.
• Communication share of step time at your intended tensor-parallel degree within a single node.

On GMI Cloud, container workloads launch from templates that include vLLM, SGLang, and JupyterLab, or from your own images, and bare metal gives full OS access and custom drivers for framework-level profiling. Access to bare metal and containers is turned on per organization; request it through Contact Support in the console.

Sizing GB300 NVL72 with GMI Cloud

On our GPU page, GB300 NVL72 is described as best for long-context and high-capacity model training, built for next-generation multi-node clusters, and suited to large-scale reasoning and high-density AI systems. It is on pre-order with the rest of our Blackwell lineup and coming soon.

NVIDIA H100 and H200 are available now, from $2.60 and $3.20 per GPU-hour, for the measurements above. Commitment options are set with our sales team. For a GB300 NVL72 pre-order, bring your inputs and any H100 or H200 measurements to the conversation, and ask for the answer in the document format above so you can check each line before you reserve.

Start the sizing conversation

Contact Sales with your model architecture, precision, sequence lengths, training and inference targets, and timeline to discuss a GB300 NVL72 pre-order. Ask for the memory budget, parallel layout, GPU and rack count, and sensitivity in writing, and run the change-one-input check before you sign.