Which GPU Providers Can Help You Choose B200, B300, or NVL72 Based on Your Workload?

The providers worth recommending can quote both eight-GPU HGX nodes and 72-GPU NVL72 racks, ask for your measured workload profile before they suggest a platform, and tie every recommendation to a number in that profile. A provider that sells only one platform class can still tell you whether your job runs on its hardware, but its quote gives you fewer options to weigh against each other. GMI Cloud covers both sides of the node-versus-rack choice: B200, GB200 NVL72, and GB300 NVL72 are open for pre-order and coming soon, and H100 and H200 are available now if your profile still has gaps. B300 is also on pre-order, and on GMI Cloud it is offered as hardware for Prime Inference endpoints on reserved GPUs. If your profile points to HGX B300 for training, get that quote from a provider that offers B300 training capacity and hold it to the same profile.

The choice usually comes down to four measurements you likely already have from H100 or H200 runs: the memory your model states and working set need at the target precision, the size of your largest tightly coupled parallel group, the share of step time spent on cross-node communication, and the number formats your code depends on. If the job fits in an eight-GPU node and its heaviest communication stays inside that node, choose between B200 and B300 on memory and precision. If the heaviest group needs more than eight GPUs, or cross-node traffic dominates the step, NVL72 is the platform to price, and the remaining question is GB200 or GB300.

The four signals in your workload profile

1. Memory fit at the target precision

Start with bytes per GPU, not parameter count. For training with mixed-precision Adam, model states take about 16 bytes per parameter before activations, as the ZeRO paper from Microsoft researchers lays out. For inference, count weights at the serving precision plus KV cache at your target context length and concurrency. Then compare the total with what each platform holds: NVIDIA's HGX page lists 1.4 TB of GPU memory per HGX B200 node and 2.1 TB per HGX B300 node, and NVIDIA lists 13.4 TB of HBM3E for a GB200 NVL72 rack and 20 TB for GB300 NVL72.

A second memory signal is often more telling: how much work your current runs spend just making the model fit. Activation recomputation, extra sharding, pipeline stages added only for capacity, and micro-batches kept small by memory limits all suggest that more memory per GPU will turn into throughput rather than idle headroom. A job with none of these gains little from extra memory.

2. The size of your largest high-bandwidth group

Find the parallel group that exchanges the most data every step: tensor parallelism in a dense model, expert parallelism in a mixture-of-experts model, or a context-parallel group for very long sequences. Inside an HGX B200 or HGX B300 node, eight GPUs share fifth-generation NVLink at 1.8 TB/s per GPU. Any group larger than eight crosses the scale-out network. NVL72 extends the NVLink domain to 72 GPUs in one liquid-cooled rack, with 130 TB/s of total NVLink bandwidth on both GB200 NVL72 and GB300 NVL72, according to NVIDIA.

If your heaviest group is eight GPUs or fewer, a node-based platform already keeps it on NVLink. If it spans 16, 32, or 64 GPUs, as wide expert-parallel layouts can, only the rack keeps that traffic off the network.

3. Cross-node communication share

Split the communication time in a representative step into intra-node and inter-node. A job that spends little time waiting on inter-node collectives gains little from a bigger NVLink domain. When inter-node all-reduce or all-to-all takes a large share of the step, there are two remedies: a larger NVLink domain, which is what NVL72 provides, or a faster scale-out network. NVIDIA's HGX specifications show HGX B300 doubling per-node networking bandwidth over HGX B200, from 0.8 TB/s to 1.6 TB/s, but that only helps if the provider's fabric actually runs at that rate.

4. Number formats

According to NVIDIA's HGX specifications, B300 raises dense FP4 throughput and memory over B200, keeps peak BF16 and FP8 throughput the same, and cuts FP64 and INT8 throughput sharply. An NVFP4 training recipe or FP4 serving plan favors Blackwell Ultra. Code that depends on FP64 or INT8 favors B200. A BF16 or FP8 job that already fits in memory sees no gain in peak BF16 or FP8 Tensor Core throughput from B300; what remains is B300's faster attention layers, which NVIDIA lists at 2x, and that matters mainly for long-sequence work.

One practical signal sits beside these four. NVL72 racks pair Blackwell GPUs with Arm-based Grace CPUs, so a stack with x86-only dependencies counts in favor of HGX nodes until those dependencies are resolved.

Decision table: signal to platform

Rows can disagree. A job might fit in one node but spend most of its step in cross-node data-parallel traffic, which points both toward B300's faster scale-out network and toward the rack. Resolve the tie with whichever signal costs the most step time in your profile, and ask each provider to show how its recommendation handles that signal.

Reading H100 and H200 measurements for Blackwell

Some numbers carry over directly: bytes per parameter at a given precision, KV cache per token, the size of each parallel group, and how much recomputation memory forces. Others are only directional. Blackwell's compute and its fifth-generation NVLink are both faster than Hopper's, so the communication share you measure on H100 or H200 can rise or fall on the new platform. Treat that share as a ranking of bottlenecks rather than a forecast of step time.

Precision is the other gap: a Hopper profile won't show a planned move to FP4, so recompute memory at the target format before comparing platforms. For a first cut at the platform class, a profile stated at the precision you plan to run is enough; benchmark on the target configuration before you commit on performance or cost.

Training versus inference changes which row matters

For training, the memory row covers model states and activations, and the group row usually decides between node and rack. For inference, memory means weights plus KV cache at your concurrency, and many deployments fit a replica inside one node and scale by adding replicas. The group signal applies to inference only when a single replica needs more than eight GPUs, for example a very large mixture-of-experts model served with wide expert parallelism.

When the same capacity will serve both, decide on the side that consumes the most GPU-hours, then confirm the other side still fits. Splitting the two is also a valid answer: training can land on one platform and serving on another, as long as the checkpoint format and precision work on both.

What makes a provider's recommendation workload-based

A recommendation built on your workload has a recognizable shape:

• It starts with your data. The provider asks for model size and target precision, memory per GPU including working set, the largest parallel group, the intra-node and inter-node communication shares, and the training-to-inference split before naming a platform.
• Each reason cites a signal. "B300, because your job spends a large share of its step recomputing activations to fit" is a workload reason. "B300, because it's newer" is not.
• It says what would change the answer. For example: if you move to an FP4 recipe, or your expert-parallel group grows past eight GPUs.
• Quotes use the same units. Price per GPU-hour at the same GPU count, with networking, storage, and delivery timing stated for each option, so a node plan and a rack plan can be compared line by line.
• It names what the provider doesn't offer. If one platform isn't in its catalog, it says so rather than steering around it.

A rack-only quote can omit an eight-GPU option that may fit the job. A node-only quote can leave a wide expert-parallel group spread across the network. Neither quote is dishonest, but neither covers the whole choice. One useful question for any provider: which platform would you recommend if you could offer all three?

Where GMI Cloud fits in this decision

On GMI Cloud GPU infrastructure, B200, GB200 NVL72, and GB300 NVL72 are open for pre-order and coming soon, so a node plan and a rack plan can be discussed in one conversation. Alongside that pre-order status, the GPU page lists pricing from $5.00 per GPU-hour for B200 and from $8.00 per GPU-hour for GB200 NVL72; GB300 NVL72 pricing comes through our sales team. The page positions GB300 NVL72 for long-context and high-capacity model training.

Our pricing is per GPU-hour, with committed usage plans discounted against on-demand rates. Networking and storage are provisioned separately based on your workload, so ask for them in every quote you compare. Bare metal servers give full root access and hardware-level control for training. Managed GPU Clusters, in early access, run Kubernetes on H200 or B200 nodes provisioned and operated by GMI; B200 nodes follow the Blackwell pre-order, and after you submit a cluster request, contact our support team to activate it. Our infrastructure is built on NVIDIA Reference Platform Cloud Architecture and validated designs. We deploy across the US, APAC, and EU, with RDMA-ready networking for high-throughput workloads.

For inference, Prime Inference runs tuned per-model runtimes on B200, B300, and GB200 NVL72, and B300 is the tier for workloads that need maximum memory per GPU when B200 isn't enough. Like the rest of our Blackwell lineup, B300 is on pre-order, and on GMI Cloud it is offered for Prime Inference.

If your profile is missing a number, H100 at $2.60 per GPU-hour and H200 at $3.20 per GPU-hour are available now for the measurements above.

FAQ

Once the rack is the answer, how do we choose between GB200 NVL72 and GB300 NVL72? Both give you a 72-GPU NVLink domain with 130 TB/s of NVLink bandwidth. GB300 NVL72 adds memory, 20 TB per rack against 13.4 TB on GB200 NVL72, along with Blackwell Ultra's FP4 gains. If your model states or KV cache fit in 13.4 TB with room for the working set, GB200 NVL72 covers the job; if memory pressure or long context is the constraint, GB300 NVL72 is the better fit. How many racks you need is a separate sizing question.

Bring your profile to a platform discussion

Contact Sales with your model size, target precision, memory per GPU, largest parallel group, communication shares, and training-to-inference split to discuss a B200, GB200 NVL72, or GB300 NVL72 pre-order. To complete the profile first, open the console to run on H100 or H200; bare metal and container access is enabled per organization, so request it through Contact Support in the console if it isn't active yet.