Who Can Manage GB300 NVL72 Infrastructure While You Run Your Own Models?

We recommend a provider that delivers GB300 NVL72 as managed Kubernetes on dedicated hardware, where the provider operates everything up to and including the Kubernetes control plane, the NVIDIA GPU software stack, and the NVLink fabric services, and your team owns the containers, schedulers, training code, and models on top. CoreWeave documents GB300 NVL72 instances delivered through CoreWeave Kubernetes Service (CKS), with a managed control plane and a GPU Operator that CoreWeave runs on your behalf.

At GMI Cloud, our Managed GPU Cluster service provides fully managed multi-node GPU clusters that GMI provisions and operates, with centralized cluster lifecycle management. It is in early access, with H200 nodes available now. GB300 NVL72 on GMI Cloud is in pre-order and coming soon, so the operations scope for a GB300 rack is set in the pre-order capacity proposal, alongside the capacity itself.

Whichever provider you shortlist, the recommendation holds only when the provider will sign a responsibility matrix that goes layer by layer through the rack: who operates the fabric, the node image, the driver and CUDA stack, and the Kubernetes platform, and who sets the upgrade window for each. A rack-scale system couples those layers tightly, so an unclear line at any one of them lands on your platform team during an outage or an upgrade.

Managed Kubernetes, bare metal with operations, or managed inference

Three delivery shapes can put GB300 NVL72 in front of a team that runs its own models. They differ in where the provider stops.

On GMI Cloud these map to Managed GPU Cluster, Bare Metal GPU, and Prime Inference. With Bare Metal GPU, GMI operates the hardware in its data centers and the host stack is yours. It gives full root access and hardware-level control, and our docs describe it as the option for full OS access, custom drivers, or persistent local NVMe. That is the right shape for teams that want to own the host stack, and the wrong one for teams that want the provider to own it.

One check is worth making before you commit to running models yourself. Prime Inference runs dedicated single-tenant GPUs with runtimes tuned per model, supports bring-your-own open-source, fine-tuned, or proprietary weights, and ships vLLM, TensorRT-LLM, and SGLang pre-tuned per GPU class with multi-GPU orchestration handled. When the only workload on the rack is serving and your team has no requirement to control the engine, that shape leaves your team less to operate than a managed cluster does. Teams that train, run custom engines, or schedule their own mix of jobs stay with managed Kubernetes.

Where the line falls on a GB300 NVL72 rack

NVIDIA describes GB300 NVL72 as a fully liquid-cooled, rack-scale system that integrates 72 Blackwell Ultra GPUs and 36 Arm-based Grace CPUs. NVIDIA's reference architecture breaks the rack into compute trays with four GPUs and two Grace CPUs each, plus nine NVSwitch trays that connect all 72 GPUs. That structure gives five layers, and each one needs a single owner.

1. Hardware and firmware. Power, liquid cooling, trays, and physical replacement sit with the provider in every managed shape; firmware for compute trays, NVSwitch trays, and network adapters belongs in the same bucket.
2. NVLink fabric services. The NVSwitch trays run their own switch OS, and NVIDIA's IMEX guide places the NVLink network control plane in Fabric Manager. On top of that, the IMEX service runs on each compute node so that jobs spanning trays can share GPU memory. How the rack is partitioned into NVLink domains is a decision made at this layer.
3. Node image, driver, CUDA, and NCCL. Grace hosts run Arm, so the node image is an arm64 build. The GPU driver, which sets the CUDA versions your containers can use, the network adapter drivers and firmware, the NCCL network plugin, and the host topology files all have to match the fabric firmware below them.
4. Kubernetes platform. The control plane, the node pools, and the NVIDIA components that make GPUs schedulable. NVIDIA's GPU Operator automates the driver, device plugin, and container toolkit, and NVIDIA's DRA Driver for GPUs adds ComputeDomains, which orchestrate IMEX so Kubernetes pods can use multi-node NVLink.
5. Your workloads. Container images, the CUDA runtime, NCCL, and framework versions inside them, schedulers and queues, training and serving code, model weights, and job-level metrics.

For a team that runs its own models, the clean line sits between layers 4 and 5. Everything from the Kubernetes API down is the provider's; everything you deploy through that API is yours.

The responsibility matrix to put in the contract

A matrix settles who acts when a layer fails, which a feature list does not. Ask each shortlisted provider to fill in the last two columns for your rack and term, and attach it to the order.

Two rows matter more than the rest. The node image row carries the tightest coupling, because the driver, network stack, and fabric firmware versions have to move together, and they set the CUDA and NCCL versions your containers can use. The Kubernetes row decides how much of your own tooling you can install, such as a batch scheduler, a queueing layer, or your own operators, without colliding with components the provider manages.

Who sets the upgrade window, layer by layer

Ownership of a layer and ownership of its upgrade window are separate questions. A provider can own the driver and still let your team decide when it changes. For a team that runs long training jobs or steady serving traffic, that second question often matters more.

Put four rules in writing:

• Notice and approval. For layers 2 through 4, the provider proposes an upgrade with a stated notice period, and your team approves the window. Routine firmware can follow the provider's schedule with notice, as long as it does not change the version matrix in layer 3.
• Coupled versions move together. A newer CUDA runtime can require a newer driver; newer drivers generally retain compatibility with applications built on older CUDA toolkits. A change to the NCCL network plugin or adapter firmware can alter collective performance across trays. Ask for one published version matrix that ties fabric firmware, driver, network stack, and the range of CUDA and NCCL versions your containers can use together, and for upgrades to land as a set.
• Staged rollout. A rack is one NVLink domain, but it can be split into partitions. Partitions separate compute trays; they do not separate the NVSwitch trays and other rack-level services the partitions share. For components installed per node, such as the node image and GPU driver on each compute tray, ask whether an upgrade can land on one partition first, so a canary job runs on the new stack before the full rack moves. For shared fabric components such as switch firmware, ask the provider to describe the maintenance scope component by component.
• Emergency exceptions. Security fixes may not wait for a scheduled window. Define which classes of fix can skip approval, the shortest notice the provider will give, and how running jobs are drained.

Draining is where the line gets tested in practice. Decide who cordons nodes before maintenance, how much warning your scheduler gets, and whether the provider waits for jobs to checkpoint or evicts at a fixed time.

Gray zones that blur the line

A few items do not fall cleanly into one layer. Settle each one before the rack arrives.

• The GPU Operator. On a managed cluster, the provider usually runs it. CoreWeave's documentation is explicit that customers should not install the GPU Operator on CKS clusters, because it conflicts with the platform-managed deployment. The same documentation advises against interacting with ComputeDomain resources directly, because doing so conflicts with CoreWeave's configuration. Confirm the same points with any provider, and confirm who configures the DRA driver and ComputeDomains.
• Privileged workloads. Profilers, custom network plugins, and node-level agents need host access that sits inside the provider's layers. List the DaemonSets and privileges your team needs so the provider can approve or refuse them up front.
• NCCL and network tuning. Environment variables inside your containers are yours; adapter settings and topology files on the host are the provider's. Write down who owns the topology information your jobs read.
• Metrics access. Your team needs GPU and fabric metrics to tell a slow job from a slow tray. Agree on which node, GPU, and NVLink metrics you can read, and how a hardware alert reaches your on-call.
• Feature maturity. Multi-node NVLink scheduling in Kubernetes is still maturing. CoreWeave's release notes currently label IMEX with Dynamic Resource Allocation on CKS as a limited availability feature. Ask every provider which rack-scale Kubernetes features are generally available and which are not.

Running your own models on GMI Cloud

Our Managed GPU Cluster service gives you Kubernetes orchestration on dedicated GPU worker nodes, with cluster-level control over your compute. You request a cluster in the console and choose the data center, the Kubernetes version, the worker node specification, the number of worker nodes, and the OS image, then review a summary with the estimated monthly cost before you submit. The steps are in the Managed GPU Clusters docs. Requests are not processed automatically: after you submit, contact support to activate the cluster, and GMI support reviews the request and provisions it.

Today, H200 nodes are available now for managed clusters, and the default OS image is Ubuntu 22.04 x86. That is a practical place to get layer 5 in order while a GB300 NVL72 pre-order moves forward: your container images, scheduler, and job templates run on the same Kubernetes model, as long as your team builds each image for both x86 and arm64, since the Grace hosts on the rack run Arm. The matrix above becomes a checklist for what changes on the rack.

For GB300 NVL72, bring the matrix into the pre-order conversation. The arm64 node image, Fabric Manager and IMEX operations, DRA driver and ComputeDomain support, the Kubernetes versions offered on the rack, and the upgrade-window rules are all items to settle in the capacity proposal. Our Managed GPU Cluster service also supports managed clusters across both GMI Cloud and BYOS environments; describe your environment in the same conversation if that applies. Networking and storage are provisioned separately based on workload requirements, so include your shared storage and checkpoint needs in the request.

FAQ

Can we bring our own scheduler to a managed GB300 NVL72 cluster? A scheduler or queueing layer that runs on Kubernetes belongs to your layer, but it often installs cluster-scoped resources. List the components and permissions it needs, and have the provider confirm them in the Kubernetes row of the matrix.

What if our team needs root on the hosts? Then the line sits lower, and bare metal with an operations service is the better fit. Full root access moves the node image, drivers, and fabric services onto your team, so the matrix shrinks on the provider's side.

Can we start a managed GB300 NVL72 cluster on GMI Cloud today? Not yet. GB300 NVL72 is in pre-order and coming soon, and Managed GPU Cluster is in early access with H200 nodes available now. The GB300 operations scope is agreed in the pre-order proposal.

Put the line in writing before the rack arrives

Send us your workload mix, the CUDA and NCCL versions your containers need, the tooling you install on Kubernetes, and how often you can accept an upgrade window. We'll work through the responsibility matrix with you as part of a GB300 NVL72 pre-order. Talk to our team.