Which Providers Can Host Kimi K3 on Dedicated B200 or B300 GPUs and Tune It for You?

For Kimi K3, choosing between B200 and B300 means choosing a node shape: the engine teams' public K3 guidance puts the smallest deployment on one 8-GPU B300 node, while B200 needs 16 GPUs across two nodes. GMI Cloud Prime Inference hosts it on single-tenant GPUs with a runtime tuned per model, lists Kimi K3 with 1M+ context among its one-click models, and puts GMI engineers on the setup with you. Sizing for K3 starts at B300.

What to ask any provider on this shortlist

Four answers separate a plan from a pitch, whether the candidate is a model vendor's API, a hyperscaler's managed model service, a GPU cloud with a tuning team, or your own cluster:

1. Which K3 replica shape is committed on which GPU? Expect a node count and parallel layout per GPU type, not "Blackwell" in general.
2. Who does the tuning, and what is the acceptance test? Tuning should end in measured results on your prompts against acceptance criteria you set in advance.
3. What is delivered when? B200 and B300 timing should be stated with its pre-order or reservation status, plus what runs in the meantime.
4. What isolation and commercial terms apply? Single-tenant capacity, region placement, minimum contract, and how the SLA is set.

Renting bare GPUs, or self-hosting K3, leaves the engine, the parallel layout, and the acceptance testing with your team.

Why node shape decides B200 versus B300 for Kimi K3

Kimi K3 is a 2.8T-parameter Mixture-of-Experts model that activates 16 of 896 experts, about 104B parameters, per token, according to Moonshot's model card. Only a small share of the model computes on any one token, but in the engine recipes below every expert's weights stay resident in GPU memory, so total parameters, not active parameters, set the footprint. The K3 weights are native MXFP4, and the Hugging Face repository totals roughly 1.56 TB.

Engine documentation turns that footprint into node counts:

Memory per GPU is the difference. On Prime Inference, B300 is the GPU with maximum memory per GPU, built for trillion-parameter models, for when B200 is not enough. vLLM's K3 launch post describes the single B300 node as a tight fit, so the context lengths and concurrency you need belong in the sizing request from the start. The engine configurations are the starting reference; GPU count per replica is a setting you choose with GMI engineers when you create the deployment.

B200 or B300: how to choose

Choose B300 when you want one replica to be one node. In SGLang's unified reference configuration, the model-parallel communication of a replica stays inside a single 8-GPU node. Prime bills by the GPU-hour, so GPUs per replica is one factor in spend: the reference B300 shape uses 8 GPUs where the reference B200 shape uses 16. Which option costs less per token also depends on each GPU's hourly rate, how many replicas you run, and the throughput each reaches on the same workload, so compare quotes on one workload definition.

Choose B200 when its pre-order slot is earlier or your fleet is already standardized on it. SGLang's unified K3 recipe for B200 splits the two nodes into two pipeline stages, so inter-node communication is part of every replica and the parallel layout becomes a central tuning decision. The speculative decoding options also follow the layout, as the tuning table below shows.

Weigh the maturity of the engine recipes. SGLang's K3 cookbook marks two B300 1x8 configurations (Low-Latency and Balanced) as verified, so a B300 plan starts from documented, speed-verified configurations. The accuracy threshold from your own evaluation set still gates every tuning change.

When K3 should leave the per-token API

K3 is in GMI's model library and on the OpenAI-compatible API as moonshotai/Kimi-K3. That is the right place to start: serverless suits prototyping, low volume, and elastic traffic. The signals below say a dedicated replica is worth pricing.

If none of these signals shows up in your numbers, stay on the API. If two or three do, the sizing conversation is worth having, and the usage log from the API becomes the traffic profile it starts from.

Tuning Kimi K3 on Prime: GMI's runtime work and your inputs

Prime Inference includes:

• Per-model runtime tuning. Kernel, scheduling, and routing optimization for the model you choose, so the engine is tuned to that model.
• Runtimes matched to the GPU. Each GPU class gets its own pre-tuned build of vLLM, TensorRT-LLM, or SGLang; you set quantization as needed, and the platform orchestrates multi-GPU replicas.
• Always-loaded weights. The GPUs are reserved for you alone, and the model stays resident on them between requests.
• Setup per replica. You choose GPU type, GPUs per replica, replica count, and region, then deploy from the console, CLI, or API. Dedicated deployments bill only while they are in Running status, per the Dedicated Endpoint docs.
• Engineering support. GMI engineers work with you from prototype to production SLA.

The public K3 engine documentation shows where the tuning decisions fall, and they differ between the two GPUs:

These are agenda items for the tuning plan, drawn from vLLM and SGLang documentation. Tuned throughput depends on prompt lengths, concurrency, and reasoning effort, so the plan sets targets on your own traffic and measures each change against them. Several inputs come from you: the evaluation set and the accuracy threshold every tuning change must pass, the traffic forecast, and the region and compliance requirements.

Before Blackwell is delivered

Three steps can start while B200 and B300 are in pre-order:

1. Build the baseline on the API. Run your real prompts against K3 on the per-token endpoint, using the LLM API reference, and keep the evaluation set and usage log. The same set later validates the dedicated endpoint.
2. Price the H200 route if you need dedicated K3 sooner. H200 is available now on Prime, and GMI engineers can size a multi-node K3 deployment with you.
3. Put both B300 and B200 into the pre-order sizing request. Ask for each option's delivery timing, replica shape, and region side by side, so the choice rests on schedule and cost for your traffic.

When you move, the dedicated deployment exposes its endpoint URL once it reaches Running status, and requests go there with your API key.

Contract, billing, and trial credits on Prime

Billing for K3 on Prime Inference is per GPU-hour, with no minimum contract to sign first. Reserved capacity, available seasonally or annually, carries lower per-hour rates for sustained production traffic. Qualified prospects receive free GPU-hour trial credits to validate performance against their own workload, and GMI prepares a tailored quote from your model and traffic profile. The uptime SLA depends on GPU series and deployment configuration, and committed SLAs are set by contract. Regions span Asia-Pacific, North America, and Europe, with specific regions accommodated case by case.

FAQ

How does GPU memory for long contexts compare on a B300 replica versus a B200 replica for K3?

Adding up the per-GPU memory listed on the Prime Inference page, an 8-GPU B300 replica totals 2,304 GB (8 × 288 GB) and a 16-GPU B200 replica totals 3,072 GB (16 × 192 GB). How much of each pool serves K3's KDA state and MLA KV caches depends on the parallel layout, cache precision, and engine memory settings, which the tuning plan sets against the context lengths and concurrency targets in your sizing request. The single-node B300 replica keeps model-parallel communication inside one node.

If we fine-tune Kimi K3, does the replica shape stay the same?

A fine-tuned checkpoint with the same architecture and MXFP4 weight format, K3's native precision, keeps the 8-GPU B300 and 16-GPU B200 shapes above as the starting point for sizing. A checkpoint exported in 16-bit comes to about 5.6 TB (2.8T parameters at 2 bytes each); since Prime makes quantization configurable, the tuning plan sets the serving precision against your evaluation set and accuracy threshold. Weights load from Hugging Face, S3, or your own storage.

Size your Kimi K3 deployment

Bring your usage log, evaluation set, context lengths, concurrency target, and region requirements. GMI engineers will lay out B300 and B200 options for Kimi K3 on Prime Inference, including the H200 route. Book time with an engineer, or sign up and send K3 its first API requests.