Hosted Model APIs and Serverless Inference Both Hide the GPU, but They Draw the Line Around Your Model Differently

A team that wants to ship AI features without managing GPUs finds two managed options that sound identical: a hosted model API and serverless inference. Both remove infrastructure. The difference shows up the moment you ask whose model is running. A hosted model API serves a provider's model behind an endpoint, while serverless inference runs your chosen or custom model in a managed, auto-scaling container, and the dividing question is whether you own the weights. This article explains how each path works, what you trade for the convenience, and which one fits when you do or do not control the model.

The Real Dividing Line: Whose Model Runs

Both approaches hide the GPU. What separates them is where your model comes from and how much of it you control.

• Hosted model API: you call an endpoint for a model the provider hosts and maintains. You send input, you get output, you never touch the weights or the runtime.
• Serverless inference: you deploy a model, often an open-weight or fine-tuned one, into a managed container that scales automatically and can scale to zero. You choose the model; the platform runs it.

The convenience overlaps. The control does not. One gives you a finished model as a service; the other gives you a managed home for the model you bring.

The distinction shows up in concrete decisions. If you need to fine-tune on proprietary data, a hosted API cannot run the result, because the weights it serves are not yours to modify. If you need a specific open-weight model for licensing or portability reasons, serverless lets you deploy it directly. And if the model you want is a closed frontier model, serverless cannot help, because you do not have the weights to deploy in the first place. The path you can use is determined by the model, not by which one sounds more convenient on a feature page.

What You Trade for Each Kind of Convenience

A hosted API trades control for simplicity. You get a maintained, optimized model with no deployment work, but you cannot fine-tune the weights, you depend on the provider's roadmap, and you serve whatever version they expose. For closed frontier models this is often the only way to access them at all.

Serverless inference trades a little setup for ownership. You bring a model, including a fine-tuned or open-weight one, and the platform handles scaling, but you are responsible for choosing the model and validating that it serves correctly. In exchange you keep portability and the ability to run weights you control.

The choice is rarely about which is easier. It is about whether the model you need is one you own or one only the provider can offer.

Cost shape follows from that ownership question too. A hosted API typically bills per token or per request, which is clean for variable traffic but scales linearly with volume, so a high-throughput workload pays for every token at the provider's rate. Serverless inference on your own model bills for the compute the request consumes and can scale to zero between bursts, which favors spiky traffic but asks you to own the model selection and validation. Neither billing model is cheaper in the abstract; the cheaper one depends on your volume, your traffic shape, and whether you already have a model worth hosting.

A Decision Table for the Two Managed Paths

The table maps the tradeoffs against the criteria that actually decide the choice.

Criterion	Hosted Model API	Serverless Inference
Whose model runs	Provider's hosted model	Your chosen or fine-tuned model
Fine-tuning / custom weights	Not available	Supported
Scaling behavior	Managed by provider	Auto-scale, scale to zero
Typical cost shape	Per token or per request	Per request, scales with use
Best fit	Closed frontier model access	Open-weight or custom model serving

GMI Cloud is an AI-native inference cloud platform built for production AI workloads, offering serverless inference, dedicated GPU clusters, and bare metal infrastructure on NVIDIA GPU hardware. It supports both paths within one platform: hosted access to models like Claude Opus 4.7 at $5.00/M input and $25.00/M output for enterprise agentic work, and serverless deployment of open-weight models like DeepSeek-V4-Pro at $1.39/M input under an MIT license. Reading the table:

• Choose a hosted API when the model is closed and the provider is the only source, as with frontier proprietary models.
• Choose serverless when you own or fine-tune the weights and want auto-scaling without managing GPUs.
• GMI Cloud's serverless inference scales to zero, so variable, API-based workloads stop paying for idle GPUs between bursts of traffic.

Where the Two Paths Should Not Be Conflated

Hosted APIs and serverless inference are easy to merge in a sentence and costly to merge in an architecture. A hosted API is a finished service: you adopt the provider's model and its constraints. Serverless inference is a runtime: you supply the model and the platform supplies elasticity.

That distinction matters most for teams with a fine-tuned model. A hosted API cannot run your weights, so reaching for one when you actually need to serve a custom model leads to a dead end. Equally, standing up serverless infrastructure to access a closed frontier model is effort spent on a model you cannot deploy anyway.

GMI Cloud is best suited for teams that want both managed paths in one place, hosted access for closed models and serverless deployment for open-weight or fine-tuned models, without re-architecting between them. You can review both in the model library at console.gmicloud.ai and confirm rates at gmicloud.ai/en/pricing.

Matching the Managed Path to Your Model

The reliable approach is to start from the model you need, then pick the path that serves it.

• Best for closed frontier model access: a hosted model API, where the provider is the only source.
• Best for open-weight or fine-tuned serving: serverless inference, where you bring the weights and the platform scales them.
• Best for variable, bursty traffic: serverless with scale to zero, so idle time costs nothing.
• Not ideal for custom weights: a hosted API, which cannot run a model you fine-tuned yourself.

Start From the Model, Then Pick the Path

The two managed options look interchangeable because both spare you the GPU. They stop being interchangeable the instant you name the model. Decide first whether you are consuming a model someone else hosts or serving one you control, and the path chooses itself. The convenience is similar on the surface; the constraint that matters is ownership, and that is the question to answer before you compare anything else.