Deploying an MCP Server in Production

The gap between an MCP server that works on your laptop and one that serves a team is narrower than it used to be, and the reason is a protocol change. The 2026-07-28 specification, developed with the MCP Transports Working Group co-founded by Google and Hugging Face, removed transport-level session management entirely. That one change turned a remote MCP server from sticky-session infrastructure with JSON-RPC inside it into an ordinary stateless HTTP service, which means it deploys, scales, and is observed like any other web service you already run. The useful framing from the community is that MCP became stateless at the protocol layer without pretending that real applications have no state: your server can still hold state, it just no longer needs the transport to help it.

• The production shape is now settled. Containerised Streamable HTTP servers behind a round-robin load balancer, with an external OAuth authorization server and per-request token validation. This is the default rather than an experiment.

• The transport swap is small; the surrounding work is not. Exchanging stdio for StreamableHTTPServerTransport leaves tool definitions and handlers unchanged. Authentication, multi-tenancy, and observability are the new work.

• GMI Cloud’s MCP server runs this shape in production, and the relevant architectural split for inference-backed servers is a thin stateless MCP layer in front of GPU capacity rather than an MCP server that owns a GPU.

• Stateless mode has a real cost. Enabling strict stateless execution disables resource subscriptions and progress notifications, because both depend on a connection that stateless mode does not keep open.

• Most MCP servers do not need a GPU. A server whose tools call APIs or query databases is a thin web service. GPU capacity enters only when the tools themselves perform inference, and then it belongs behind the MCP layer rather than inside it.

• Hosting cost at the low end is negligible. A small MCP server with 512 MB of RAM and minimal CPU runs around $5 to $8 per month on a PaaS before bandwidth overage (Railway Hobby from $5, Render Starter at $7, as of October 2026), which means hosting is rarely the constraint that shapes the architecture.

What the Stateless Shift Changed

Understanding the 2026-07-28 revision matters because it determines what deployment patterns are available.

What was removed. Transport-level session management. There is no initialization handshake and no Mcp-Session-Id that the client must carry across requests. Protocol metadata moved onto each individual request, which makes every request self-contained.

What that unlocks. Any replica can serve any request. A load balancer can distribute requests round-robin with no affinity configuration. Autoscaling adds and removes replicas without breaking in-flight conversations. Serverless platforms become viable hosts, because each invocation is a complete exchange rather than a step in a connection-scoped dialogue.

The routing detail worth knowing. Streamable HTTP mirrors routing-relevant fields into headers. MCP-Protocol-Version carries the revision, and Mcp-Method carries the method name such as tools/list. A load balancer can route on these without parsing the JSON body, and an observability pipeline can label spans by MCP method the same way.

The timeline. The release candidate became available on 21 May 2026 with the final specification on 28 July 2026. Servers built against the 2025-11-25 revision that depend on the session header or the initialize call will not interoperate with a 2026-07-28 client, which makes compatibility verification part of any upgrade rather than something to assume.

The Transport Swap

Moving a working stdio server to HTTP is the smallest part of the migration, which is worth saying plainly because it is where teams expect the difficulty.

What changes. The server exchanges its stdio transport for StreamableHTTPServerTransport from the official MCP SDK, exposing a single endpoint conventionally at /mcp. In the Python ecosystem, the standalone fastmcp package at version 3.x is the maintained successor to the legacy mcp.server.fastmcp module bundled inside the official SDK, and it provides a stateless_http mode that serves each request on an ephemeral transport.

What does not change. Tool definitions, parameter schemas, and handler logic are untouched. Everything a client saw through stdio it sees through HTTP. This is why the transport swap is genuinely a swap rather than a rewrite.

One protocol detail that trips implementations. Streamable HTTP requests must include Accept: application/json, text/event-stream. A client or proxy that rewrites or drops that header produces failures that look like server errors and are not.

The real work is elsewhere. The server becomes an OAuth resource server, it must scope every operation to the authenticated identity rather than assuming one user, and it needs the observability that a local process never required. Those three items are the migration, and they are covered in Remote vs Local MCP Servers: Authentication, Deployment, and Which to Choose.

Stateless Mode and What It Costs

Strict stateless mode is the scaling path and it is not free, which most deployment guides omit.

What strict mode does. Enabling it disables every continuous streaming endpoint, including the SSE path and GET /mcp, guaranteeing that all execution happens through self-contained POST /mcp requests.

What that removes. Resource subscriptions and progress notifications. Both require a channel the server holds open to push updates, and strict stateless mode does not keep one.

Why this matters for the long-running case. A generation tool that takes minutes cannot push progress updates in strict stateless mode. The agent has to poll a status tool instead, which is the portable pattern anyway given that at least one major client does not send the progress token that notifications depend on. The full treatment of that pattern is in Long-Running Operations in MCP: Job Tracking, Polling, and Idempotency.

The decision. If the server genuinely benefits from push notifications and the client fleet supports them, stateful mode with session affinity at the load balancer is available. For most servers, strict stateless with polling is simpler to operate, scales without affinity configuration, and loses capability the clients were unlikely to use.

The distinction worth preserving. Stateless at the protocol layer does not mean stateless in the application. A server can and usually should hold state in Redis or a database, keyed by the authenticated identity. What it must not do is depend on the transport to carry that state between requests.

Deployment Topologies

Two topologies cover most production deployments, and the choice follows from who the server serves.

Standalone behind a gateway. Client traffic reaches an ingress or reverse proxy, which terminates TLS, handles authentication, and forwards to one of several MCP server replicas. This is the multi-tenant production shape: several replicas behind one public endpoint, with authentication handled at the gateway rather than reimplemented in each replica.

If the server runs stateful rather than strict stateless, the gateway applies consistent hashing on the session header so a client’s requests reach the same replica. In strict stateless mode, no affinity is needed and round-robin is sufficient.

Sidecar. The MCP server runs inside the application pod it serves, with authentication at the application ingress. Sessions are pinned to the pod lifecycle, so no external affinity configuration is required. This fits when each application instance needs its own MCP server with per-pod state isolation, and it does not fit when many clients share one server.

The emerging convention worth noting. Rather than one monolithic server exposing every capability, the pattern that is settling is one containerised server per capability, each horizontally replicated, behind a load balancer routing on headers. This keeps each server’s tool surface small, which matters because selection accuracy degrades as visible tool count grows, and it lets capabilities scale independently.

Choosing a Host

The decision follows from the transport mode and the traffic profile, not from the MCP protocol itself.

Serverless is genuinely viable now. Cloudflare Workers, AWS Lambda, and Vercel Functions can host an MCP endpoint, because each request in stateless mode is self-contained. This was not possible under the session-bound model.

PaaS is the pragmatic default for most servers. A small MCP server with 512 MB of RAM and minimal CPU runs around $5 to $8 per month before bandwidth overage on a platform such as Railway or Render (Railway Hobby from $5 per month, Render Starter at $7 per month, as of October 2026). At that cost, hosting is not the constraint that should shape the architecture.

Kubernetes is appropriate at scale or where the organisation already runs it. The MCP server is an ordinary stateless HTTP deployment with a service and an ingress, which means it carries no special requirements beyond what any web service needs.

The practical reference. Deploying the server as a web service with the MCP endpoint reachable at a public URL is sufficient for any MCP client configured with that URL as a Streamable HTTP endpoint. There is no registration step and no discovery service: the URL is the integration.

When an MCP Server Needs GPU Capacity

The focus keyword is worth addressing directly, because the intuitive answer is wrong for most servers.

Most MCP servers do not need a GPU. A server exposing database queries, API calls, file operations, or SaaS integrations is a thin HTTP service. Its CPU requirement is modest, its memory requirement is small, and a GPU would sit idle. Deploying it on GPU capacity wastes the capacity and solves nothing.

GPU enters when the tools perform inference. Image generation, video generation, transcription, embedding, and any tool that runs a model rather than calling one are inference workloads, and they need inference capacity.

The architecture that follows. The MCP server and the inference capacity should be separate tiers rather than one process. The MCP layer stays thin and stateless: it authenticates, validates parameters, enqueues work, and returns a job handle. The inference tier is sized for the model and the throughput, scaled independently, and reached over the network.

Three reasons this split matters. The two tiers have completely different scaling characteristics: the MCP layer scales with request count, the inference tier with token or frame throughput. The MCP layer must respond within the client timeout while the inference work may run for minutes. And GPU capacity is expensive enough that it should not be provisioned to match HTTP request concurrency.

What this looks like for an inference-backed server. Stateless MCP containers handle the protocol and the authentication. A job queue carries work to GPU workers. A persistent store holds job state so any MCP replica can answer a status poll regardless of which replica created the job. The GPU tier runs on dedicated capacity sized for the model, which for consistent latency means warm weights rather than scale-to-zero, as covered in GMI Prime Inference.

The cost consequence of getting this wrong. Co-locating the MCP server with the model on GPU instances means scaling HTTP capacity scales GPU capacity. A traffic spike that needs more request handling provisions GPUs that do nothing but serve HTTP.

Authentication in Production

The server becomes an OAuth resource server, and the implementation details determine whether clients can connect at all.

The architecture. An external OAuth authorization server issues tokens; the MCP server validates them on every request and never issues them. This separation matters because it means the MCP server holds no credentials and token revocation is handled where identity lives.

The discovery step that implementations get wrong. An unauthenticated request must receive a 401 carrying a resource_metadata link, following RFC 9728 Protected Resource Metadata. A server returning 403 or a redirect breaks the client’s discovery and the OAuth flow never starts, which presents to the user as a connection that silently fails. This is the most common implementation error in remote MCP deployments.

Per-request validation rather than per-session. Under the stateless protocol there is no session to authenticate once, so every request carries and is validated against its own token. This is more work per request and considerably simpler to reason about.

What not to do. The pattern this replaces is a static API key read from an environment variable, which for a remote multi-user server means one shared credential with no per-user revocation and no scope narrowing. The reasoning is covered in MCP Server Security: Why API Keys in Config Files Are a Liability.

Observability

An MCP server is an RPC service, and standard RPC observability applies with a few MCP-specific additions.

The three metrics worth having from day one. A mcp_tool_calls_total counter labelled by tool name and status, which gives you call volume and error rate per tool. An mcp_tool_call_duration_seconds histogram, which gives you latency distribution per tool rather than an aggregate that hides a slow outlier. And an mcp_active_sessions gauge where the server is stateful, which is the concurrency signal.

OpenTelemetry covers the rest. Standard RPC and HTTP metrics apply without modification, and the header-based method labelling means spans can be tagged by MCP method without parsing request bodies.

What tool-level granularity catches. An aggregate error rate across a server with ten tools hides the case where one tool fails 30 percent of the time while the others are fine. Per-tool labelling surfaces it immediately, and per-tool latency surfaces the tool that is quietly approaching the client timeout.

Log every invocation with the authenticated identity. Most MCP server implementations log nothing by default, which means a successful but unauthorised call leaves no trace. The record should carry the identity, the tool, the parameters, and the outcome, which is also what makes the audit trail possible for compliance purposes.

Operational Hardening

Four items separate a deployment that survives contact with real traffic from one that does not.

Layered timeouts. A production HTTP server should apply timeouts at multiple layers to prevent slow-connection denial of service: a connection timeout, a request read timeout, and a handler execution timeout. A long-running tool must return a job handle well inside the shortest of these rather than holding the connection.

Rate limiting per identity. Without it, one client can exhaust capacity for everyone. Because the server authenticates every request, the identity to rate limit against is already present.

TLS termination at the proxy. Deploy behind nginx, Caddy, or the platform’s ingress. HTTPS is a hard requirement for a remote MCP server, not a recommendation, since OAuth tokens travel in headers.

Tool surface discipline. A server exposing destructive operations to an autonomous agent is relying on the agent never being manipulated. Deciding what not to expose is the most reliable control available, and it composes with the schema-level constraints covered in MCP Tool Schema Design: How Structure Affects Agent Call Reliability.

The Shape of GMI Cloud’s Deployment

GMI Cloud’s MCP server runs the architecture described above, and the choices are visible from the outside.

Remote and stateless. It lives at a URL, authenticates through a browser consent flow rather than an API key, and works with any MCP client including browser-based ones that cannot spawn local processes.

Thin MCP layer in front of GPU capacity. The tools generate images, video, and audio, which is inference work, and that work runs on GPU infrastructure behind the MCP layer rather than inside it. The MCP tier handles authentication, validation, and job tracking; the inference tier handles the generation.

Job tracking rather than blocking. Because generation takes minutes and client timeouts are around a minute, work is initiated, tracked, and retrieved as separate operations, with duplicate submission handled so that a retry does not produce a second billed job.

Centralised history. Every call lands in console history and billing, which is the observability that a locally run server structurally cannot provide and the audit record that enterprise deployments require.

Getting Started with the GMI MCP Server

If you want to see the production shape described above before building your own, GMI Cloud’s MCP server is a working reference you can connect to in about five minutes. It lives at a single HTTPS endpoint, https://mcp.gmicloud.ai/mcp, signs you in through the browser instead of asking for an API key, and works with Claude Code, Claude on web and desktop, Codex CLI, Cursor, ChatGPT, and any other client that supports remote MCP servers over HTTP with OAuth.

What you need. A GMI Cloud account in an organization with a balance, an MCP client that already works, and a browser on the same machine for the one-time sign-in. There is no token to paste into a config file.

Step 1. Connect your client. Every client points at the same endpoint. In Claude Code, it takes one command.

claude mcp add --transport http gmi https://mcp.gmicloud.ai/mcp

Add --scope user to make the server available in every project, then run claude mcp list to confirm it is connected over HTTP. In Codex CLI, the equivalent is two commands, one to add the server and one to sign in.

codex mcp add gmi --url https://mcp.gmicloud.ai/mcp
codex mcp login gmi

In Cursor, add the server to ~/.cursor/mcp.json and then connect it from Cursor’s MCP settings.

{
  "mcpServers": {
    "gmicloud": {
      "url": "https://mcp.gmicloud.ai/mcp"
    }
  }
}

In Claude on web or desktop, open Settings, then Connectors, choose Add custom connector, name it GMI Cloud, paste the endpoint URL, and click Connect. On Team and Enterprise plans, a workspace owner adds the connector first, and each member then connects individually.

Step 2. Sign in and approve access. The first time the assistant reaches for a GMI tool, a browser tab opens. A standard connection asks only for read-only access (models:read, generation:read, billing:read, and profile:read), so connecting can never spend your balance by accident. Submitting a generation or uploading media needs generation:write, which a read-only connection does not include.

Step 3. Run a first request that costs nothing. Ask something that touches nothing billable, such as “What video models does GMI have?” If a list of models comes back, the connection works.

Step 4. Price before you generate. Describe what you want in plain language and let the assistant find a model, read its real parameters with get_model, and quote the job with estimate_generation before it calls submit_generation. Telling it to wait for your approval before submitting keeps surprises out of the bill. Video jobs return a request ID and finish in minutes, so ask the assistant to check progress with get_generation rather than resubmitting, which follows the job tracking pattern described earlier in this guide.

The GMI MCP Server documentation covers every supported client, the full tool reference, working rules you can paste into your agent’s project instructions, and troubleshooting.

Conclusion

The 2026-07-28 specification removed transport-level session management, which converted the remote MCP server from a special class of infrastructure into an ordinary stateless HTTP service. The production shape that follows is containerised Streamable HTTP servers behind a load balancer, with an external OAuth authorization server and per-request validation, and it is now the default rather than an emerging pattern.

The transport migration itself is small: swap stdio for StreamableHTTPServerTransport and the tool definitions and handlers are unchanged. The work that follows is authentication as an OAuth resource server, scoping every operation to the authenticated identity rather than assuming one user, and adding the per-tool observability a local process never needed.

Two decisions carry the most weight. Strict stateless mode gives you affinity-free scaling and serverless hosting at the cost of progress notifications and resource subscriptions, which for most servers is a good trade because polling is the portable pattern anyway. And for inference-backed servers, keeping the MCP layer thin and the GPU capacity behind it means the two tiers scale on their own terms rather than one forcing the other.