Building multi-model agents on AgentBox: one key, 200+ models, zero routing glue

Most production agents use several models across a single run. A support agent might classify an incoming ticket using a small, fast model, pull context using an embedding model, and then hand the hard reasoning step to a frontier model. A coding agent might draft with one model and review with another. The moment your agent gets good, it reaches for more models.

That is where the bill and the engineering hours start climbing. Route every step to a frontier model, and you overpay, often by a wide margin, since most steps run fine on a lighter model. Route to several providers directly, and you inherit a new integration for each one: a separate key, a separate SDK quirk, a separate rate limit, and policy and rate-limit changes outside your control.

AgentBox is built so the model layer becomes infrastructure you call instead of code you maintain.

The problem is structural

Here is what a multi-provider agent looks like when you wire it up yourself. Three providers, three keys, three failure modes:

# The version you end up maintaining
import anthropic, openai
from some_other_sdk import DeepSeekClient

claude = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_KEY"])
gpt = openai.OpenAI(api_key=os.environ["OPENAI_KEY"])
deepseek = DeepSeekClient(api_key=os.environ["DEEPSEEK_KEY"])

# Each carries its own request shape, its own rate limit,
# its own 429 behavior, its own account that can get restricted.

Every provider you add is another surface that can rate-limit you under load, another billing relationship to reconcile, and another place exposed to policy and rate-limit changes outside your control. For a consumer-facing agent with steady traffic, that can take your product down for users who simply showed up to use it.

How AgentBox handles it

On AgentBox, inference goes through GMI Models. Your container gets two environment variables injected at runtime, never shipped inside your image:

import os
from openai import OpenAI

# Injected by the platform at container start.
client = OpenAI(
    api_key=os.environ["GMI_MAAS_API_KEY"],
    base_url=os.environ["GMI_MAAS_BASE_URL"] + "/v1",
)

From there, switching models is a one-string change:

# Cheap, fast step
triage = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash",
    messages=[{"role": "user", "content": ticket}],
    stream=True,
)

# Hard reasoning step, same client, same key
answer = client.chat.completions.create(
    model="anthropic/claude-opus-4.8",
    messages=[{"role": "user", "content": context}],
    stream=True,
)

One key reaches 200+ open source and frontier models, all on-network. Claude, GPT, Llama, DeepSeek, Qwen, and more sit behind the same endpoint, so routing by cost, latency, or task quality becomes a routing table in your code, replacing a stack of SDKs. Billing is centralized too: every call across every model lands on one account, which gives you a clean monthly number in place of six invoices to reconcile. You pick which models your agent calls in Step 2 of the register wizard, so select them there before referencing them in code.

A couple of implementation notes that save you a deploy cycle. For long multi-step runs, use the async job pattern: return a job_id and poll for status. Streaming is also useful for surfacing partial output sooner. And because calls run on-network through GMI Models, your headroom comes from GMI's aggregate capacity rather than one external provider's rate-limit tier.

What this looks like in practice

The teams already building this way describe the same three wins: model choice, cost, and a model layer that runs itself.

On model choice and the engineering cost of getting it:

"Now we have this solution with GMI Cloud, which makes it a lot easier for us to build these things, reducing not just the infra costs, but also the engineering hours." Joshua Sum, CEO, Morphic

On what one account across many models buys a builder:

"With just one GMI account, developers and founders like me can get access to a lot of different things at low cost and low latency. That benefit was very clear." Steven Enamakel, CEO, TinyHumans

On switching models inside a running product with a one-line change:

"Having multiple model support is very important for us. GMI makes switching models extremely simple, and that helps our productivity a lot." Chenglin Wei, CTO, Topify

When to reach for this

For an agent that calls a single model and stays that way, a direct integration works well. The case for routing through AgentBox shows up the moment any of these is true: you want a cheaper model for the easy steps, you are starting to hit a provider's rate limits under steady traffic, you want more than one provider so your product keeps serving users even if one account gets restricted, or you would rather see one bill than reconcile several. At that point the model layer becomes infrastructure you call, freeing your team to build the agent itself.

Try it

Browse the catalog, deploy an agent, and route it through 200+ models from one key.

Start using AgentBox: https://www.gmicloud.ai/en/models/agentbox

Read the docs: https://docs.gmicloud.ai/agentbox-marketplace/overview