Which Hosting Platforms Handle Short Agent Tests and Always-On Workers With Inference and Runtime Managed Together?

Instance lifecycle is what separates a short agent test from a 24/7 worker, and Option 03, Compute + Models, on Agentbox runs both from one registered template. Ephemeral instances spin up and tear down in seconds for tests, and workers stay online around the clock. The same agent image moves between the two shapes, calls the same models through the same endpoint on the same network, and you control its lifecycle through an API. Agentbox is in early access.

Six requirements for hosting tests and workers together

Three shapes on one platform

Agentbox describes three workload shapes and runs all three on one platform.

Oqoqo, an Agentbox customer, shows the ephemeral end at scale. It evaluates agents, MCPs, SDKs, APIs, and CLIs, and each run needs a clean, throwaway environment. On Agentbox it runs up to 50,000 concurrent sandboxes in a single evaluation run, built from its own Docker images and ephemeral by default, with no sandbox infrastructure to build or maintain. For a large parallel run of your own, GMI sales can size the capacity with you.

On the other end, an always-on worker is an instance you provision once and leave running. Agentbox supports digital workers, monitoring agents, voice and real-time agents, and hosted MCP gateways in this shape.

Put inference and runtime in the same deployment

Agentbox gives you three adoption paths: Option 01 Compute, where GMI hosts the container and you bring your own model layer; Option 02 Models, where you call 200+ models through the API and run your own runtime; and Option 03 Compute + Models, which combines both. Option 03 puts inference and runtime in one deployment.

Switch on MaaS integration in the wizard's Infrastructure step and tick each model the agent may call; the list stays editable. GMI sets the key and base URL as locked variables when the container starts, so the image itself never holds a model key. The registration guide lists three environment variables for the MaaS connection:

• GMI_MAAS_API_KEY, the API key, injected at runtime
• GMI_MAAS_BASE_URL, the OpenAI-compatible base URL
• GMI_MODELS, the model ID your agent calls

Once GMI_MODELS holds a model ID, the agent reads these values the way any OpenAI-compatible client would:

from openai import OpenAI

client = OpenAI(
    base_url="<GMI_MAAS_BASE_URL>/v1",
    api_key="<GMI_MAAS_API_KEY>",
)

response = client.chat.completions.create(
    model="<GMI_MODELS>",
    messages=[{"role": "user", "content": "Hello"}],
)

If your agent also needs third-party credentials, add them as SECRET variables in the Env Variables step. SECRET values are write-once, encrypted at rest with AES-256, and kept out of logs and audit history.

Inference on Agentbox runs on GMI's own models over the same network as the container, and model usage is billed alongside runtime.

Run the lifecycle through the API

Once the agent is registered, the My Agents dashboard shows the template ID and the lifecycle as four calls with your GMI API token:

1. Provision an instance from the template.
2. List the tasks running under the deployment.
3. Poll until the status is running, then route your traffic to it. Each launched instance gets its own public URL.
4. Terminate when the session ends.

You can also launch an instance from the Monitor tab. That tab counts instances by Active, Error, and Creating state, stamps the last provisioned time, and lists each instance with a log view beside it. The Analytics tab shows usage by model.

Every instance is sized at the wizard's container tier, 2 vCPU and 4 GB RAM with 10 GiB of ephemeral and 30 GiB of data storage.

For short tests, a test harness can provision an instance, run a case, read the result, and terminate. For a worker, provision once and leave it running. The calls are the same in both cases.

How billing behaves for each shape

Billing has two metered parts. Container compute is metered per second from the moment an instance reaches Running status until it is terminated. GMI model tokens are metered on top when the agent calls GMI models. Both appear separately in Console, Settings, Usage & Billing, and each agent's Analytics tab shows its usage. When you provision instances yourself for testing or development, you pay for those instances. There is no listing fee and no minimum spend.

Each lifecycle follows from this:

• Ephemeral tests. Terminate the instance as soon as the test ends and make deletion the last step of your harness; the Agentbox API exposes list and terminate calls.
• Always-on workers. A worker runs continuously, so plan its compute as a standing line item and track its usage in the agent's Analytics tab.

Keep state outside the container

GMI containers are stateless, so keep anything a worker must remember outside the container. Watch the Error count in the Monitor tab, or poll the list-tasks call from your own supervisor, and decide in your application how to retry work from the state you saved. A new instance has its own URL, so keep the address your clients call behind a router you control. Workers that need memory across sessions should write it to Redis or a database, and the connection credentials should go in as SECRET variables.

Slow tasks get their own pattern. Agent tasks can take anywhere from 30 seconds to several minutes, and the gateway closes connections that stay open too long with a 504 Gateway Timeout. The long-running requests guide answers with an async handoff: reply at once with a 202 and a job_id, do the work behind the scenes, and have callers poll a status endpoint every 3 to 5 seconds. Use it for any request whose work can outlast the gateway window, in long-running jobs and always-on workers alike, and return clear failed states so callers can react.

From first test to a standing worker

1. Package the agent as a Docker image that serves HTTP on a port (8080 by default), answers health checks, and reads its model settings from the environment.
2. Open the register guide, choose GMI CE Deployment, point at your registry, turn on MaaS integration, and select your models.
3. Register the agent. It stays in Draft, deployable only by you, and registering starts no instance.
4. Provision a test instance, send sample requests, and check latency, output, and error handling. Terminate when done.
5. Wrap your tests in a harness that provisions and terminates through the API.
6. Provision one instance for the always-on worker, watch it in the Monitor tab, and move its job state into Redis or a database.

Publishing to the Agentbox marketplace is optional and separate. The Agentbox overview covers listing and badges.

FAQ

How do we test a new build of the agent without touching the always-on worker that is already running?

Register the new build as its own agent. It stays in Draft, deployable only by you, and each registered agent's Integration tab shows its own Template ID, so test instances provision from that template while the worker keeps using the original. Iterate on the new build by re-registering that same test agent.

What should a test harness do when a job never finishes?

The harness keeps its own overall deadline and polls every 3 to 5 seconds; when the deadline passes, it records the case as timed out and calls terminate on the test instance, whatever the job endpoint last reported. The endpoint can also return timeout alongside completed and failed, so a job that is still responding ends early, and the harness treats all three as end states. Job state written to Redis or a database outlives the instance, so the harness reads the last saved state from there after terminating it.

Register once, run a test, keep a worker up

In the GMI console, register one agent with MaaS integration on, provision a test instance from it, then provision a second instance as the standing worker. If the worker fleet will be larger, take your expected instance count to GMI sales.