Which Managed Model Routers Learn Which Model Fits a Task, Without Hand-Written Rules or a Judge-Model Call?

For a managed model router that learns which model fits each task, without hand-written rules or a separate judge-model call, we recommend GMI Router. GMI's own trained model reads the task behind each prompt and picks the best-fit model from a pool your org owner approves, guided by a cost-quality mode you set on each request. There is no judge model in the loop, so choosing the route does not add a second model call. Microsoft Foundry model router, Amazon Bedrock Intelligent Prompt Routing, Not Diamond, and OpenRouter Auto Router also select or recommend models automatically; the comparison below shows what each one learns from and where its controls sit.

Most teams asking this question already route with rules that pick models by prompt length, keywords, or feature flags, and every new model means another round of edits. Not every rule disappears when a router takes over. Before you switch, separate selection heuristics from constraints and fixed routing requirements, such as "this request type always goes to this model," because each needs a different replacement. The rest of this guide starts from that rule set.

Signs your routing rules have hit their limit

Rule-based routing breaks in recognizable ways:

• Every model launch is a code change. A new model gets traffic only after someone edits the rules, and every older rule has to be rechecked against it.
• Proxies drift away from the task. A short prompt can be a hard reasoning problem, a long one a simple extraction job. A prompt that mixes tasks lands wherever the first matching keyword rule sends it, and the fix is another rule that has to be ordered against the first.
• Price and quality assumptions go stale. A cheapest-acceptable-model rule freezes one moment's prices and quality. Both change; the rule does not.
• Nobody can say which rule decided. As exceptions stack up and overlap, explaining a routing decision means tracing the code.

Sort your rules into three kinds before you switch

List every rule your routing layer applies and tag it with one of three kinds.

1. Selection rules pick a model based on what the request is: task type, difficulty, or a proxy for them. "Classification goes to the small model" and "long prompts go to the large model" both belong here. Fallback rules, which pick a model when the first one fails, count here too.
2. Constraint rules set the boundary a choice must stay inside: which models are approved for your data, or whether a request type should favor cost or quality. These move into router settings.
3. Hard rules fix an outcome regardless of the task: a request type or a customer that must always run on one specific model, or a rule that must override whatever the router would pick. Managed routers differ most here, so get documented answers before you switch.

What GMI Router takes over, and the controls that replace your rules

GMI Router takes over selection rules. Each model it can choose carries a quality score and a cost score per task category. Quality scores come from GMI's internal benchmark bank, and cost scores come from an internal auto-eval. Each request goes through four steps:

1. Understand. Router identifies the task type and what the request requires.
2. Evaluate. It weighs the eligible models on quality and cost for that task, against your chosen objective.
3. Select. It picks the best-fit model within the allowed pool.
4. Route. It sends the request to the selected model.

GMI describes its router as a trained model that makes one decision per request without a separate judge model, and reports routing latency under 200 ms.

Your constraint rules map onto three documented controls:

• Mode, per request. A cost-first rule such as "support summaries should be cheap" becomes mode: cost on those requests, and a rule that reserves top models for high-value work becomes mode: quality. The optional mode field also accepts balanced.
• Allowed Model Pool, per workspace. A rule like "never send this data to models outside our approved list" becomes pool membership, and your org owner decides which models the pool contains.
• Auto Mode, workspace-wide. A rule that switches routing off for everything maps to Auto Mode, which turns automatic routing on or off for the whole workspace.

The routing endpoint applies your workspace settings and returns routing metadata, including selected_model, task_type, and selected_mode, so you can inspect the model and mode selected for each request. The Router documentation covers the endpoint and authentication with your GMI API key. The documented request fields are messages, mode, and stream, and unknown request fields are ignored.

Routing is free during the preview, for a limited time. You pay only for the model that runs your prompt.

Translating a rule set into Router settings

Most rows move into Router. The hard rules that stay in your application are the cases where a specific model is a requirement, not a guess about fit.

Rule in your routing layer today	Kind	What handles it with GMI Router
Send a task type or difficulty level to a specific model	Selection	Router's task detection and per-category quality and cost scores
Use prompt length or keywords as a stand-in for task	Selection	Router's task detection; retire the proxy once the side-by-side run agrees
Keep cheap, high-volume request types on low-cost models	Constraint	mode: cost on those requests
Use the strongest models for high-value requests	Constraint	mode: quality on those requests
Only use models approved for your data or contracts	Constraint	Allowed Model Pool, approved by your org owner
Retry on a backup model when the primary errors	Selection	Router's documented fallback on transient errors (5xx, 429, or a first-token timeout when streaming): one backup model before the first token when streaming, the primary plus one backup otherwise. A rule naming a specific backup model is a hard rule
A request type must always run on one named model	Hard	The documentation does not list a per-request field for pinning a model. Route those requests in your application before the router, straight to the model through the LLM API with an explicit model ID
One customer must use a different model or pool	Hard	Controls are documented at the workspace level. Keep customer-specific model selection in the application, and use an explicit model ID for requests with a fixed destination

Managed routers compared by what replaces your rules

Each managed router learns from a different signal and puts its controls at a different level, which decides how much of your rule set survives.

Router	What it learns from	What replaces your constraint rules	Where the controls apply	Selected model visible?
GMI Router	Per-category quality and cost scores from GMI's benchmark bank and auto-eval; no judge model	Cost, Balanced, or Quality mode; Allowed Model Pool; Auto Mode	Mode per request; pool and Auto Mode per workspace	Yes, in routing metadata
Microsoft Foundry model router	A router model trained by Microsoft, which analyzes prompts by complexity, reasoning, and task type	Routing mode (Balanced by default, Cost, or Quality) and a model subset, which also serves as the fallback set; Azure Policy can govern which models a subset may include	Per deployment	Yes, in the response model field
Amazon Bedrock Intelligent Prompt Routing	Predicted response quality for each of two models	Exactly two models from the same family, a fallback model, and a response quality difference threshold	Per configured router	Yes
Not Diamond	Your own evaluation data, when you train a custom router (minimum of 15 samples, per its router training quickstart)	Your training data and labels	Per custom router	Yes, as the recommendation
OpenRouter Auto Router	A lightweight classifier across roughly 30 task types, ranked by community spend share over the past seven days	allowed_models, excluded_models, and five cost tiers	Per request, or saved on the workspace Routing page with an optional "prevent overrides" toggle	Yes, in the response model field

Microsoft's Learn documentation describes Foundry's model router as matching each prompt rather than relying on static rules, and says effective context is limited by the smallest underlying model. The Amazon Bedrock User Guide says Intelligent Prompt Routing is optimized for English prompts and can't adjust routing based on application-specific performance data. OpenRouter's documentation says saved workspace values apply unless a request sets the same field, and turning on "prevent overrides" makes the saved values final.

Questions to ask before your rules go away

Whichever router you choose, get documented answers to these before you delete a rule:

• Can a single request be pinned to a specific model? If not, your hard rules stay in the application, in front of the router.
• Can a hard rule override the router's choice?
• What happens when a model joins the pool? Ask whether a new model is scored for each task category before it receives traffic, and how quickly a removed model stops being selected.
• What happens to requests when automatic routing is off? A workspace-wide switch helps with rollback only if you know what the endpoint does then.
• Can different products or customers get different policies? Ask how many workspaces or deployments you would need to separate them.
• Can the router be calibrated on your own evaluation data? This decides whether the router's idea of quality matches yours.

Run the router beside your rules before you retire them

Retire rules by measuring, one rule at a time, where the router and your rules disagree:

1. Pick a fixed request set. Pull real prompts from each workload your rules cover, including the hard cases your exceptions were written for.
2. Record both decisions. For each request, log the model your rules would choose and GMI Router's selected_model, task_type, and selected_mode.
3. Compare only the disagreements. Run both models on those requests and compare quality and cost. The routing endpoint runs the model it picks and returns a completion, so the comparison set is billed as model usage.
4. Give every rule one of four verdicts. - It agrees with the router: retire it only after representative validation and a rollback plan are in place. - The router does better: retire it after checking quality, cost, and required constraints across the affected workload. - The rule does better on one task type: keep it in your application and send those requests straight to the model through the LLM API. - The rule is really a constraint: convert it into a mode value or pool membership.
5. Rerun after every pool or mode change.

Hard rules stay in your application until the questions above have answers.

See how Router reads your prompts

Open Test Routing in the GMI console and paste prompts from the categories your rules cover to see the task type and model Router picks. For workspace policies or fixed routing requirements, talk to our team.