Auto Routing

Send model: "auto" (or no model) and the gateway resolves it to a concrete model with the first matching rule, lowest priority number first. A target the caller's policy forbids is skipped, so routing can never pick a model the caller may not use.

routing:
  rules:
    - {name: images,    priority: 10,   has_images: true,       target: azure-openai}
    - {name: simple,    priority: 20,   complexity: [SIMPLE],   target: azure-openai}
    - {name: demanding, priority: 30,   min_complexity: MEDIUM, target: bedrock}
    - {name: otherwise, priority: 1000,                         target: azure-openai}

Rule fields

Field	Meaning
name	Rule name (1 to 80 characters).
priority	1 to 100000; lower runs first (default 100).
target	The model alias to use.
targets and strategy	Several aliases; strategy: cheapest picks the lowest estimated cost.
task_types	Match the declared task type (X-Task-Type header, task_type in /v1/invoke, or key/team metadata).
domains	Match the caller's domain.
has_images / has_tools	Match requests with images or tools.
min_prompt_tokens / max_prompt_tokens	Match by prompt size.
complexity / min_complexity	Match the automatic complexity tier.
enabled	Turn a rule off without deleting it.

Complexity

The gateway scores each prompt locally as SIMPLE, MEDIUM, COMPLEX or REASONING. It weighs prompt length, code, reasoning wording, technical terms and multi-step patterns. It makes no model call, takes about a millisecond and is deterministic.

TIP

The score judges the wording and length of the prompt, not its meaning. A long but repetitive prompt can still score SIMPLE.

If the scorer cannot be loaded, rules that ask for a complexity simply do not match and one warning is logged.

Cost-based routing

- {name: easy, complexity: [SIMPLE], strategy: cheapest, targets: [gpt-4o-mini, haiku, gemini-flash]}

Targets are ordered by estimated cost: prompt estimate × input price plus expected completion × output price, using the deployment's input_cost_per_token / output_cost_per_token or LiteLLM's price list. Models without a price sort last. Policy and image checks still apply, and the next cheapest is used if one is not permitted.

What was decided

On the server, the response carries x-gateway-requested-model, x-gateway-model, x-gateway-routed-by, x-gateway-complexity, x-gateway-complexity-score and, after a budget downgrade, x-gateway-downgraded-from.

In the SDK the same information is in resp._hidden_params["gateway"]:

{'requested_model': 'auto', 'model': 'azure-openai', 'routed_by': 'simple', 'complexity': 'SIMPLE', 'complexity_score': -0.15}

If no rule can serve a request, it is refused with no_route.