> The Markdown version of [The model gateway and lanes](how-models.html) on the aayudha docs site. Every page is listed in [llms.txt](llms.txt). # The model gateway and lanes Every verb that needs judgement runs in a **lane**: a named class of model work, such as `build` or `review`, with a settled default model for each host. Most of the time the host's own model does the reasoning inside a skill. A few verbs call a model directly from the `aayu` binary, and those calls go through one **model gateway**: it picks the model, keeps the reviewer in a different model family from the builder, speaks the provider's wire format, prices every reply, and stops all agent lanes when spend reaches the budget. ![A direct model call resolves its lane, passes the family check and the budget breaker, is routed to the AI gateway or the family's own provider, and lands priced in the actions log.](docs/guide/img/how-models.svg) *Each priced reply adds to the breaker's count, so the call that reaches the limit is the last one.* ## Lanes and their defaults The registry in `internal/verbs/registry.go` declares six lanes. Each verb names exactly one lane, and each lane has a default for each host. The table shows Amp, Cursor, and Claude Code; `aayu models show` lists every host, OpenAI Codex included. | Lane | What it does | Amp | Cursor | Claude Code | |---|---|---|---|---| | `build` | Writes code and artifacts | Amp medium | Grok 4.7 (legacy plan: Claude Opus 5.5) | Claude Opus 5.5 | | `spec` | Specs, decisions, briefs, copy | Amp medium | Grok 4.7 (legacy plan: Claude Opus 5.5) | Claude Opus 5.5 | | `plan` | Slices and plans work | Amp medium | Grok 4.7 (legacy plan: Claude Opus 5.5) | Claude Opus 5.5 | | `review` | Adversarial code and spec review | GPT-6.1 Sol | Grok 4.6 (legacy plan: Grok 4.7) | GPT-6.1 Sol | | `ui-review` | Visual and interaction critique | Amp high | Grok 4.6 (legacy plan: Gemini 3.8 Flash) | Gemini 3.8 Flash | | `chore` | Cheap mechanical work | Amp low | Composer 2.5 (legacy plan: Haiku 5.5) | Haiku 5.5 | Cursor's column follows `[hosts.cursor] plan` (the review-ladder decision of 2026-10-17). Cursor's plans from 12 October 2026, `x` and the default, include only Cursor's own models (Grok 4.7, 4.6, 4.5, and Composer 2.5); Claude, GPT, and Gemini are list-price pay-as-you-go there. aayu cannot tell the plans apart, so it defaults to the one that never spends list-price money by surprise. Set `plan = "legacy"` for a legacy Pro, Pro+, or Ultra plan, which still includes Claude and Gemini, in your user config (or `AAYU_HOSTS__CURSOR__PLAN`); the plan is yours, so a checkout's config cannot set it. A checkpoint records the build model the plan picked, so a later plan change does not relabel past builds. On `x` every lane is one family, so `review` and `ui-review` use another Grok tier and their reviews are labelled same-family; the merge gate's reviewer app can still be of another family. Composer's base model is unpublished, so its family is unknown, and a review between it and any build is same-family. Three rules shape this table: - **The family rule.** A reviewer must come from a different model family than the builder. A model that wrote the code is a poor judge of it, and two models from the same vendor tend to share blind spots. `review` and `ui-review` are marked `Adversarial`, and their defaults are never in the builder's family, except on a host plan that includes one family (Cursor's `x`), where the review is labelled same-family. - **Host exclusions.** A host lists the families it cannot run. Cursor excludes the `openai` family, so the `review` lane on Cursor defaults to Grok (`xai`) instead of GPT-6.1 Sol. - **Host-managed modes.** On Amp, the default is a mode (Amp low, medium, high) and the host picks the model behind it. The binary cannot call a mode, so a direct call from an Amp session uses the Claude Code default for that lane (`DirectHost`). The `review` default also names an escalation: GPT-6 Astra at high reasoning effort, used when a caller marks a review as heavy. The `ui-review` default on Cursor, Claude Code, and Codex is Gemini 3.8 Flash at high reasoning effort (`gemini-3.8-flash`, `google` family), so it is never in the family of an anthropic or OpenAI builder. Every model judgement of a UI goes through this lane: `aayu review`'s lane A, `polish`, which re-runs it, and `aayu verify`'s advisory look at its screenshots, which never decides whether a floor holds. It is interim until the AI gateway routes the target, `gemini-4-argon-high`, which needs a paid Google plan. > **Needs setup:** without Copilot (below), the ui-review lane needs a model key: the AI gateway's > (`[models.gateway]`) or Google's. Without one, and in the merge gate, which has no model key, `aayu verify` records `screenshot_review.skipped` with the reason > instead of findings, and `aayu review` stops. Set the key with `aayu auth set-secret `. Verify sends only > screenshots that its redaction pass read with OCR, so it also needs `tesseract` installed. **Gemini 4 Argon through Copilot (the default where Copilot is set up).** The binary's ui-review calls (review lane A, `aayu polish`, and verify's screenshot review) run on Gemini 4 Argon at high effort through Copilot CLI whenever `copilot` is on your `PATH` and signed in, on any host. Otherwise they run on `gemini-3.8-flash` at high effort through the AI gateway, as above. - **How it runs.** `copilot --model gemini-4-argon --reasoning-effort high`, the prompt on stdin, the screenshots as attachments, no tools, in an empty directory with a fresh `COPILOT_HOME`, so no repository hooks, MCP servers, or custom instructions load. It signs in with the `copilot` secret, else `COPILOT_GITHUB_TOKEN`, which goes to that process only, with `COPILOT_GH_HOST` (else `GH_HOST`) for a GitHub Enterprise host. - **What it costs.** Each call is charged the AI credits Copilot reports: input at 2 USD and visible output at 10 USD per million tokens, with reasoning not billed, about 0.043 USD for a lane A call (two probes, 2026-10-18). The budget breaker counts it like any other call. - **When it falls back.** When `copilot` is not installed or not signed in, lane A uses flash from the start, and stderr and the label say why (`gemini-3.8-flash (gemini-4-argon through Copilot is unavailable: …)`). When a call fails or its reply misses the schema, that call and every later one go to flash (`… (the Copilot route failed: …)`). The review always runs. - **Reviewer and builder.** Argon is google family, so it still never reviews a google builder's work. - **Turning it off.** In your user config (a repository's config cannot set this row): ```toml [models.ui-review.copilot] model = "" # lane A always uses gemini-3.8-flash through the gateway ``` Another Copilot model ID in that row runs that model through Copilot instead. > **Honest status:** the ux evals that gate this lane measured `gemini-3.8-flash` only. CI has no Copilot credential, > so it always takes the fallback, and its numbers are flash's. A local run with Copilot signed in reviews on Argon, > which those gating runs did not measure. `CopilotCLI` and `CopilotReady` in `internal/models/copilot.go`, `TestCopilotCLI` and `TestCopilotUIReviewRoute`. ## How `aayu models` resolves a lane `aayu models` prints the model, family, source, and route for every lane on every host (or only one host with `--host`). It uses the same resolver as a real call, in `internal/models/resolve.go`: 1. Look up the registry default for the lane and host. 2. If `[models..]` exists in config, take its `model`. The family is the `family` key, or it is inferred from the model ID prefix (`claude-`, `gpt-`, `gemini-`, `grok-`). An unknown family is a configuration error. 3. If the host excludes that family, fall back to the registry default, set the source to `fallback`, and log one line that explains the substitution. 4. For a direct call, if the result is a host-managed mode, resolve the lane again for the direct host (source `direct`). A heavy call then swaps in the escalation model, unless config chose the model. The source column tells you which rule won: `default`, `config`, `fallback`, or `direct`. ## Who calls models from the binary Ground rule 5 in [the build plan](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/docs/BUILD-PLAN.md?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha) says that inside a host, the host's model reasons. The binary calls a model itself only where the work must not depend on the host: - `inspect` with `[inspect] reviewer = "model"` (registry mode `direct`); by default the merge gate reads an agent app's review instead. - `review` lane A (`hybrid`: the host reasons, then the binary makes one direct pass). - The `brief` adversarial pass (`hybrid`), which checks the draft for claims its sources do not support. - `triage` when a person runs it without `--as` and holds a key (`hybrid`); it never runs in product CI, where the host's model or an agent app you @-mention classifies the issue. - The eval judge in the eval harness, which is not a verb. `inspect`, `review`, and `brief` call `models.CheckFamilies` before they call the model. It refuses a reviewer whose family equals the builder's, and a reviewer with no concrete family at all. ## The gateway: routes and wire formats `RouteFor` in `internal/models/client.go` decides where a call goes. When `[models.gateway] base_url` is set, every model goes to that one AI gateway, with the key from the secret that `[models.gateway] secret` names, sent as a Bearer token. Otherwise each family goes to its own provider, with a key in a secret named after the family. `base_url` and `secret` are read only from the user config, `AAYU_MODELS__GATEWAY__*` variables, or `--set`, never from the checkout: a PR's `.aayudha/config.toml` cannot choose where your keys are sent (`Load` in `internal/config/load.go`). The client speaks three wire formats: | Style | API | Path | Default for | |---|---|---|---| | `messages` | Anthropic Messages | `POST /messages` | `anthropic` | | `responses` | OpenAI Responses | `POST /responses` | `openai` | | `chat` | OpenAI Chat Completions | `POST /chat/completions` | `google`, `xai` | Through the gateway, the style is chosen per model: a `styles` entry for the model ID, then one for its family, then `api_style`, then the family default. The family always comes from the model ID, never from the endpoint, so the family rule holds even when every model sits behind one URL. Keys never live in config. Config names a secret, and `internal/secrets` resolves the name across the OS keychain, an environment variable, and an encrypted file. The environment variable is `AAYU_SECRET_` plus the name in upper case, with `-` as `_`: the secret `ai-gateway` is `AAYU_SECRET_AI_GATEWAY`, and a direct Anthropic key is `AAYU_SECRET_ANTHROPIC`. ## Cost and budgets Every direct call is one `model.call` record in the actions log, with input and output tokens, the cost in US dollars, and where the cost came from. `Cost` in `internal/models/gateway.go` picks the first figure it has: 1. `reported`: the cost the endpoint returned in `usage.cost`. 2. `price-table`: tokens times the `[[models.prices]]` entry for the model ID. 3. `list-price`: tokens times the model's published price in `models.ListPrices` (`internal/models/prices.go`), each with its provider's pricing page and the date it was read. The AI gateway the evals use reports no cost, so this is what they are charged (Appendix B D-3.4p). 4. `fallback-rate`: 15 USD per million input tokens and 75 USD per million output tokens. This is deliberately high, so an unpriced model trips the breaker early rather than never; the evals name such a model in their results' warnings. Cache reads and cache writes are charged at their own rates when the price has them, and a prompt past a model's long-context threshold is charged at its long-context rates. The **breaker** enforces `[budget]`. Before each call, `Check` refuses if agent lanes are paused, if the story has gone over its limit, or if the local day has reached its limit. After each call, `Add` adds the spend. When a story goes over `per_story_usd` (exactly the limit is still allowed, as it is for a build's `cost_usd`) or the day reaches `per_day_usd` (reaching it exactly counts), it writes the pause file `.aayudha/state/paused`, pages through `[notify]`, and returns `ErrBudget`. The CLI maps that error to exit code 6. Pages are keyed by scope, and `[notify]` sends each key at most once per local day, so a tripped breaker pages once. While the pause file exists, every direct call is refused, and the loop guard (`aayu hook loop:`, the first step of every scheduled loop) exits 6 too. `aayu resume` removes the pause file. The other `[budget]` keys are enforced elsewhere. `per_pr_inspect_max` caps `inspect` spend on one pull request across all its cycles. `ci_minutes_month` makes the loop guard exit 6 in GitHub Actions once the loops' runs this month have used that many minutes. `[autonomy.budget]` is a different budget: the wall time, spend, and failed attempts one `build` of a story may use. Its `cost_usd` applies only when model calls are pay-as-you-go: a `[models.gateway]` endpoint or a direct provider's key is configured, or a host reports its spend in `AAYU_COST_USD`. Otherwise there is no cost limit, because subscription use reports no cost. `per_story_usd` is at least `cost_usd`, so the breaker never stops a story before its build budget does. When `build` exceeds the budget, the story moves to `status:blocked`, the owner is paged, and `build` exits 4. ## Key data and file formats The lane types and table, from `internal/verbs/registry.go`: ```go // Lane is a class of model work with per-host defaults. type Lane struct { ID LaneID Summary string Adversarial bool // reviewer lanes: the model family must differ from the builder's Defaults []ModelDefault } … // Lanes lists the model lanes with their settled per-host defaults. var Lanes = []Lane{ {ID: LaneBuild, Summary: "Writes code and artifacts.", Defaults: builderDefaults()}, {ID: LaneSpec, Summary: "Authors specs, decisions, briefs, and copy.", Defaults: builderDefaults()}, {ID: LanePlan, Summary: "Slices and plans work.", Defaults: builderDefaults()}, {ID: LaneReview, Summary: "Adversarial code and spec review.", Adversarial: true, Defaults: []ModelDefault{ reviewDefault(HostAmp), {Host: HostCursor, Model: "Grok 4.6", ID: ModelGrokPrior, Family: FamilyXAI, Plan: PlanX}, {Host: HostCursor, Model: "Grok 4.7", ID: ModelGrok, Family: FamilyXAI, Plan: PlanLegacy}, reviewDefault(HostClaude), }}, … } ``` The `ModelDefault` type, from the same file (trimmed): ```go type ModelDefault struct { Host HostID Model string // display name, or the host mode (e.g. "Amp medium") ID string // model ID at the AI gateway or provider; empty for host-managed modes Family Family Effort string // reasoning effort to request; empty means the provider default Escalation string // what to use on escalation or for heavy work; empty if none EscalationID string // model ID of the escalation, when it is a model rather than a host mode … } ``` The budget defaults, from `internal/config/defaults.toml`: ```toml # A build's budget. cost_usd applies only when aayu's own model calls are pay-as-you-go (a [models.gateway] endpoint or a # direct provider's key is configured, or a host reports spend in AAYU_COST_USD); otherwise there is no cost limit, # since subscription use reports no cost. [autonomy.budget] wall_hours = 8 cost_usd = 60.0 attempts = 3 # Spend limits enforced by the model gateway's circuit breaker (WP1.6). [budget] per_story_usd = 60.0 # at least autonomy.budget.cost_usd, so the breaker never stops a story before its build budget per_day_usd = 100.0 per_pr_inspect_max = 5.0 ci_minutes_month = 2000 ``` The provider-neutral reply every wire format decodes into, from `internal/models/client.go`: ```go // Response is a provider-neutral reply. type Response struct { Text string Model string TokensIn int TokensOut int CostUSD float64 // reported by the endpoint, when it reports one; else 0 … StopReason string Truncated bool } ``` ## Invariants 1. Every lane has a default on every host; a default's model ID belongs to its declared family; an escalation never changes family; no default uses a family its host excludes; and an adversarial lane's default is never in the builder's family. Enforced by `internal/verbs/registry_test.go`, `TestRegistry`. 2. A configured model whose family the host excludes falls back to the registry default, and the substitution is logged. Enforced in `internal/models/resolve.go`, `TestLaneResolution`. 3. A reviewer's family differs from the builder's on every story, and a host-managed reviewer is refused. Enforced by `CheckFamilies` in `internal/models/resolve.go`, `TestReviewerFamilyMustDiffer`; called from `internal/cli/inspect.go`, `internal/cli/review.go`, `internal/cli/brief.go`, and `internal/evals/tier2.go`. 4. A host-managed mode is never called directly. Enforced by `RouteFor` in `internal/models/client.go`, `TestRouting`. 5. A reported cost wins over the price table, which wins over the fallback rate; an unpriced model is charged at 15 and 75 USD per million tokens. Enforced by `Cost` in `internal/models/gateway.go`, `TestBudgetBreaker`. 6. Spend that reaches a limit trips the breaker once, pauses agent lanes, pages once, and refuses every later call before it reaches the provider. Enforced by `Breaker` in `internal/models/gateway.go`, `TestBudgetBreaker` and `TestNotifyGatingAndDedupe`; exit 6 is mapped in `internal/cli/root.go`. 7. Each wire format sends the right body and reads tokens, stop reason, and truncation from its own reply shape. Enforced in `internal/models/client.go`, `TestWireFormats`, `TestStopReasons`, and `TestCacheTokensCount`. ## Where it lives in the code - [internal/verbs/registry.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/verbs/registry.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): lanes, hosts, families, model IDs, `DirectHost`. - [internal/models/resolve.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/models/resolve.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): lane resolution and the family check. - [internal/models/client.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/models/client.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): wire formats, routing, direct providers. - [internal/models/gateway.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/models/gateway.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): the gateway call, cost, and the breaker. - [internal/models/structured.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/models/structured.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): JSON replies validated against a schema. - [internal/cli/models.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/cli/models.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): `aayu models`. - [internal/config/defaults.toml](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/config/defaults.toml?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): `[budget]` and `[autonomy.budget]`. - [internal/secrets/secrets.go](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/internal/secrets/secrets.go?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): secret names and `AAYU_SECRET_*` variables. - [.github/workflows/models-live.yml](https://github.com/rahulnakmol/agentic-ayudha/blob/de79988060cd4c1522adb9e991f78cc9e58b9a68/.github/workflows/models-live.yml?utm_campaign=docs-site&utm_medium=docs&utm_source=agentic-ayudha): live calls through the AI gateway. > **Honest status:** this repository's only model key is the secret of the `ai-gateway` environment, which only the > `dev`, `rolling`, and `main` branches may use. Neither the founder's machine nor an agent's sandbox holds a key, so > `review` lane A and the `brief` adversarial pass run in the `aayudha-model-lanes.yml` workflow, and live gateway > tests never run on pull requests. Each process seeds the breaker from its clone's actions log when it builds the > gateway (`Breaker.SeedLog`), so the budgets count every process in that clone; a CI run starts from a fresh clone, so > its breaker sees only that run's spend. The breaker also cannot see what a host's own model spends inside a skill. ## Read next - [The eval harness](how-evals.md): the judge is a direct call through this gateway. - [The merge gate](how-merge-gate.md): where `inspect` makes its direct call.