Swapping the brain - building agents that aren't married to one model

Every few weeks a new frontier model lands, someone benchmarks it against the last one, and a Slack thread starts with "should we switch?" If answering that question means rewriting your agent, you've built the wrong thing. The reasoning model is the most replaceable part of an agent — the part that changes fastest — so it should be the easiest to swap, not the hardest.

Here's how to keep an agent model-agnostic: a thin boundary the model sits behind, an evaluation harness that measures the things that actually matter for agent work, and a set of guardrails that don't move when the model does.

Why you don't want to be married to one model

"Just use the best model" sounds fine until you notice how many axes "best" moves on, and how often:

• Price. A model that's 5× cheaper at the same quality resets your unit economics overnight — but only if you can adopt it without a migration.
• Capability. One model is better at long-context document work, another at tight tool-calling, another at code. A multi-agent system rarely wants the same brain in every seat.
• Availability & latency. Rate limits, regional capacity and outages are real. A fallback model is a reliability feature, not a nice-to-have.
• Data residency. "This workload must run in-region, on this provider" is a contractual requirement for enterprise clients, and it varies per client.

None of those are prompt problems. They're architecture problems — and the fix is to treat the model as a component with a contract, not a hardcoded dependency.

The model boundary

Define the smallest interface your agent actually needs, and make every model implement it. For agent work that's four things: turn-based chat with tool-calls, structured output you can parse, streaming for responsive UX, and honest token accounting for cost control.

 // Model is the only surface the agent loop knows about. Gemini, Claude, // Kimi and GPT each get an adapter; the orchestration code never imports // a vendor SDK directly. type Model interface { // Generate runs one turn: messages in, either a text answer or a set // of tool calls out. The agent loop decides what to do with tool calls. Generate(ctx context.Context, req Request) (Response, error) // Stream is the same turn, emitted incrementally for the UI. Stream(ctx context.Context, req Request) iter.Seq2[Chunk, error] // Name + pricing let the router and the cost guard reason about a // model without special-casing it. Info() ModelInfo } type Request struct { Messages []Message Tools []ToolSpec // JSON-schema tool definitions (MCP-shaped) Schema *jsonschema.Schema // when set, force structured output MaxTokens int } type Response struct { Text string ToolCalls []ToolCall Usage Usage // input/output tokens, cache hits — for cost } 

The win here isn't the interface itself — it's that everything above it (the agent loop, human-in-the-loop, tool routing, audit) is written once, against Model, and never learns which vendor answered. Swapping a model becomes a config change plus an adapter, not a refactor.

Evaluate for agent work, not for benchmarks

Leaderboard scores tell you almost nothing about whether a model will run your agent well. MMLU doesn't measure whether the model calls the right tool with the right arguments, recovers from a tool error, or produces JSON that parses on the first try. So we don't grade models on trivia — we replay a fixed set of real tasks and score the outcomes.

• Task completion. Did the agent reach the correct end state? Graded by a rubric, sometimes by a second model acting as judge.
• Tool-call accuracy. Right tool, right arguments, no hallucinated parameters. This is where cheap models most often fall down.
• Structured-output validity. Does the output validate against the schema without a repair pass?
• Cost per solved task. The number that actually matters — not price-per-token, but dollars per correct outcome. A "cheap" model that needs three retries isn't cheap.

 // One eval case = a frozen scenario + a grader. We run every candidate // model through the same cases and compare on the metrics above. type Case struct { Name string Seed []Message // the setup + user request Tools []ToolSpec Grade func(final AgentResult) Score // task-specific rubric } func Run(ctx context.Context, m Model, cases []Case) Report { var r Report for _, c := range cases { res := agent.Run(ctx, m, c.Seed, c.Tools) s := c.Grade(res) r.Add(Result{ Case: c.Name, Passed: s.Passed, ToolAccuracy: s.ToolAccuracy, CostUSD: res.Usage.CostUSD(m.Info().Pricing), }) } return r // pass rate, tool accuracy, and $ / solved task per model } 

Freeze the cases in the repo and the harness doubles as a regression test: when a provider quietly updates a model behind the same name, your suite catches the behaviour change before a client does.

A concrete swap: Kimi K2 as a cheaper sub-agent

Not every seat in a multi-agent system needs your most capable (and most expensive) model. In a research workflow, the orchestrator does the hard planning, but the sub-agents doing fan-out retrieval and summarisation are doing narrower work. That's exactly where a cheaper, strong open model earns its place.

Run an open model — Kimi K2, say — through the harness as the summariser seat against your default, and you'll typically find it loses a little on multi-step planning (so it stays out of the orchestrator seat) but holds its own on constrained summarisation at a fraction of the cost. The result: a router that picks the model per role, not per system.

What must not move when the model does

Model-agnostic doesn't mean anything-goes. The guardrails are exactly what let you swap freely, because they don't trust the model in the first place:

• Budget caps. A per-run cost ceiling that halts the loop, independent of which model is answering. See letting an agent act as you — safely for how this sits next to approval and audit.
• Output validation. Structured output is validated against the schema and repaired or rejected — never trusted because "the good model usually gets it right."
• Human-in-the-loop. Approval gates on any write are a property of the architecture, not the model. Swapping the brain never widens what an agent can do unsupervised.

Get this boundary right and the "should we switch?" thread stops being scary. You run the new model through the harness, look at pass rate and cost per solved task, and flip a config value for the seats where it wins. The agent — the part your clients actually depend on — never notices. That's the point: durable systems outlive whichever model is winning this month.