← All case studies

AI Engineering · Cross-provider review, routing, and restraint

How I Work With AI Agents

What thousands of hours of building with AI agents taught me: every model has blind spots, so a reviewer from another provider checks the work; frontier models orchestrate while cheaper ones implement; and the simplest setup usually wins.

The Project

I have spent thousands of hours building with AI coding agents, at work and on my own projects. I am a staff engineer without AI. What AI changed is how much ground I can cover, and that only works if my attention goes to the decisions that matter instead of re-checking routine output. This page is the working practice I have settled on, with numbers from one real session: the one that rebuilt this portfolio for two tracks of roles.

Every model has blind spots

Every model has biases and gaps it cannot see in itself. So a reviewer from a different provider checks every change, and on risky work, two. It costs more per change and far less than thrashing on a solution with holes in it.

In the session that rebuilt this site, an Anthropic model led, an OpenAI model reviewed every commit, and a third provider reviewed the riskiest content. Across ten reviews they surfaced real issues, from a test suite that would break on an older Node version to a security claim that held for script tags but not inline event handlers. The two reviewers caught different problems, which is the point.

Reviewers check the work; they do not own it. Findings are claims to verify against the code, not instructions, and several were rejected with reasons. And the most important correction in that session came from me, because it was a fact no model could know: how my own teams are structured.

Frontier models orchestrate, cheaper models build

I use the frontier models, currently Opus and Sol, as orchestrators and as the interface to me. They scope the work, delegate it, and check what comes back. Most implementation goes to cheaper models, currently Luna and Haiku, running at high reasoning effort. The orchestrator escalates to a bigger model only when a stage needs it, and that is rare.

In the portfolio session, eight delegated runs on the smaller model handled the implementation, job-title research, and six case-study drafts. None needed to be escalated. The orchestrator checked each result, and everything that shipped also went through cross-provider review. In my experience, the mid-priced tiers in between rarely earn their cost against a small model run hard or a large model run light.

Simpler beats clever

My setup once got complicated: lots of hooks, a task tracker wired into the agents, and a routing system I was growing into its own tool, Guild Hall, to send each task to the right model. Most of it did not last. The labs are improving their harnesses so quickly that any edge I built was soon matched in the product. A terminal-first setup I had invested in became tedious next to the native apps once they added computer control and built-in browsers.

So I learned when to say no. I stay close to the vanilla harness, with small tweaks. My agent instructions file is now short. It condenses what I have learned into the few rules that keep changing outcomes, like the review rule above.

Sensitive work stays local

Anything sensitive runs on local models through Ollama and LM Studio, currently Gemma 4 and the newest Qwen models. Hosted models handle the rest. My daily tools are Claude Code, Codex, and OMP as harnesses, with Herdr, Ghostty, and Neovim around them, plus two pieces I built: Harness Deck to review agent output across harnesses and herdr-ask for terminal commands that never run without me.

Wrap Up

The pattern is the same one I use with teams: give the work to whoever is best suited to it, verify it independently, and keep the process as simple as the risk allows. Cross-provider review does not replace my judgment. It clears out the obvious problems so I can spend my attention on architecture, risk, and whether the result is actually right. AI is the ultimate automation tool, and the human in the loop is the one part I would never automate.