Most agents that fail in production fail around the model, not inside it. The demo works because a person is watching every step. In production nobody is watching, so whatever you did not build around the model becomes the bug: a tool that cannot be retried, a context window full of old history, a permission nobody scoped, a release nobody tested.
I call that surrounding layer the harness. This is the checklist I use before I let an agent run on real work. It comes from building agents at work and in my own projects, including the local content studio that drafts posts for this site.
1. Tools: few, typed, safe to retry
An agent is only as good as the actions you give it. Anthropic’s guide to building effective agents argues that the agent-computer interface deserves the same care as a human interface: clear names, documentation, examples and testing. Their follow-up on writing tools for agents names the failure I see most often: too many tools, or tools that overlap, make the agent worse at choosing.
- Fewer tools, each with one job and a name that says what it does.
- Typed inputs, validated before the tool runs. Reject bad input with a message the model can act on.
- Every tool safe to call twice. Agents retry, and a tool that double-posts on retry will eventually double-post.
- Outputs trimmed to what the next step needs. Token efficiency is part of the tool’s contract.
2. Skills: load instructions when needed, not all at once
One giant system prompt is the easiest harness to build and the hardest to keep correct. Anthropic describes Agent Skills as packaged instructions loaded through progressive disclosure: the agent sees a short description of each skill and reads the full instructions only when the task calls for it.
- One skill per job (research, drafting, publishing), each with its own instructions and checks.
- Short descriptions that say when the skill applies, so the agent picks the right one.
- Shared rules such as voice, limits and banned sources live in one config file the skills read, not copied into every prompt.
3. Memory: decide what persists, and what must not
Context is a finite resource. Every token you load is paid for on every turn and competes for the model’s attention. I learned this the expensive way when my own CLAUDE.md grew into a 46KB project diary, which I wrote about in Your CLAUDE.md is probably costing you more than your prompts.
- Instructions persist. History does not. Logs, finished tasks and old specs go to files the agent reads only when asked.
- Load just in time: give the agent a path or a query, not the whole document.
- Write results to disk as you go, so a crash or a usage limit does not lose the work.
4. Permissions: read-only by default, allow-list the rest
Assume the agent will try everything it is allowed to try. Claude Code’s permission rules let you allow, ask or deny per tool and per command, and hooks let you run your own check before a tool call and block it. Use both.
- Default to read-only. Grant write, network and shell access per task, not per agent.
- Pass only the credentials a step needs. In my content studio the research step runs with an allow-listed environment that carries its own tokens and nothing else, and every secret lives in the macOS Keychain, never in a file or a log.
- Treat anything the agent reads from the web as data, never as instructions.
5. Budgets: cap runs, agents and fetches
Permissions stop the wrong action. Budgets stop the right action repeated until the bill arrives.
I learned this one the hard way too. A research run I started fanned out to 20 agents at once, some of them launching their own agents. It made 1,589 page fetches, hit my usage limit in about eight minutes, and lost everything, because no agent wrote its findings down until the end. The rules I run with now:
- Confirm the plan, the agent count and the model before any fan-out.
- No nested agents. Workers never launch workers.
- Cheaper models for workers. The expensive model plans and reviews.
- A fetch budget per agent, about 25, and one shared list of pages already fetched.
- Pilot two workstreams first, check usage, then launch the rest.
The studio applies the same idea to itself: at most three research runs a day, and only one at a time.
6. Approval gates: a person signs off on anything irreversible
The Model Context Protocol specification puts the floor plainly: hosts must obtain explicit user consent before invoking any tool. For anything that cannot be undone, the gate should go further and bind to exactly what will happen.
- List the irreversible actions: publish, send, pay, delete, merge, push. Each one waits for a person.
- Approval binds to the exact content. In the studio I am building, any edit after approval clears the approval.
- Show what the action will do, in plain words, before the person approves.
7. Evals: the release gate, not a dashboard
An agent that passed yesterday’s demo is not tested. Prompts, models and tools all change underneath it. Litmus, the evaluation platform I architected, scores every response against a 15-dimension framework and gates every release on the result. You can start much smaller, but the principle holds: a release ships when the eval suite passes, not when the demo looks good.
- A fixed set of real cases, including every case that failed before.
- Checks in code wherever code can decide (format, length, citations, banned content), and a model judge only for what code cannot check.
- A blocking gate in the release path, not a report someone reads later.
The checklist
Copy this into the pull request template for any agent change:
- Each tool has one job, typed input, and is safe to call twice.
- Instructions are split into skills that load on demand.
- History is out of the always-loaded context, and results are written to disk as the agent goes.
- Permissions are read-only by default, and credentials are scoped to the step that needs them.
- Runs, agents, model calls and fetches have budgets.
- Every irreversible action waits for a person, and approval binds to the exact content.
- The eval suite runs on every change and blocks the release when it fails.
None of this makes the model smarter. It makes the agent survivable, which is what production asks for.