The prompt carries judgment. The code carries the things that must never happen.
My prompt told the agent never to open a pull request without a committed change to the manifest. It opened one anyway.
Here’s how. The fixer agent I’ve been building hit a string of 404s from an API it was calling. It retried and got nowhere. It should have stopped. It opened an empty PR instead. The rule it broke sat in the system prompt in plain English. It read the rule. It broke the rule.
I could have written a sterner prompt, with all caps and more emphasis. I’ve done that before, and it holds until the model hits a state it hasn’t seen. So I moved the rule out of the prompt and into the tool. The function that opens a PR now compares the two refs and refuses when the branch matches the base. The model can’t open an empty PR, because the code won’t produce one. I reran the whole thing clean, and the fixer stopped the way it should have the first time: package not in the manifest, zero PRs.
That’s the pattern I’ve settled on. The prompt carries judgment. The code carries the things that must never happen.
What moved into code
I pushed down the constraints that would be a real problem if the model ignored them even once.
- The agent works on one repo and one branch. Both are fixed at start and read from the database. The model never picks either.
- It can edit the manifest and nothing else. The edit tool has a path allowlist and throws on anything that looks like source code.
- Every PR it opens is a draft. The draft flag is hardcoded. No code path marks a PR ready for review.
- It never auto-fixes a high-severity finding, and it works on one repo at a time. The code refuses both before any work goes out.
- On a red CI result, the code decides the next step. If the same failure happens twice, or the run goes past six attempts, the run halts and the card is marked blocked.
None of that lives in the prompt anymore. The prompt still does real work. It decides which override to use, when to narrow the scope, and what the smallest safe change is. That’s judgment, and the model is good at it. Judgment and hard limits do different jobs. You can recover from a wrong judgment call. A hard limit that fails once opens the empty PR, touches the wrong file, or ships a change nobody approved. Keep those limits out of a paragraph the model can reinterpret when it hits a state you never tested.
The same idea sits behind harness discipline at team scale. The structure around the model holds no matter what the model does. A smarter prompt doesn’t.
Too much confinement fails too
This is the part I didn’t see coming, and it turned out to be the more useful half.
Once the hard limits were in code, I got confident and pushed the same instinct into the triage rubric. Be conservative. When you’re not sure which repo a security ticket belongs to, don’t guess. Escalate to a human.
It sounded safe. I ran it against 50 real security tickets from our backlog. All 50 went to escalate-to-a-human. The gate worked exactly as written, and it was useless. I had built a system whose only output was “ask me.”
The tickets weren’t ambiguous. Most named the repo in the title. The agent had the answer in front of it. It lacked permission to act on an inference, because I’d told it never to guess. Under a rule that absolute, reading a repo name out of a title counts as guessing.
That’s the trap. A deny-by-default gate with no way to say yes moves all the work onto a person. It’s the same failure as an agent that fixes everything in sight, in reverse. It’s harder to spot because it looks responsible. Nothing broke. Nothing shipped either.
You need both halves
So the rule I run on now has two parts, and you need both.
Put the constraints that must never break into code, where the model can’t talk itself past them. Then put real effort into the inference layer that lets the gate say yes when it’s safe. Without it, the gate blocks everything and hands your backlog back to you. One half keeps the agent from doing damage. The other keeps it from doing nothing. Ship one without the other and you get an agent with no limits, or an expensive way to generate a to-do list with your name on every line.
Use the prompt to tell the model how to think. Use code to decide what it can never do, and what it may do without you. Get that line wrong in either direction and the agent makes more work for you.