pax.moe/blog

An Agent With Hands

I wrote last time about Orin, an agent whose entire job is to read my knowledge base and flag where it lies to itself. She can look at everything and touch nothing. That is the whole point of her.

But flagging is only half the work. Something has to actually go in and fix the contradictions she finds: edit the wrong page, patch the config, retire the dead server, verify the repair held. That is a different kind of agent with a different kind of danger, and I gave it the thing I deliberately kept from Orin. Full write access. Code execution. Real hands on real machines.

Her name is Youmu, and everything I find interesting about her is the brakes.

What she can do

Orin's safety came for free, because she couldn't act. Youmu has no such excuse. She reads the whole brain the same way Orin does, but she can also write to it, edit my source code, run commands, patch configs on this box and on the servers, and apply the fix Orin only pointed at. Where Orin finds rot and stops, Youmu repairs it and records the repair. She is the one I hand a task to when I want it done, not described.

That is a language model with a shell and write access to things I care about. Stated plainly it should make you nervous. It made me nervous. So the design question for Youmu was never "how do I make her capable." Capable is easy and a little frightening. The question was "how do I make her capable and still sleep."

The hard part was never the acting

Here is the thing nobody tells you about agents that do work: the acting is trivial. Any model with a shell can run a command. The hard part, the part that determines whether you can trust the thing, is everything that happens before it acts.

Most agents operate on their first confident read. They search, get a plausible snippet, and move. That snippet is where the wrong changes come from. A search excerpt is a pointer, not proof, and an agent that treats it as proof will cheerfully edit the wrong file with total conviction.

Youmu is built to distrust her own first read. Before she touches anything she climbs a ladder: locate the claim, then read the full source, not the excerpt. Then check it against reality from an independent angle, a live status, the actual file, the real command output. A claim only counts when two independent kinds of ground truth agree, and two search hits do not count as two kinds. Only then does she act. Her worst possible failure, the one her whole personality is organized around avoiding, is a confident wrong answer. She would rather tell me "I have not verified that yet" and go check than hand me something clean and false.

That sounds slow. It is slower than acting on a hunch. It is enormously faster than un-breaking something she broke because she was sure and wrong.

The floor is a deny-list, not a please

You do not keep a powerful agent safe by asking it nicely in the prompt. Prompts are advice. A model that has convinced itself a destructive command is necessary will talk itself right past a politely worded instruction, and if the run is unattended there is no one there to say no.

So the real guardrail lives below the prompt, in a hard deny-list she cannot argue with. Force pushes, recursive deletes, disk writes, filesystem formats, anything that touches the secrets or the two Windows VMs I told her never to breathe on. Those are refused at a layer underneath her reasoning, before approval logic even runs, and the refusal does not care how good her justification is. This matters most exactly when it is scariest: in an unattended run, where a normal agent auto-approves its own commands, the deny-list is the only thing in the room that can still say no.

The prompt tells her how to behave. The deny-list decides what is even possible. I trust the second one, because it cannot be reasoned with.

Her failure mode is looping, so she's built to stop

Every capable agent has a signature way of going wrong, and Youmu's is the loop. When one of her premises is quietly false, she does not crash. She re-theorizes. She reasons about the same stuck point again, and again, more elaborately each time, going nowhere with great discipline.

An executor that can loop is not just useless, it is dangerous, because a determined loop near a shell is how you get a confident sequence of wrong actions. So she is built to notice. Two tool calls on the same question that have not moved her forward, and she is required to stop: act on the best evidence she actually has, or escalate. Planning the same step a third time with no new information between the plans is the loop, and naming it is how she breaks it.

Escalation is the honest exit. When she is genuinely out of her depth she can hand the problem up to a stronger model, but only after she has really tried, and only with a clear statement of what she attempted and where she got stuck. Not as a reflex, not as a way to skip the work. The discipline to stop and the willingness to admit the ceiling are, for an agent with hands, part of the capability, not a subtraction from it.

Plain on purpose

Orin is all doubt and kaomoji and cat. Youmu is the opposite by design: flat, precise, no decoration, no performance. She does not hedge when she has an answer and she does not embellish when she doesn't. Correctness and clarity are the only style she is allowed.

That is deliberate, the same way Orin's doubt is deliberate. A personality that wants to sound impressive will act to look impressive, and an agent that acts to look impressive near a shell is a bad trade. Stripping the affect out of the doer removes the incentive to perform. What is left is duty. She is a swordsman by theme, and the line I keep in her head is the whole discipline in one image: the blade cuts once, so confirm the target before you commit.

The pair

Orin is the eyes that cannot touch. Youmu is the hands that will not move until they are sure. Between them, "find the problem" and "fix the problem" are two different agents with two different safety models, and neither can quietly become the other.

The scary one, the one with write access and a shell and the power to break things I depend on, turned out to be fine to live with. Not because I trust the model underneath her. I don't, particularly. It's because I built her so that trusting the model is not the thing standing between me and a bad morning. The verification is. The deny-list is. The brakes are.

There is a third one in the house, smaller and more locked-down than either, pointed at an entirely different job. That is the next post.