When a model says “run npm test”, no test runs. What actually runs it is the software wrapped around the model: the harness, or control layer. Suppose a harness handles that request like this. It first checks whether npm test is on its list of allowed commands, then runs it in a subprocess with a five-minute limit. When the run ends, it hands the exit code and the last 200 lines of output back to the model and writes the same record to its log. If the process crashes midway, the next attempt resumes from that log entry.
In that one step the model only proposed the next move; the harness made the move executable, kept it within bounds, left a record to inspect afterwards, and made it recoverable after failure. A harness is therefore more than a chat interface: it connects the model, repository, terminal, permissions, and verification workflow.
The table below places all eight responsibilities on the same npm test request.
Layer
Responsibility
What it does for npm test
Loop
Alternate model turns, tool calls, results, and stopping conditions
Hand the test result back so the model can choose the next step; stop when tests pass or the step budget runs out
Context
Select instructions, history, files, and retrieved evidence
Put only the last 200 lines of output into context, not the whole log
Tools
Validate arguments, execute calls, and return structured results
Check that the command and working directory are valid, return the exit code and output; a subprocess inherits its parent’s environment variables and open resources (see Processes, Address Spaces, and Memory), so decide first what it may carry
Policy
Enforce scope, permissions, approvals, budgets, and safety boundaries
Check the allowlist: npm test may run directly, while a command that deletes a directory or pushes code must stop and ask a person
State
Preserve task progress, artifacts, and resumable checkpoints
Record that this step ran the tests and what the result was, so work can resume here after a crash
Reliability
Handle timeouts, retries, idempotency, cancellation, and failures
End the process when five minutes are up; rerunning tests changes nothing, so a retry is safe
Observability
Retain traces, logs, token/cost data, and tool evidence
Log the command, duration, and exit code, so anyone can later see what ran and when
Evaluation
Test outcomes on representative tasks instead of judging prose alone
Check whether the tests actually passed, not whether the model sounds finished
“All tests pass” from the model does not count: a dependable harness reads the exit code of npm test itself and then decides whether to stop. How to wire that kind of external acceptance into the loop is covered in Bounded Agent Loops.
Define the outcome and an external success check, such as “npm test exits with 0”.
Expose the smallest useful tool set. Small local models depend on this step even more: iterations should be short, the tool set narrow, and output length capped.
Separate read-only actions from writes and irreversible actions.
Preserve artifacts and evidence outside the conversational transcript.
Add time, token, retry, and concurrency limits; stop conditions and the circuit breaker are detailed in Bounded Agent Loops.
Trace model turns and tool calls with sensitive-data controls.
Replay representative tasks before changing the default model, harness, or reasoning level.
The checks that decide whether a result can ship, such as tests, permissions, and end-to-end acceptance, belong to the harness and workflow; the model is replaceable. Fast-changing model and pricing choices belong in the Model & API Radar, and public test results and tool-specific failure modes live in Choosing an AI Coding Agent. A stronger model may reduce retries, but it removes none of the need for evidence, permissions, and end-to-end checks.
Where These Controls Live in Claude Code, Codex, and Pi
A coding agent you install already implements most of the responsibilities above; what you adjust are its switches. The table below follows the official documentation as of 2026-10-29, and Pi 1.4.0.
Control
Claude Code
Codex
Pi
Project instructions
CLAUDE.md; it can also read the repository’s AGENTS.md
AGENTS.md: ~/.codex/AGENTS.md first, then each level from the repository root down
AGENTS.md and CLAUDE.md always load, regardless of project trust (security doc)
Permissions and approval
/permissions, or permissions.allow / ask / deny in settings.json, evaluated deny → ask → allow
approval_policy (such as on-request or never) and /permissions; approvals_reviewer = "auto_review" hands approvals to a reviewer agent
No prompt before tool calls; --approve / --no-approve only decide whether project settings and extensions load
Sandbox
Off by default; turn it on with /sandbox or sandbox.enabled; it wraps shell commands only
sandbox_mode: read-only, workspace-write, danger-full-access; workspace-write has no network by default
No built-in sandbox; run the whole of Pi in a container or virtual machine
pi -c continues the most recent one; pi -r opens the session list
The same word “sandbox” covers different ground in each of the three; Claude Code’s sandbox, for example, can still read ~/.ssh once it is on. Before changing anything, read the comparison and starter configuration in Agent Security.