Long-running agents accumulate state that no transcript captures. A coding agent at step 10 holds edited files, a running dev server, installed packages, and a warm prompt cache. When it misreads a traceback and rewrites a file that was already correct, neither available recovery path is cheap: patching forward grows the context and the token bill, and restarting from step one re-pays every model and tool call while reproducing nothing exactly, because runs are non-deterministic. Jumping back to step eight is the option engineers actually want, and it is the one existing runtimes cannot offer. Git versions files, not a live process or a cache. Researchers at Northeastern University and Stanford University have released Shepherd, a Python runtime substrate that records an agent run as a Git-like trace of typed events. Shepherd brings Git-like fork and replay to agent runs, solving exactly this problem.
What Is Shepherd? A Git-Like Runtime Substrate for Agent Processes
Shepherd is a Python runtime substrate that records an agent’s execution as a first-class object. Every agent-environment interaction becomes a typed event in a Git-like execution trace. Core operations are formalized as functions and mechanized in Lean. This allows any past state to be forked and replayed, complete with the live process, filesystem state, and prompt cache. Unlike Git, a commit in Shepherd covers the agent process and the filesystem together, copy-on-write. A branch therefore carries live state, not just files. Returning to an earlier point is a single fork from that commit. The research team reports that Shepherd forks the agent process and its filesystem 5x faster than Docker. Because the prompt prefix through the branch point is unchanged, replay achieves over 95% prompt-cache reuse.
The Limits of Existing Recovery Strategies: Why Docker and Git Are Not Enough
The immediate question from any engineer familiar with Docker or Git is straightforward: why not just checkpoint the container or revert the file changes? The answer reveals a fundamental gap in the existing tooling. Docker checkpoints capture the entire container state, but they are slow, coarse-grained, and expensive to store. They were not designed for sub-second branching at every step of an agent loop. Git, on the other hand, versions files perfectly, but it has no awareness of a running process, an active virtual environment, or a language model’s prompt cache. A transcript records what the agent said and which tools it called, but it does not record the live process, the dev server, the installed packages, or the prompt cache. When a run goes off the rails, engineers are left with two bad options: patch forward with an ever-growing context window and token bill, or restart from scratch and hope for a better outcome. Shepherd introduces a third path: fork from the last correct commit and continue with full process state and minimal cost.
How Shepherd Works: Tasks, Effects, Runs, and Workspaces
Shepherd organizes agent execution around four core concepts: tasks, effects, runs, and workspaces. A task is a typed function whose body the model fills in. The signature serves as the contract between the agent and the runtime. An effect is every crossing of the task boundary. It can be watched, answered, or refused. A run is the durable record of those crossings. The workspace is the copy-on-write filesystem that follows the branch.
Permissions are declared in the signature. A May[GitRepo, ReadOnly]codecode binding, for example, is compiled to that run’s writable roots and enforced at the native syscall jail. This means the agent cannot accidentally write to a path it was not granted access to, even if the underlying model attempts to do so. The syscall jail runs on macOS using Seatbelt and on Linux using Landlock inside a privileged container. This gives operators granular control over what an agent can touch, at the level of the operating system.
Because every interaction is a typed event, the trace is a first-class object. It can be inspected, branched, and used as a dataset. This opens the door to higher-order agents that observe a trace and intervene before a bad write commits. The research team demonstrated three applications: runtime intervention, counterfactual meta-optimization, and Tree-RL training.
Forking Process State: The Copy-on-Write Advantage
The key technical innovation is the copy-on-write (CoW) fork at the process level. When Shepherd forks a run at step eight, it does not copy the entire memory space of the agent. It shares the parent process pages until the child writes to them. This is the same mechanism that makes Unix fork fast, but applied to the full stack: the agent process, the filesystem, the network ports, and the prompt cache. Because the prompt prefix through the branch point is unchanged, the next call to the language model hits the cache perfectly. The research team reports over 95% prompt-cache reuse on replay. This is the economic engine behind Shepherd: rewinding does not re-pay the prompt prefix.
Concrete Gains: Live Supervision, Counterfactual Optimization, and Tree-RL Training
Shepherd is more than a debugging tool. It is a substrate for building reliable, observable agent systems. The research team put it through rigorous benchmarks with results that speak directly to the bottom line of agent deployment.
Runtime Intervention: Paired Coding with a Supervisor Agent
In the runtime intervention experiment, a live supervisor agent observed the trace of a primary coding agent. When the primary agent was about to write a bad file based on a misread traceback, the supervisor intervened. On the CooperBench pair-coding benchmark, this simple loop raised the pass rate from 28.8% to 54.7%. The supervisor did not need to re-run the primary agent. It inspected the events, identified the bad write, and forked the run to a safe checkpoint. This is the “undo” button for agent actions, implemented in real time.
Counterfactual Meta-Optimization: Exploring Branching Strategies
Counterfactual meta-optimization allows an operator to fork an agent run at a specific step, try a different tool call or prompt variation, and compare the outcomes side by side. The research team found that this branching exploration beat baselines across four benchmarks by up to 11 points while cutting wall-clock time by up to 58%. The 58% reduction comes from not having to restart the agent from scratch for each trial. The prompt prefix and early process state are reused, so each alternative branch costs only a fraction of a full run.
Tree-RL Training: Generating Rollouts from Real Traces
Reinforcement learning for language agents, often called RLHF for tool use or Tree-RL, requires high-quality rollout trajectories. Shepherd makes this process dramatically more efficient by forking rollouts at selected turns instead of generating each trajectory from scratch. The research team improved TerminalBench-2 from 34.2% to 39.4% using Tree-RL training on forked rollouts. This is a 5.2 percentage point gain on a complex terminal-based benchmark, achieved without expensive human annotation or brute-force exploration.
Deployability and Maturity: Early Alpha With a Clear Roadmap
Shepherd is available now in early alpha. It is MIT-licensed and installable with pip install shepherd-aicodecode from PyPI. It requires Python 3.11 or later. OS-level grant enforcement runs on macOS using Seatbelt and on Linux using Landlock inside a privileged container. The research team has clearly stated that it is not ready for production, but the foundation is solid. The Lean mechanization of core operations provides a formal correctness guarantee that is rare in infrastructure software. For teams building long-horizon agent systems, Shepherd is worth installing and experimenting with today. The common trait is not a specific vertical. It is long-horizon agent runs against heavy sandbox state, where a failed run is expensive to redo. Industries that should be watching closely include software engineering and DevOps, AI infrastructure and agent-platform vendors, quantitative finance research, security tooling and offensive-security research, and data engineering.
The Boundaries of Stateful Forking: What Shepherd Cannot Undo
It is important to understand what Shepherd does and does not do. Forking undoes filesystem and process state inside the sandbox. It does not undo the outside world. A sent email, a real charge, or a third-party write stays sent. Shepherd’s answer to external side effects is refusal rather than rollback. Every crossing of the task boundary is an effect that a handler within scope can watch, answer, or refuse before it happens. This is a defensible architectural choice. Attempting to undo external state is often impossible or dangerous. Refusing a dangerous action before it executes is strictly better than rolling it back after the fact. Teams adopting Shepherd must design their agent tasks with this boundary in mind.
Shepherd introduces a primitive that the industry has sorely lacked: a reliable, deterministic rewind button for live agent processes. The implications extend far beyond fixing a bad code write. When every action is versioned like a commit, agent orchestration becomes a branch management problem instead of a prompt engineering guessing game. Meta-agents can observe, intervene, and optimize with surgical precision. Reinforcement learning can generate diverse, high-quality rollouts from a single real trace. The economics of this are compelling: 5x faster forks than Docker and over 95% prompt-cache reuse on replay mean that the cost of recovery and exploration drops dramatically. As agents grow more autonomous and long-running, the ability to manage their state with the rigor of version control is not just an advantage. It is a requirement. Shepherd is an early but convincing answer to that requirement, and it is available to the community today.
