Shishir Pai

building a harness workflow from first principles

how i actually run a multi-agent coding harness.

· 9 min read

supership []

llms are trained on the output of humans. so if you want to get the most out of an agent, the move is to step back and copy how development actually happens.

when you build a feature you start on top of an existing codebase. maybe you know it, maybe you have to go read it. call that exploration. then you scope the work.

  • what's the outcome
  • what can i reuse
  • is there already a library for this
  • do i even need a new dependency
  • how long will it take

that's planning.

then you actually write the thing, which is mostly mechanical until you fall into a debugging rabbit hole with a dozen stackoverflow tabs open and the error pasted into chatgpt, and sometimes that sends you back to change the plan.

finally you review

  • does it compile
  • are there tests
  • is there an edge case
  • does it actually work!?

if it's good you open a pull request, someone reviews it, you make changes, and you go around in a loop until it merges. that whole loop is the thing to reproduce. not "prompt an agent to be smart" but the actual shape of how work gets done.

a model on its own only predicts text. it can't read a file, run a command, or check its own work. the harness is the software wrapped around the model that lets it do those things. it hands the model its tools, feeds it context, runs whatever the model decides to do, and loops the result back in.

most people never set this up. it's tempting to open one agent and tell it to act as an orchestrator. but a job title doesn't change how the agent works. it still has one context window. every file it reads and every error it pulls lands in that same window. you gave it a title, not a second set of hands, and the bloat never gets delegated.

real delegation needs three things. the system has to spawn a separate context the parent doesn't share. it has to hand that context a small slice of work with only what the slice needs. and it has to take back a clean result, so the parent gets the conclusion and not the hundred tool calls behind it. prompting one agent to "orchestrate" gives you none of that. without a setup that can spawn real subagents, nothing below works.

a smart agent drives the harness. it barely touches a file. its whole job is to decompose, delegate, and decide. context is scarce, and quality matters more than size. you don't want the least context, you want the least sufficient, correct context. too much and the window floods and the model forgets. too little and it hallucinates to fill the gap. delegation is how you hold both ends. the detail-heavy work goes to subagents, each handed a focused slice that returns only a distilled result, and the driver stays lean and reasons over conclusions.

you kick it off with a task and as much information as you can give it. the best way i've found is to just talk, ramble into voice for as long as you can. intent is the one piece of context the agent can't fetch for itself, so everything you leave unsaid it has to guess. it should grill you first, until it has enough clarity that it won't make an awkward call on your behalf.

bar lengths are illustrative, as in the diagram.

bar lengths are illustrative, as in the diagram.

then it decomposes. it takes what you asked and breaks it into the questions that have to be answered before a line gets written.

to answer them it spawns explorers. same exploration step as before, except no human is doing the sitting. cheap, fast subagents fan out across the codebase. one traces how the relevant module flows end to end. one checks whether a helper or a pattern already lives a few files over. one goes to the internet, reads the docs for a library you're weighing or compares two of them, and comes back with the exact answer. each works in its own clean context and hands back a small, dense summary. if the driver did this itself its window would already be full of tool calls and irrelevant search results. instead it just gets the conclusions.

now it can plan for real. it knows what exists, what's reusable, and what actually has to be built. it decides whether a new dependency earns its weight or a few lines of the standard library will do. it decides whether the work is one straight line of edits or several independent pieces. most work is sequential and one worker is enough, so it doesn't over-decompose. once in a while the work splits cleanly, and then each piece gets its own isolated space (git worktrees) so parallel edits don't collide. the output is a roadmap.

this is where the mechanical work happens, and it's deliberately boring. the plan goes to worker agents whose only job is to make the change. they don't need to be as smart as the planner, because the thinking is already done. each gets a fresh window and the smallest slice of context that lets it finish. when a worker needs a fact from outside it doesn't go browsing and drown, it sends a scout, the scout returns the one fact, and the worker keeps building lean.

every real implementation eventually hits a failure that won't yield to the obvious fix. the human version is a dozen tabs and a paste into chatgpt. the worker does the disciplined version. instead of thrashing for another hour it stops and writes down three things, what it tried, the exact error, and the precise question it can't answer on its own. that goes up to the harness, and the harness decides who to call.

not every stuck is the same. some are mechanical, the code breaks and the worker can't see why. that goes to a debugger, a slow and expensive model whose only job is root cause. it reads the report, digs through the code with its own scouts, and comes back with the cause, the fix, and how to verify it.

some aren't bugs at all. the design is ambiguous, or the plan is just wrong for that piece. think about what a junior does here. they debugged, they searched, nothing worked, so they walk over to whoever designed the thing. the worker does the same. the question goes back to the planner, the same architect that wrote the plan, handed its own plan plus what was tried and what's unclear. it clarifies the intent, adjusts the piece, or cuts it. sometimes the plan was wrong and it says so. this is where "the plan changes during implementation" actually happens.

then review, the pull request loop without waiting a day for anyone to free up. reviewers fan out in parallel, each with a different lens, one for correctness, one for security, one for edge cases, one hunting over-engineering. they run on different models on purpose, because different models disagree in useful ways and catch what a single family misses. each hands back structured findings.

then judgment, because reviewers produce nits, duplicates, and false positives just like people do. the findings go to a judge that decides what's real and worth fixing and what's noise. the accepted issues go to fresh workers, the fixes land, the reviewers run again. review, judge, fix, verify, around and around until a round comes back clean. same loop you run with coworkers on a pull request, compressed from days to minutes.

a lesson shouldn't die with the subagent that learned it. every agent finishes by passing up what it got wrong. each parent folds its children's lessons into its own and hands the bundle up, so by the end the driver holds everything the whole tree learned. that gets written to per-repo memory, and the next session starts already knowing the traps. the harness gets a little smarter every time you run it.

decompose, delegate, decide. the expensive model spends its tokens on judgment, the cheap ones do everything else, and no context window gets big enough to go dumb. you don't touch it until it converges.

none of the above is hypothetical. it runs. the tool is supership, a workflow kit for oh-my-pi, public and MIT, so you can read it instead of taking my word for any of this.

https://github.com/mgpai22/supership

two commands. /supership grills you, plans, lets you approve the plan, then builds and reviews. /shipit is the same pipeline with no interview and no gate.

the thing i was most wrong about at the start is the orchestration itself. it is not a prompt. the driver doesn't "act as an orchestrator," it writes python and runs it, agent() and parallel() calls with real control flow inside an eval cell. a loop is a loop, not a paragraph asking the model to please keep looping. the whole pipeline is code the main agent authors at depth 0, and the recursion cap keeps anyone from quietly nesting an orchestrator inside an orchestrator.

state lives in one plan.html that renders itself. open it in a browser and it live-refreshes while the run goes. the file is the source of truth, not the agent's memory, so if the session dies you resume where it left off.

the abstract roles above are real agents, each pinned to the model that fits.

agent runs on what it does
planner biggest reasoning model the architect. builds the clarify question tree, writes the plan, and adjudicates when a worker is stuck on a design call.
task mid model the worker. makes the actual change, and returns "stuck" instead of thrashing.
david-research cheap, flash the scout. fetches one fact from the web, docs, or a repo and returns just the answer, so the driver context stays clean.
deep-debugger biggest, slow root cause. takes a stuck worker report and returns the cause, the fix, and how to verify it.
deep-reviewer varied per lens the reviewers. one per lens (correctness, security, edge cases, over-engineering), each on a different model so they disagree usefully.
judge biggest keeps the findings that are real and in scope; drops the nits, dupes, and false positives.

[back to writing]