I put an AI agent on our board

Part 1 of 5 in a series on autonomous software delivery. This post introduces Ralph, our experimental autonomous development agent, and the thesis of the series: agent autonomy is an engineering problem, and it needs engineered frameworks, not vibes.

Last month, a colleague typed a comment on an Azure DevOps work item:

@ralph the export button downloads an empty file when the filter is active. Fix it.

Forty minutes later, the same work item had a new comment. It described the root cause, linked a pull request that had already been merged and deployed to the integration environment, listed which kinds of tests had run, and showed a screenshot of the export working, with the filter active. No engineer had opened an IDE. The human contribution to that bug fix was one sentence and, later, one judgment call: does the running feature do what I asked?

That agent is called Ralph. It is experimental, it is opinionated, and it has been quietly delivering real changes to real repositories in our organization for weeks. This series is about what we built around it, because the agent turned out to be the easy part.

The dumbest loop that could possibly work

The name comes from the Ralph loop: the technique, named after Ralph Wiggum, of running a coding agent in a loop that re-reads the same prompt on every iteration until the work is done. In its purest form it is five lines of bash: while :; do cat PROMPT.md | claude-code; done. Deterministically bad, in an undeterministic world.

The dumb loop is smarter than it looks, for one reason: context rot is the enemy. As an agent session fills up, output quality drops and details get lost to compaction. The loop sidesteps that entirely. Every iteration starts with a fresh context window and reloads the spec, the instructions and the plan from disk. Progress lives in files, a TODO list and Git history, not in the model’s memory. What keeps it honest is back-pressure: tests, type-checks, lints and builds that reject invalid work, and specs with testable acceptance criteria that tell the loop when it is actually done. The payoff is reliable convergence at machine scale.

But! A Bash loop on a laptop, pushing to main with permissions switched off, is a brilliant way to build a greenfield side-project. It is also nowhere near something you can run against a customer’s repository. Ralph is what happened when we asked: what does the Ralph loop need around it to be enterprise-grade?

What Ralph actually is

Ralph is not a chatbot with a Git plugin. It is a small service that owns exactly one thing: the lifecycle of an autonomous development run. Everything else is delegated, deliberately:

• The agent is a standard coding agent (think Codex or Copilot CLI) speaking the Agent Client Protocol, a JSON-RPC standard for driving coding agents. Ralph can swap harnesses without changing a line of workflow.
• The tools come over MCP, the Model Context Protocol, and as plain CLIs. The agent gets an Azure DevOps MCP server (work items, wiki), a Playwright MCP server (a headless browser, for taking evidence screenshots of the running app), and the ordinary command line: git and friends. The board is swappable too: Azure DevOps today, Jira or GitHub or whatever else you can attach tomorrow.
• The models come from whichever provider the harness is configured for.
• Ralph itself does the plumbing: receive the webhook when someone comments @ralph …, clone the target repository, overlay the agent’s instructions and tool config, start one session, and report what happened.

One principle carries the whole architecture: ACP standardizes the agent, MCP standardizes the tools, the providers supply the models, and Ralph owns the lifecycle. Every piece is replaceable except the discipline around it.

Perhaps surprising: Ralph doesn’t have a workflow in code. There is no state machine that says “first refine, then implement, then test.” The entire operating procedure lives in prompts: what to do with a refinement request, when to open a PR, how to report results, what counts as done. They are skill documents the agent reads inside the session, versioned in Git like everything else. When the workflow is wrong, we edit a Markdown file, not a service.

The discussion thread is the state

Ralph keeps no database of runs. The work item’s discussion thread is the state:

• A human comment starting with @ralph is a command.
• A signed result comment from the agent, newer than the command, means the command is answered.
• While a run is active, a status comment holds a link to the live session, and the agent updates it at milestones.
• If a run dies, the next sweep notices an unanswered command and reports the failure, on the thread.

This sounds like an implementation detail. It is actually the design decision I defend hardest. Because the thread is the state, every decision, human or agent, is in one auditable place, in chronological order, written in language a product owner can read. There is no admin panel where the “real” truth lives. If Ralph’s server disappears tomorrow, the complete history of what was asked, what was built, what evidence was produced, and who approved it survives on the board, where the team already works.

And there is a subtler benefit: anything that can write a comment can drive the system, and anything that can read the thread can audit it. The webhook, the fallback poller, a human typing by hand, a future integration: all equivalent. No API to version, no client to build.

Where do the humans go?

Here is the picture I keep drawing on whiteboards:

In the traditional flow, a human drives every step: refine the story, write the code, review the diff, test it, deploy it, verify it. Tools assist; humans drive.

In the Ralph loop, the agent drives one autonomous run (plan, code, test, merge, deploy, collect evidence) and the human is only needed twice: to request something, and to judge the outcome. Not zero times. Twice. The entire design question of autonomous delivery is what must be true so that those two human moments are sufficient.

That question does not answer itself, and this is where most “we let AI write our code now” stories quietly fall apart. If a human no longer reviews every diff before merge, what stops a broken build from landing on the shared environment? If nobody watches the agent work, how do you know it tested anything? If it claims success, why would you believe it?

Our answers are boring, named, and written down. That is precisely the point:

• AATB: Aggressive Autonomous Trunk-Based delivery. The agent merges its own pull request once CI is green. The human approves after integration, judging the deployed, running feature, and rejection is a one-comment revert. Branch protection makes the safe path the only path. That flow, including why “zero required approvals” is a feature and not negligence, is Part 2.
• ARBT: Autonomous Risk-Based Testing. Eight named testing steps, four before the merge and four after the deploy. Four always run; four run “when needed”, and when needed is a risk decision the agent must take and report, never a silent skip. Plus a definition of evidence strict enough that “trust me, it works” is structurally impossible. Part 3.
• A design phase with a human gate. A feature too big for one autonomous run gets a written design first, and a human approves that design before any code is written. Even the rule that decides when a feature is “too big” gets tested against real past work items, so we know it keeps deciding correctly. Part 4.
• What this means for your organisation. Including ours, and including organisations that are nowhere near letting an agent merge to trunk. Part 5.

Why frameworks, why now

I believe autonomous agents in the delivery pipeline will be a defining topic of the next few years. Not because it is fashionable, but because the economics are brutal. Writing code used to be the slow part. The moment an agent writes it in minutes, everything around the code becomes the slow part instead: the review queue, the hand-offs, the test phase planned in weeks. All of it was designed for a world that no longer exists.

But here is the part I want to push back on, in both directions.

To the sceptics: “we would never let an AI merge to trunk” usually means “we have no framework in which that could be safe.” Fair. Neither did we, until we built one. But notice what the real question is. You don’t trust the agent. You trust the system around it: the guardrails, the gates, the evidence, and the one-click way back. You already build exactly that system for people. It is why you have CI, branch protection and code review for humans.

To the enthusiasts: an agent without a delivery framework is a very fast intern with commit rights. The demos look amazing. But the third week looks like archaeology: what changed, why, where is the test that proves it, who approved it?

The uncomfortable truth we ran into: most of the work of autonomous delivery is not prompting. It is delivery engineering. Revert strategies. Evidence formats. Failure reporting when the platform kills your process mid-run. Testing steps with names, so a skipped testing step is a visible decision instead of an invisible omission. That work is unglamorous, but it is the thing you need the most if you want your agents to touch your code.

One more thing these frameworks turned out to be, which I did not expect: a maturity ladder. AATB and ARBT describe a disciplined delivery flow (small vertical slices, trunk-based, evidence-first, reversible by default) that is worth adopting with or without an agent, and with or without a human behind the keyboard. An organisation still exploring AI-assisted development can use them as a roadmap: each practice you adopt makes both your humans and your future agents safer. More on that in Part 5.

What’s coming

Ralph is an experiment, and I will write about it like one, failures included, because the failures built the frameworks. That is the original Ralph bet, after all: assume the models are exactly as flawed as they are today, and they still produce useful changes remarkably often. We have taken the loop a few steps further, but the math has not changed. When three out of five features land successfully, unattended, that is not a disappointing score. That is a massive win, as long as the other two fail safely and visibly. The frameworks are what makes that second part true. There is a work item number behind almost every rule in our documentation: the run that skipped a local test on a “purely visual” change and shipped a visual defect. The rejected feature that taught us a reject must end the work, not restart it. The monitoring probe that took down the whole host mid-run because of one unhandled timeout.

Next up, Part 2: AATB, or what happens when the agent merges its own pull request, and why the human gate belongs after integration, not before.

The agents write the code. Engineering the loop is the job now.

Ralph is built at Rubicon, part of BBTG. The frameworks described in this series, AATB and ARBT, are our working answers, not industry standards. Yet.