AI-native software development: from one agent to a software factory

When agents write the code, the bottleneck moves to specifying it, checking it and deciding what ships.

Sam Wen, XInfer.AI · current to 2026-11-22

This page collects what we know so far, from one developer with one agent to systems that run many agents. It is based on what the major AI companies and open-source projects have published. Every source is checked, and most are linked. The page also lists the problems that are still open. Examples use Claude Code, and most ideas apply to any coding agent. XInfer.AI, a production AI platform, maintains this page as a free, public reference.

Why

The question

Software teams are moving from AI that helps one developer, to several agents working in parallel, to systems that take a goal and run agents until a verified change is merged. How can a team take each step without losing quality, safety or control? This page collects what works today, what the major AI companies and open-source projects recommend, and what is still unsolved.

Who it’s for

This page is for engineers, tech leads and platform teams who use coding agents. It is also for people who build systems to coordinate agents, such as an AI-orchestrated software engineering platform. You need only a coding agent and a Git repository. The examples use Claude Code.

What changes when agents write the code

Anthropic’s AI-native SDLC playbook says code is no longer the slow part: “The bottleneck moves to the steps to the left and right of the build phase”. Before the build, that means planning and design. After it, review, testing and release (Anthropic’s playbook, Read).

Old controls assume a person reads every line. That doesn’t work when agents write the code. Anthropic’s playbook keeps the same goals and changes how they are enforced. It “combines the old control objectives with new enforcement”: hooks, CI checks and branch protection do the enforcing.

OpenAI, Google, Microsoft with GitHub, and AWS have each published their own process guide (Read). We also studied Meta and xAI, and found no process guide from either. See What we know so far.

Our reading. The five guides mostly agree. For each stage, they say what the person owns and what the stage produces. They all say verification is now the bottleneck. In all of them, a person still approves the merge. They differ mainly in how much they enforce. So the main work moves from writing code to specifying it, checking it and deciding what ships.

How this page is organized

The page uses two axes.

Three levels of coordination. This framework comes from our working notes. We checked it against the published sources.

  1. AI-assisted coding. One agent helps with one problem at a time. The developer coordinates.
  2. Parallel AI development. Several sessions run at once, each in its own worktree. The developer plans, assigns, reviews and merges the work.
  3. AI-orchestrated development. A system splits a goal into tasks, runs them in dependency order, checks the results, handles failures and prepares verified pull requests. Code does the coordination. People approve the important steps.

Six lifecycle stages. These come from Anthropic’s playbook: planning, design, building, testing, deploying and maintaining.

Each level has its own section. It starts with the main practices, then covers the six stages. If we know nothing reliable about a stage at a level, the cell says Not known yet. Open questions lists the problems nobody has solved. Companies already in this business covers 25 companies that build software with agents, or sell the tools for it.

Our reading. Get level 2 working well before you build level 3. An automated scheduler repeats a bad split as easily as a good one.

How we work

How each source was checked

We read what AI companies and open-source projects publish, and we keep notes on each source. Each claim about a company’s work has one of three labels:

If we can’t read a document, we record that, what we used instead, and how to get the original. A script checks every link and quote on this page against our notes each time we publish it.

Our notes

Our notes cover:

Some parts of this page rely on the working notes, not on a published source. Those parts say our working notes.

Three kinds of statement

We keep three kinds of statement apart:

Experiments

We try ideas on a copy of a real codebase, never on the live product. We call an example tested only if it ran. See the case study.

The coding tool

Examples use Claude Code. We checked its features against its documentation on 2026-11-14, for version 2.1.325. Most ideas apply to other coding agents. When another vendor’s tool has a feature that Claude Code lacks, we name it.

Case study: one production codebase

These measurements and experiments come from XInfer’s codebase. XInfer is a production AI platform with many channels, developed in Claude Code sessions. Each part starts with a lesson for any team.

Measure how your work would collide before adding sessions

The lesson. How many agents can work in parallel depends mostly on how your codebase is split, not on the orchestrator. Measure it from your own history before you design anything.

At XInfer. We replayed 1,594 merged pull requests. We asked how often they would have collided if they had run as parallel sessions.

Our reading. As the work is split today, this codebase supports about two sessions at once. This matches the advice in Anthropic’s playbook to start with two or three. To get more, change how features connect to the shared files that every channel uses. An orchestrator alone won’t raise this limit.

ArtCraft, an open-source project that builds app clones with many agents, expects 65% to 80% of its remaining work to run in parallel. There, each new command goes in its own module, so a new feature needs only a small edit to shared files (PhotoCraft, Repo). That is the change our measurement points to for this codebase.

Enforce a must-hold rule twice

The lesson. A rule that must always hold needs two checks:

At XInfer. We applied this to one real rule. Money columns, such as credits and refunds, must be updated in place. Code must not read the value, compute a new one and write it back, because a change that arrives in between, such as a top-up, would be lost.

Let an agent look for duplicate code

The lesson. Some reviews work better as an agent hook than as a fixed rule. After Claude adds a function, a subagent searches the codebase for an existing function that does the same job. If it finds one, it tells Claude.

At XInfer. We tested this in 12 real Claude Code sessions. It isn’t installed. One reason the hook gave was wrong in its details, so check its findings like any other reviewer’s.

What we know so far

The table links to each cell: three levels, each across six lifecycle stages. Each level starts with the main practices and what Claude Code provides.

Level
Level 1 Planning Design Building Testing Deploying Maintaining
Level 2 Planning Design Building Testing Deploying Maintaining
Level 3 Planning Design Building Testing Deploying Maintaining

Level 1: AI-assisted coding

One agent works on one problem at a time. The developer coordinates.

Main practices.

What Claude Code provides.

Source: Claude Code documentation, version 2.1.325 (Read).

Our reading. The sources agree on three steps for one session: agree on the goal before building, give the agent a check it can run, and have someone else review the code. They differ on how much process to use, and on whether to check the reviewer’s findings. Use a full process for a big change and a simple prompt for a small one. Get level 1 right first. Higher levels run many level-1 sessions, so any weakness is multiplied.

Level 1 · Planning

Agree on the goal before building, and write it down. Anthropic’s playbook (Read) starts each change with an intent.md file in the requester’s words. It covers the problem, the outcome, who is affected, constraints and open questions. It is committed to the repository and approved by the product owner. Claude Code’s best-practices documentation (Read) suggests asking Claude to interview you until it could write the spec, then building from the spec in a new session.

OpenAI’s guide Building an AI-Native Engineering Team (Read) lets the agent do the first pass at planning. The agent maps the feature to the code, points out unclear parts and estimates difficulty. Engineers decide priorities and trade-offs. How OpenAI uses Codex (Read) gives a size rule: Codex works best on tasks that take “about an hour to complete”, or a few hundred lines of code.

Our reading. A clearer request is the cheapest fix for a wrong result. Ask the agent to name what is unclear instead of guessing.

Level 1 · Design

Turn the request into a spec and keep it in the repository. The playbook’s spec.md has numbered requirements, the design, how each constraint is met, and areas of concern. It is written with the organization’s skills, so conflicts with company policy show up early. Later stages refer to the requirement numbers (Anthropic’s playbook, Read).

Several tools work this way:

Matt Pocock’s skills add a shared vocabulary: a glossary of project terms and a record of decisions. This keeps names in the code consistent (Repo). The Karpathy CLAUDE.md on our watch list asks for the least code that solves the problem, and for changing only what the request needs (Repo).

Level 1 · Building

Start in plan mode. Claude reads the code without editing it, and the engineer questions the plan. The plan is committed as plan.md and kept up to date with the code. Keep CLAUDE.md under one page and commit it. When the same mistake happens twice, add the fix to CLAUDE.md. Turn each policy that people apply unevenly into a skill, approved by the policy owner (Anthropic’s playbook, Read).

The setup around the model matters as much as the model. Google’s Kaggle paper The New SDLC With Vibe Coding (Read) says “Agent = Model + Harness”, and “Most agent failures, examined honestly, are configuration failures”. In OpenAI’s harness engineering post (Read), the repository’s docs/ folder is “the system of record”. A short AGENTS.md works as its table of contents. Custom linters enforce the architecture, and their error messages tell the agent how to fix the problem.

For long tasks, Anthropic’s Effective harnesses for long-running agents (Read) has each session finish one feature, commit it and update a progress file. The feature list is kept in JSON, because the model is less likely to overwrite JSON than Markdown.

Level 1 · Testing

Give the agent a check that returns pass or fail, and let it run the check itself. The check can be a test, a build, a linter, a comparison with a known output, or a screenshot next to the design. Claude Code’s best-practices documentation (Read) says “The single most useful line in your CLAUDE.md is the command that runs it”. For a bug, write the failing test first. During the fix, a hook that runs before each edit blocks changes to the tests, so the agent can’t pass the test by changing it (Anthropic’s playbook, Read).

The playbook also keeps 20 to 50 real past tasks as evals. They run again whenever CLAUDE.md, skills or hooks change, and each incident adds a new one. The Kaggle paper says “Set the bar at the eval, not the demo” (The New SDLC With Vibe Coding, Read).

The agent must not be able to change the check. AWS’s AI-DLC 2.0 specification (Read) asks that each stage’s results be “checkable by a program the AI cannot modify”. Anthropic found that “Agents reliably skew positive when grading their own work” (Harness design for long-running application development, Read).

Make the app itself easy for agents to test. In ArtCraft’s PhotoCraft (Repo), every menu item, tool and dialog runs a command from one registry of more than 500 commands. The UI, the command line, a JSON control channel and an MCP server all call the same commands. So an agent can open a file, run commands and check the result without using the GUI.

Our reading. Make the check required, not an instruction the session can skip. A Stop hook or /goal keeps the session working until the check passes.

Level 1 · Deploying

Before the merge, someone who didn’t watch the code being written reviews it. Claude Code’s best-practices documentation (Read) says: “A fresh reviewer does help, because it hasn’t seen your reasoning”. Review against the spec. The playbook’s REVIEW.md lists the review passes: bugs, security, and whether the code matches the spec and plan. It allows at most five minor comments and leaves the approval to a code owner (Anthropic’s playbook, Read).

New tools also check the reviewer:

Anthropic’s playbook turns each required approval into a hook that allows, asks about or blocks a tool call. It enforces each must-hold rule twice: with a hook at the edit, and with the same script as a required CI check. An edit hook doesn’t see changes made by a shell command, so the CI check is needed. In the release pipeline, Claude starts with read-only access. Deploy, status and rollback become tools, with one server per environment, and each environment has its own level of autonomy.

Level 1 · Maintaining

Anthropic’s playbook uses a script, not a model, to watch each metric against its normal range:

Findings come back as a new intent.md. Each practice gets one leading and one lagging indicator, using data the organization already has (Anthropic’s playbook, Read).

DORA’s ROI of AI-assisted Software Development (Read) lists experiment frequency and deployment frequency as leading indicators. For stability, it lists the change failure rate and rework. It expects returns to drop before they rise, and says to plan for the drop.

Improve the setup as you go. Add repeated mistakes to CLAUDE.md, read Claude Code’s /insights report, and hold a short review after each piece of work. The setup also goes out of date. Anthropic’s Managed Agents post (Read) found that context resets added for one model were no longer needed with the next. Its harness-design post says “the space of interesting harness combinations doesn’t shrink as models improve. Instead, it moves.”

Level 2: Parallel AI development

Several sessions run at once, each in its own worktree. The developer splits the work, hands it out, reviews the results and merges them.

Main practices.

What Claude Code provides.

Source: Claude Code documentation, version 2.1.325 (Read).

Our reading. Three vendor sources say parallel coding has limits. The limits are of two kinds:

In our case study, at most about two tasks at a time could run without changing the same files. The lower limit applies. Measure both for your own team and codebase.

Parallel across projects. Another way of working avoids both limits: one long session in each of several projects, instead of several sessions in one project.

  1. Spend a block of time on one project, in one session with the agent. Settle the intent and the design, and answer the questions the agent is likely to ask. Write the answers into the repository.
  2. Hand over the whole cycle as short steps, each with its own check. The agent builds, tests and checks its work, and stops only for what you listed. Addy Osmani (Reported) gives Dex Horthy’s rule of thumb: “an agent holds up for three to ten steps, then starts losing the thread past twenty.”
  3. Move to the next project and do the same. The agents run for hours, or overnight.
  4. When you come back, review each project’s results and the evidence for them.

The sources point the same way, though none of them measures this pattern. Four agents at once leave Simon Willison “wiped out for the day” by 11 AM, and Adam Tornhill writes: “It is the parallelisation of human attention that does not scale.” (Both Read.) METR (Read) found developers who “would often work an unrelated task while waiting for the agent to complete its work”. Meta’s Forum team (Read) names the condition: “Ambiguity that an engineer might resolve through intuition results in agent failure.”

Our reading. Separate projects share no files, so their sessions can’t collide. Each project gets your attention at two set points, the planning block and the review, instead of all day. What changes for the developer is when to switch: between projects, at those two points. It pays off only when:

At XInfer. We have written a template for the planning block: intent, scope, design, answers, done checks, stop rule, permissions, progress and a final report. We haven’t measured the pattern yet.

Level 2 · Planning

Sources differ on who splits the work:

They agree on what a good split looks like:

How many sessions at once: Anthropic’s playbook (Read) says start with 2 or 3, because “The practical ceiling is how many streams one person can review properly”. Anthropic’s multi-agent research system post (Read) adds that “Most coding tasks involve fewer truly parallelizable tasks than research”.

At XInfer. At most about two sessions at once. The case study counted how many tasks at a time changed no files in common. Plans run in one session today.

Level 2 · Design

Keep the agents apart. The common base is one worktree per task, enforced in code (Symphony; Claude Code’s worktrees and /batch). Coinbase’s internal tool Mux gives “each agent … its own git worktree, its own branch, its own terminal.” One engineer “might have three or four agents working in parallel” (Coinbase, Read). Agent teams share one checkout, so give each teammate its own files: “Two teammates editing the same file leads to overwrites” (Claude Code agent-teams documentation, Read). Worktrees separate files, but not the running system. Ports, databases and queues are still shared.

Each task needs a contract: its goal, the files it owns, the interfaces it must not change, how it is checked, and what it reports back. Our working notes suggest one line for every task contract: “Do not change shared interfaces or unrelated files. If your task requires that, stop and report the dependency.”

ArtCraft builds open-source clones of commercial apps with many agents at once. Until a shared file replaced it on 2026-11-20, PhotoCraft’s AGENTS.md set these rules (earlier version, Repo):

Its roadmaps estimate that 65% to 80% of the remaining work can run in parallel. CADCraft and GridCraft plan for 4 to 6 agents at once (Repo). These are the projects’ own estimates.

Our reading. Enforce file ownership with a hook, not a prompt. Block edits outside the task’s files, and run the same check on the pull request. Also design features so they don’t all plug into the same shared files. This, more than the orchestrator, decides how many sessions a codebase can take. ArtCraft shows the idea at scale: a new feature plugs in by adding a module.

Level 2 · Building

The playbook uses one worktree per independent task. It turns repeated jobs, such as simplifying, verifying or researching, into subagent definitions (Anthropic’s playbook, Read). OpenAI’s Codex subagents documentation (Read) says: “Use parallel agents for read-heavy tasks such as exploration, tests, triage, and summarization”. It warns that agents editing code at the same time can cause conflicts and extra coordination. A setting limits how many run at once. The Thoughtworks Technology Radar of November 2026 suggests “running two to four subagents at a time and keeping tasks that require the same understanding of the code within one agent” (Radar, Read).

Each subagent needs a goal, an output format, guidance on tools and clear limits. When the tasks were vague, Anthropic’s subagents repeated each other’s work (multi-agent research system, Read).

When one agent hands work to another, pass links to the spec, tickets and commits, not a retold summary. Start each worker with a fresh context (Matt Pocock’s implement-spec, Repo). Keep the chain of agents short. The agent-skills project’s rule is “Personas do not invoke other personas” (Repo). Chains of agents that retell context drift away from the task and use about twice the tokens.

ArtCraft’s earlier house rules for every agent are short and easy to copy (PhotoCraft’s AGENTS.md, version before 2026-11-20, Repo):

Level 2 · Testing

Each agent checks its own work, and the merged result is checked again. As our working notes put it: “Branches that each pass don’t prove the combination works”.

At this level, independent review often means a different model. In one setup from our working notes, Claude writes the code, Codex reviews it read-only, Claude fixes the confirmed issues, and CI decides. A different model gives some independence, but no guarantee. GitHub’s Copilot code review Max (Read) “runs several independent review agents across models from different providers”. Another agent combines their findings.

GitHub also lets Copilot’s approval count toward a repository’s required approvals. This is off by default. Admins choose which file paths it may approve, and new commits cancel the approval (same changelog, Read).

Our reading. Whether a review by the same kind of model counts as independent used to be a policy question. On GitHub it is now a repository setting. Decide it on purpose.

Level 2 · Deploying

There are two common ways to merge parallel work:

Failed checks, review comments and merge conflicts go back to the agent that owns the branch.

GitHub’s Agent Merge (Read) “carries pull requests through review, CI, and merge”. It fixes failing workflows and merge conflicts in a cloud environment.

DORA’s AI Capabilities Model report (Read in part) recommends trunk-based development, short-lived branches and small batches. It suggests tracking the merge conflict rate and the number of active branches. It also suggests keeping agent configuration files, such as CLAUDE.md, and prompts in version control.

Level 2 · Maintaining

Supervising agents becomes a job:

Measure the whole flow, not the agent: lead time, how often work is accepted the first time, rework at merge, defects that reach users, and cost per accepted task. An agent is not more productive if it “completes a feature in ten minutes but creates two hours of integration work” (our working notes).

Watch the design, not only the tests. Software Improvement Group (SIG) analyzed seven of ArtCraft’s apps (Read), more than 1.2 million lines of code. All seven scored below 2 out of 5 for maintainability, against a market average of 3. PhotoCraft scored 0.7 for modularity, despite more than 1,700 tests. SIG measured code structure, not whether the apps work, and it sells the tool it used.

Our reading. Avoiding merge conflicts is not the same as good design. Agents that never edit each other’s files can still produce tangled components. Measure code structure on every merge.

Microsoft’s study of its own rollout (Read; not peer reviewed) used telemetry from tens of thousands of engineers. First use “spread primarily through social networks”. Engineers who adopted the tools merged about 24% more pull requests than a synthetic control group (95% interval: 14.5% to 33.7%). Weeks with Copilot CLI showed a bigger increase than weeks with Claude Code. But the authors work for Microsoft. They suggest the difference may come from the kind of tasks, or from how well each tool fits Microsoft. They also say merged pull requests don’t measure quality.

Our reading. Treat the tool comparison as a warning, not a result. Like other sources, it suggests that the setup around the model, and how well it fits the organization, matters as much as the model.

Level 3: AI-orchestrated development

A system splits a goal into tasks, runs them in dependency order, checks the results, handles failures and prepares verified pull requests. Code does the coordination. People approve the important steps. A software factory works at this level.

Main practices.

What Claude Code doesn’t provide (our reading). Claude Code already has the agent setup, isolation, review and a way to run workflows. An orchestrator adds five things:

  1. splitting work by measured overlap and file ownership;
  2. a merge queue;
  3. a record of work that lasts beyond one session;
  4. budgets per agent that pause work;
  5. a score for which tasks are safe to give an agent.

Our reading. Vendors now cover three of these, in part:

No vendor yet splits work by measuring overlap, and no vendor scores tasks beyond rules of thumb. SuperPlane, an open-source project, scores the backlog (Repo; see Open questions).

Level 3 · Planning

Our working notes describe two orchestrators:

The rule for both: “Do not reimplement the agent runtime”. The coding agent stays the worker.

AWS’s AI-DLC workflows (Repo and Read) already work this way inside Claude Code. A fixed engine, not the model, decides which stage comes next. The Inception stage produces a dependency graph of units of work. A person approves each stage from Ideation on. In Symphony (Read), an agent can turn a plan into a tree of tasks with dependencies, and agents start only tasks that aren’t blocked.

Google’s Kaggle paper calls this “The factory model”. The developer’s output is the system that produces the code: specs and context, agents, tests and quality gates, feedback loops and guardrails (The New SDLC With Vibe Coding, Read).

Level 3 · Design

Approve the acceptance criteria before implementation starts. They are a numbered list of requirements, each with its own check, such as an authorization test, a state-transition test, an idempotency test or failure injection. From our working notes: “The coding agent shouldn’t be able to redefine a requirement to make its own output appear successful.”

AWS’s AI-DLC 2.0 specification (Read) writes each stage in three parts:

Corrections from people become candidate rules. A rule is added only after a person approves it.

Microsoft’s new primitives of agentic software delivery (Read) designs each stage around three things: the person’s job, the artifact the stage produces, and a shared knowledge layer that records each item’s source and version. It warns that “Vast quantities of raw data do not improve agent behaviors”. Its test for memory: does the agent do the job better with it than without it?

Level 3 · Building

Symphony (Read) is OpenAI’s reference orchestrator. The issue tracker becomes the control center. Each open issue gets its own workspace and agent. If the agent crashes or stalls, it restarts. Ticket states drive the work. By default, it runs 10 agents at once. Some OpenAI teams saw 500% more merged pull requests in the first three weeks. OpenAI doesn’t plan to maintain it as a product.

Anthropic’s C compiler experiment (Read) ran 16 agents on one repository with no lead agent. Each agent claimed a task with a lock file. About 2,000 sessions produced a 100,000-line compiler that builds Linux. When the work became one big task, the extra agents stopped helping. Anthropic then restructured the work so each agent could fix a different file.

For coordination, Azure’s orchestration patterns (Read) say: “Use the lowest level of complexity that reliably meets your requirements.” Magentic-One (Read) keeps a task ledger and a progress ledger, and makes a new plan when progress stalls. AI-DLC runs 29 of its 33 stages in the main session. Its 2.0 specification says “No hidden delegation”: agents don’t start other agents, and one conductor coordinates the work.

Our reading. The compiler result is the vendor finding closest to our own conflict measurement. Parallel agents help only while the work can still be split.

Level 3 · Testing

The most serious failure: an agent “changes code and tests together until both agree”, but the business requirement is still not met. Our working notes suggest three defences:

Our working notes test an orchestration system at several levels:

AWS’s AI-DLC 2.0 specification (Read) gives every loop a stop condition. A stage judged only by a model “will not self-halt on its own verification”, so it still goes to a person. In Anthropic’s harness design post (Read), the builder and the evaluator agree on the checks before each round of work. A single agent took 20 minutes and $9, and its main feature was broken. The full setup took 6 hours and $200 and produced a working app.

Level 3 · Deploying

Our working notes describe three levels of autonomy:

Approval depends on risk, not on the task. Authentication, payments, tenant isolation, migrations, infrastructure and customer data always get a stricter review. The rules are enforced outside the agent: “The agent can’t change its own approval criteria.”

GitHub Agentic Workflows (Read) are one example in production. Workflows written in Markdown compile to GitHub Actions. The agent runs read-only in a firewalled container and can write only through declared outputs. A separate job scans the changes for threats first. Default limits: 1,000 AI credits ($10) per run, 5,000 a day, and a 20-minute timeout.

Some teams go much further. OpenAI’s harness engineering team kept very few blocking merge checks, and warns this “should not be assumed to generalize without similar investment”. Anthropic now merges over 30,000 changes a day. It tests after the merge and before the release (Redesigning source control at Anthropic, Read).

Level 3 · Maintaining

Long-running agents need infrastructure that can fail safely. Anthropic’s Managed Agents (Read) keep three parts separate: the session (a log that is only added to), the harness and the sandbox. Each can fail or be replaced on its own. Credentials are never reachable from the sandbox where the agent’s code runs. Azure advises storing task progress outside the agents, so they can resume after an interruption.

There are now agents for operations. AWS’s DevOps Agent (Read) handles on-call triage and root-cause analysis. It has been generally available since 2026-03-31. Inside Amazon, it reached an “estimated root cause identification rate of over 86%”.

Limit what agents can do. In Microsoft’s Magentic-One tests, agents that were locked out of a site tried to reset its password. Some tried to recruit people on social media. The article recommends giving agents the least access they need, and pausing for a person before any action that can’t be undone.

DORA’s ROI report (Read) expects productivity to drop before it rises. It names three causes: learning the tools, a “verification tax” (more time spent reviewing generated code), and changing the pipeline to handle more changes. It says: “We strongly recommend organizations do not adopt a headcount-reduction strategy.”

Open questions

The sources have not solved these problems, or answer them only with rules of thumb. They matter most if you are building toward level 3.

Can one model write the code, the tests and the review?

If the model misunderstands the request, it can write wrong code and a test that passes. Nothing in the loop catches this. Partial answers so far:

None of these fully solves it.

What counts as an independent review?

The playbook’s rule is that “the agent that wrote the code has no way to approve it” (Anthropic’s playbook, Read). That covers approval, not independence. GitHub now lets Copilot’s approval count toward a repository’s required approvals (changelog, Read). So each team must decide whether a review by the same kind of model is independent enough.

The Thoughtworks Technology Radar of November 2026 warns against “Human-in-the-loop theater”: “keeping a human in an agent’s loop as an empty approval step rather than a real check.” It suggests matching the review to the risk. “The riskiest or hardest-to-reverse changes are reserved for humans to apply. Intermediate-risk changes go through human review. Low-risk changes can be auto-merged by coding agents through a continuous integration pipeline.” (Radar, Read).

Who approves on a small team?

The roles in Anthropic’s playbook assume a large company: a product owner, policy owners, a tech lead, a release manager and a service owner. On a team of one, or a small startup, one person holds most of these roles.

Dex Horthy of HumanLayer used to advise reviewing the plan instead of the code. In April 2026 he reversed it: “a thousand line plan was about a thousand lines of code”, so “the new advice is don’t read the plans, just read the code” (talk at AI Engineer Miami, quoted from YouTube’s automatic transcript, Read).

Our reading. On a small team, the most important approval is the acceptance criteria before work starts, more than reading the code afterwards. Keep the criteria short, so approving them doesn’t turn into reading a long plan.

How should work be split for parallel agents?

No vendor splits work by measuring how much tasks overlap. Today the split comes from people writing tickets, from a lead agent’s judgment, or from a dependency graph made at the start. Measuring conflicts from a codebase’s own history, as in our case study, is one possible input. No orchestrator uses it yet.

How do you keep the design clean when agents write most of the code?

Tests show that features work. They don’t show whether the code stays easy to change. In SIG’s analysis of seven ArtCraft apps, all scored below 2 out of 5 for maintainability, and PhotoCraft scored 1.3 for testability despite more than 1,700 tests (SIG, Read). SIG’s explanation: “Tangled code is harder to test thoroughly, however many tests you write.”

Our reading. A software factory needs a design check on every merge. Measure coupling and modularity, and block a merge that makes them worse.

Which tasks are safe to give an agent?

No vendor offers a score yet. SuperPlane, an open-source project, scores the backlog for tasks agents can handle well (Repo). The vendors give only rules of thumb:

What does a good result look like?

The sources say what to measure, but not what a good number is. The indicators in Anthropic’s playbook have no baselines. DORA’s figures and Microsoft’s measured gain are averages across many organizations, measured in different ways. Use them for context, not as targets.

How should evals handle run-to-run variation?

Anthropic’s playbook reruns 20 to 50 real past tasks whenever the setup changes, and requires “every eval passing”. A model’s output varies from run to run. The playbook doesn’t say how one pass should account for that.

Companies already in this business

Many companies already build software with AI agents, or sell the tools for it. We studied 25 of them, in five categories, on 2026-11-16. Most of the results below come from the companies themselves. Company sizes change fast, so this section keeps to how the work is done.

Category What they sell Companies
AI-native software services Software built for the client by engineers using agents Ode with Anthropic (formerly Fractional AI), 8090, Globant, Thoughtworks, Builder.ai (failed in 2025)
Low-code platforms with AI A governed platform where AI generates enterprise apps OutSystems, Mendix, Microsoft Power Apps, Retool, Pega
No-code platforms with AI Visual builders where AI assembles apps from tested blocks Bubble, Airtable, Glide, Softr, Adalo
Prompt-to-app builders A chat that builds and hosts a full-stack web app Replit, Lovable, Bolt.new, Vercel v0, Base44
Autonomous coding agents Agents that take a ticket and return a pull request Factory, Cognition, Cursor, Augment Code (agreed in October 2026 to sell its agent products to Harness), OpenHands

Our reading. The services firms sell what a software factory produces: delivered software. The coding-agent companies sell its engineering layer, at levels 2 and 3. The low-code, no-code and prompt-to-app builders sell a platform where people, or agents, build apps that then run on that platform.

“Software factory” is already a product name

They follow a spec-first chain

Augment’s Cosmos has agents for each step, and a person at four points (Augment, Read):

Step Agents What people do
Requirements to ticket Project Builder “approve the design”
Ticket to pull request PR Author, Ticket Manager “review auto-fixes on PR”
Pull request to production Risk Analyzer, Deep Reviewer, Pair Reviewer, Verifier “make the final merge decision”
Production to ticket Feedback Triager, Incident Investigator “approve remediation action”

Augment has since agreed to sell Cosmos to Harness.

Thoughtworks’ AI/works turns legacy code into specifications. “These specifications guide agentic workflows that generate production-grade code, automated tests and deployment pipelines.” (Thoughtworks, Read). After release, the platform “detects system changes, automatically updates the Super Spec, and regenerates impacted code” (AI/works, Read).

Our reading. This is the chain in Anthropic’s playbook: intent, spec, plan, pull request. A person approves the spec and the merge. Some companies make an exception for low-risk changes: Augment’s Risk Analyzer “auto-approves low-risk PRs”, and an OpenHands agent merges its own low-risk CVE fixes (OpenHands, Read). The spec is the artifact that lasts, and the code can be generated again from it.

One agent that writes works better than a free-running team

Running the app safely is harder than generating it

The generated apps are first drafts. OutSystems reports “60-70% of the required functionality”, and adds: “Although workflows and UI refinements were needed.” (OutSystems, Read). In a Pega demo, the generated plan left tasks for people: “I’m not going to be able to do all of those today because there’s 75 of them.” (Pega, Read).

Keep agents away from live data and secrets

Our reading. In each incident, the agent, or the app it built, had more access than the user knew. Each company fixed it in the platform, not by telling users to be careful.

Say what the agents did and what people did

Builder.ai sold custom apps, and its AI, Natasha, was part of how it built them. Former engineers told The Pragmatic Engineer that “A team of 15 engineers worked on Natasha Codegen”, and that “There were around 500-1,000 engineers employed through outsourcing companies like Globant, TatvaSoft, and others.” The company failed over its revenue: “a prior $220mn estimate for 2024 revenues had been revised to around $55mn”. It became insolvent in May 2025 (The Pragmatic Engineer, Reported).

Our reading. A buyer of AI-built software needs to know who did the work. Record it for each delivery.

The price must cover the model

What results exist

Our reading. Almost all the results come from the vendors. The large speed-ups are first drafts, or migrations with a clear test for “done”. Augment’s numbers are the most useful, because they track speed, quality and human effort together.

Who owns what changed in 2026

For anyone building a software factory

Our reading.

When

When the main sources came out

Most sources here were published in 2025 and 2026. This field changes fast, so check a source’s date before you rely on it.

Date Source What it added
2024-11-04 Microsoft Research: Magentic-One An orchestrator that keeps a task ledger and a progress ledger
2025-06-13 Anthropic: multi-agent research system A lead agent with parallel subagents, and why coding is harder to split than research
2025-08-21 GitHub Spec Kit created An open-source toolkit for spec-driven development
2025-11-13 AWS AI-DLC workflows created A development method that runs as a workflow inside Claude Code and other tools
2025-11-26 Anthropic: effective harnesses for long-running agents Progress files and feature lists that carry work across sessions
2025-12-09 Linux Foundation: Agentic AI Foundation MCP for tools and AGENTS.md for repo instructions, backed by all five companies
2026-02-05 Anthropic: C compiler built by 16 parallel agents Where parallel agents stop helping
2026-02-11 OpenAI: harness engineering A product of about a million lines with no hand-written code
2026-03-24 Anthropic: harness design A planner, a builder and an evaluator, measured against one agent
2026-04-08 Anthropic: Managed Agents Session, harness and sandbox as separate parts
2026-04-22 DORA: ROI of AI-assisted software development Why productivity drops before it rises, and what to measure
2026-04-27 OpenAI: Symphony An open-source orchestrator driven by the issue tracker
May 2026 Google (Kaggle): The New SDLC With Vibe Coding How much the setup around the model matters, and the factory model
2026-07-01 Microsoft: study of its own rollout Telemetry from tens of thousands of engineers
2026-07-22 AWS: AI-DLC 2.0 specification Rules for when an AI may rely on its own checks
2026-08-21 Anthropic: the AI-native SDLC playbook Six stages, an artifact at each, and rules enforced by hooks and CI
2026-09-03 DORA: no 2026 survey No survey this year, and more qualitative and observational research
2026-09-30 ArtCraft: first app clones built by parallel agents Written rules for many agents in one repo, and plans in agent-hours
2026-10-13 Anthropic: redesigning source control Over 30,000 merges a day
2026-10-27 Microsoft: new primitives of agentic software delivery The person’s job, the artifact and a knowledge layer for each stage
2026-10-28 GitHub Universe Agent Merge, code review Max, Copilot approvals, Agentic Workflows
2026-11-04 SIG: code analysis of seven ArtCraft apps Low maintainability despite many tests
2026-11-05 GitHub: inside its new Git storage 5,000 agents on one repository, in GitHub’s own test
November 2026 Thoughtworks Technology Radar, volume 35 Risk-tiered human review, and two to four subagents at a time
2026-11-11 How autonomous AI agents built AWS Emulator Amazon’s own account of a released product built only by agents: three agent teams test writers, issue finders, fixers , one issue per fixer
2026-11-16 How the Claude Code team thinks about verification Anthropic’s own account of how the Claude Code team checks code that Claude writes and reviews, which bears on the open question of whether
2026-11-17 Building a gated review pipeline for AI coding agents on AWS AWS describes six controls that combine fixed checks, AI review and risk-tiered human gates, and cites a study of 1.02 million pull requests
2026-11-19 Stripe evals: Measuring real-world coding agent performance Stripe’s own method for grading coding agents on real integration tasks with fixed graders may help with the open question on how evals shou
2026-11-19 Inside our CVE software factory: how OpenHands patches its own images OpenHands’ own description of a working pipeline: Trivy and Grype scans, then pull requests written by agents, then a daily report. It adds
2026-11-17 Do They Do as They Are Told? Investigating Coding Agents’ Adherence to Explicit Instructions in Cont This paper measures whether agents follow context-file instructions in real commits and finds that adherence varies by instruction type and
2026-08-21 Human judgment doesn’t leave the software factory. It relocates. Osmani says people keep ownership, set the quality bar, choose which verification signals to trust, and decide when the evidence is sufficie

This page

We started this research on 2026-11-13. This version is current to 2026-11-22.

What comes next

A daily workflow reads new publications, checks them and updates this page.

Planned, not yet started:

Top 10 sources

We ranked sources by how many times What we know so far and Open questions link to them. Eight sources tie at three links for the last five places. We kept the three that report measured results, then the two that describe a method in depth. That left out GitHub’s code review changelog, Matt Pocock’s skills and SuperPlane. Claude Code’s documentation is cited often but not counted, because our notes don’t record its links.

  1. The AI-native SDLC playbook: Anthropic, Louis Claxton, 2026-08-21 (Read). Cited 14 times. Covers all six stages, with an artifact at each. In our reading, it is the only vendor guide that enforces each must-hold rule twice, with a hook and a required CI check. This page uses its six stages.
  2. Symphony: OpenAI, 2026-04-27 (Read). Cited 7 times. A working open-source orchestrator for level 3, driven by the issue tracker. The source of the three-to-five-sessions limit.
  3. AI-DLC Workflows 2.0 Specification: AWS, Raja SP, 2026-07-22 (Read). Cited 4 times. Clear rules for when an AI may rely on its own checks. The checks must run in a program the AI can’t change, and a stage judged only by a model goes to a person.
  4. PhotoCraft: ArtCraft, 2026-09-30 (Repo). Cited 4 times. Written rules for many agents in one repository, a command registry that agents can test through, and roadmaps planned in agent-hours.
  5. The New SDLC With Vibe Coding: Google (Kaggle), Addy Osmani, Shubham Saboo and Sokratis Kartakis, May 2026 (Read). Cited 4 times. Shows how much the setup around the model matters. Describes the factory model, where the developer builds the system that produces the code.
  6. ROI of AI-assisted Software Development: DORA, 2026-04-22 (Read). Cited 3 times. An economic model: why productivity drops before it rises, and what to measure.
  7. Harness design for long-running application development: Anthropic, Prithvi Rajasekaran, 2026-03-24 (Read). Cited 3 times. Measures the cost and quality of a planner, a builder and an evaluator against a single agent.
  8. How we built our multi-agent research system: Anthropic, Jeremy Hadfield and others, 2025-06-13 (Read). Cited 3 times. What each subagent needs, and why coding is harder to split than research.
  9. Magentic-One: Microsoft Research, 2024-11-04 (Read). Cited 3 times. An orchestrator that keeps a task ledger and a progress ledger and makes a new plan when progress stalls. Its tests also show why agents need limits.
  10. AI-DLC workflows: AWS, created 2025-11-13 (Repo, with its README and user guide Read). Cited 3 times. AWS’s development method as a workflow that runs inside Claude Code and other tools, with a person approving each stage.

All sources

Every source cited on this page, grouped by publisher, with its date, how we checked it, and a link where our notes have one. Claude Code’s documentation is the only source without a link.

Anthropic

OpenAI

Google

Microsoft and GitHub

AWS and Amazon

Meta

Other

Companies already in this business

AI-native software services:

Low-code and no-code platforms:

Prompt-to-app builders:

Autonomous coding agents:

Open-source projects

Added by the daily update

Sources the daily update added, oldest first.