# We automated the development lifecycle, not just CI

AI agents now write, review and test each change before our developers merge it. Merges went from about 7 a week to 37-66.

Topic: Agents & review
Tested: 2026-10-28
Revision: 1
By Roman Vaisman
Page: https://eng.vaisman.me/entries/automated-dev-lifecycle/

## In 30 seconds

- The win wasn't just faster CI. It came from automating the whole lifecycle around it.
- Agents write each change from a short plan, then several AI reviewers, the full test suite and a browser walk-through check it.
- Hosted CI runs once, at the end. People still review, merge and release.
- Merged changes went from about 7 a week to between 37 and 66, and none got reverted in October.

## Problem

At [NASA](https://drivenasa.com/) (the National Auto Sport Association), our developers told me "looks like you have automated CI." I said yeah, I automated CI, and since then we've been shipping like crazy.

That undersells it, though. Sure, CI got faster: the full suite, about 17,000 tests, took about 24 minutes run one test at a time, and now runs in parallel on one powerful machine in about 3. But what I automated is the whole software development lifecycle: who writes a change, who reviews it, where the tests run, and when CI and people get to see it. CI is just one step, and it's the one they noticed.

I'm writing it up with real numbers, minus the specifics, because it's real-world AI use in a working business, not a demo, and I think other businesses can borrow the idea.

## What I tried

| Attempt | What happened |
| --- | --- |
| Reviewer tests with the network open | One reached a payment provider's test environment. Nothing was charged. Tests now block every outside network call by default. |
| Browser tests that depended on timing | They failed at random. We fixed each one at its cause instead of retrying it. |
| Line-number references in our docs | They caused merge conflicts once changes started piling up, so we dropped them. |

## What worked

**A short plan, then an agent writes it.** Every change starts as a short written plan. An AI agent writes the change from that plan.

**Several AI reviewers, one version.** Several independent AI reviewers check each change, all on the same exact version. Changes that touch money or security get more of them.

**The full suite on a powerful machine.** It's fast enough that a change runs it many times before a person ever looks at it. Ours happens to be local, but a big cloud machine does the same job. The point is the horsepower, not where it sits or who owns it.

**A look in the browser.** A walk-through in a real browser checks what users actually see.

**Hosted CI and people come last.** Only once the reviews pass does a change go to hosted CI, which runs once per change as a final independent check, and then to our developers. They review and merge, and I approve the sensitive releases.

**Try it on a small team.** Start with the split: AI does the writing and the first rounds of review, and people keep review, merge and release. And write each mistake down as a one-line rule the agents read next time.

## Numbers

Straight from our repo's history. That's roughly 5 to 9 times the merges a week, and a change now waits about 10 hours less to get merged. Reverts didn't go up either: 3 in August and September, none so far in October. It's the one quality number I've got.

| Label | Value | Kind |
| --- | --- | --- |
| Merged changes a week, Aug-Sep 2026 (average) | about 7 | measured |
| Merged changes in three full weeks of October | 43, 37 and 66 | measured |
| Merged changes, 2026-10-26 to 28 (first 3 days of a week) | 51 | measured |
| Median time from opening a change to merging it | about 23 h in Aug-Sep, 13 h in Oct | measured |
| Full test suite (about 17,000 tests) | about 24 min one at a time, about 3 min in parallel | measured |
| Merged changes later reverted | 3 of 64 in Aug-Sep, 0 of 202 in Oct 1-28 | measured |

## When it stops being true

This rests on people still reviewing and merging every change. If your developers can't keep up with the reviews, the agents just stack up changes waiting for them.

It also rests on the order. Put the hosted run on every push instead of once at the end, and you're back to waiting on it.

And it needs a full suite that runs in parallel and finishes fast. If yours still takes an hour, that step just becomes the new wait.

These are one team's numbers over about three months. Count your own merges and open-to-merge times before you believe a multiplier, ours included.

## Agent brief

```agent-brief
You are helping me automate my team's development lifecycle, not just CI.

First ask me:
- How a change goes from idea to merge today, and who does each step.
- Which changes touch money or security.
- Where our tests run and whether the full suite can run in parallel on one powerful machine, local or in the cloud.
- Who reviews, merges and releases.

Constraints:
- People keep review, merge and release; agents never merge or release.
- Every reviewer approves the same exact version; any change after that needs a new review (general).
- Tests block every outside network call by default.
- Hosted CI runs once per change, at the end, after reviews pass.

Steps:
- Measure first: merges a week, median open-to-merge time and reverts, from our repo host's history (general).
- Have an agent write each change from a short written plan.
- Add several independent AI reviewers, more for money and security changes.
- Run the full suite in parallel on a powerful machine, ours or a rented cloud one, before hosted CI.
- Add a browser walk-through that checks what users see.
- Only then send the change to hosted CI and to our developers.
- Write each mistake down as a one-line rule the agents read next time.

Pitfalls:
- A test with network access can reach a payment provider's test environment: block outside calls by default.
- Browser tests that depend on timing fail at random: fix the cause, don't retry.
- Line-number references in docs conflict as changes pile up: drop them.
- If reviewers can't keep up, changes queue for them: watch open-to-merge time (general).

Done when:
- Every merged change went through the plan, the reviews, the full suite and the walk-through first.
- Hosted CI runs once per change.
- A month's numbers show merges a week, median open-to-merge time and reverts against the month before.
```

## Related

- [Moving CI off GitHub: the queue was never the problem](https://eng.vaisman.me/entries/local-ci.md)
- [One project's lesson becomes every project's rule](https://eng.vaisman.me/entries/fleet-effect.md)