# Moving CI off GitHub: the queue was never the problem

Two projects moved CI off GitHub to my own machines. GitHub never queued us; the win came from cutting duplicate and heavy runs and the wait to staging.

Topic: CI & dispatch
Tested: 2026-10-28
Revision: 1
By Roman Vaisman
Page: https://eng.vaisman.me/entries/local-ci/

## In 30 seconds

- GitHub never made us wait in line; the time went into long runs, many runs, and the same suite run more than once.
- Each moved project's gate is now one recorded local pass at the exact commit; GitHub keeps a nightly or a fallback.
- Per run, local is about twice as fast on some projects, and the same or slower in some windows.
- Two of my projects never used GitHub runners at all.

## Problem

My agents write and review code fast now, and CI felt like the bottleneck: GitHub runners take forever. On 2026-10-11 I ruled local first.

The numbers said something else. My orchestrator agent pulled a month of GitHub Actions runs: on the projects measured, the median wait for a runner was 0.00 minutes. The time went into how long runs took, how many there were, and suites running more than once for the same change.

Only two projects moved: the tax plugin and the road-conditions site. The iOS app and the dictation app never used GitHub runners; they were local from day one.

## What I tried

| Attempt | What happened |
| --- | --- |
| GitHub CI on every PR push, plus local checks | The tax plugin ran its full suite three times per change: locally, on GitHub, and again in the staging deploy. |
| Local first, GitHub only when a PR leaves draft (2026-10-11) | Fewer GitHub runs, but every ready PR still waited on GitHub. |
| GitHub only on main, staging after it | Staging deployed only once main's CI passed; those runs took a median 77 minutes (2026-10-20 to 22). |
| Shared runner boxes in a strict sandbox | Good tests failed on its port rules, loopback and a 12 GB cap. |
| One of my laptops as a runner | Hung twice under CI load; withdrawn. |

## What worked

**One local check, one result line.** The tax plugin's agent wrote a single check script with a pinned Node, printing one PASS or FAIL line that names the exact commit. On 2026-10-14 the local pass recorded on the PR became the merge condition; on 2026-10-16 GitHub CI came off its PRs.

**Cut the duplicates.** On 2026-10-16 the tax plugin's agent reported the triple run, now cut to two. On my yes, the lesson went to every project as three checks: CI on both PR and push for one commit, deploy jobs re-running tests, and merge gates needing GitHub on top of a local pass. The iOS app found and fixed one. On 2026-10-22 the tax plugin's staging deploys stopped re-running the gate, which removed the wait between merge and staging.

**GitHub keeps a small job.** The tax plugin keeps an informational nightly on main. The road-conditions site keeps more: full CI on main after every merge and a daily run. If its local runner hasn't started a job within 30 minutes, GitHub becomes the merge gate for that change. Which to copy: keep CI on main after every merge if you want an independent check of main; keep only a nightly once you trust your local gate.

**Only the gate's run counts.** On the road-conditions site, one run per PR version is the gate; an implementer's own run never counts, which caught skipped tests. One recorded retry per version is allowed.

**A container per job.** Sandbox failures stopped once each job got its own podman container and network namespace; a flaky test went from failing 2 of 3 runs to passing 14 of 14. Jobs could also reach the box's public address; a firewall reject and a guard closed that. Size slots by memory; the CI-that-yields entry covers slots and caps.

**More runners when the local queue grows.** The iOS app's always-on Mac got so busy that one change waited 40 to 55 minutes to start. A second Mac now takes overflow, capped at three simulators, only when its load is low.

## Numbers

- **The tax plugin:** PR CI on GitHub took about 16.5 to 17.5 minutes (2026-10-11); its last 30 local checks have a median of about 8. Wider windows show a different picture: GitHub PR passes took a median 7.3 minutes (2026-10-04 to 10), local 11 (2026-10-18 to 24). So per run, local is about twice as fast, or the same or slower in some windows.
- **The road-conditions site:** the full gate went from a median 10:41 on GitHub (last 82 green PR runs) to 5:30 locally (235 runs). A second count roughly agrees: 9.6 minutes, about 6 locally.
- **The iOS app:** a full pass takes 20 to 22 minutes on either Mac; on 2026-10-27 they ran 95 builds between them.
- **The dictation app:** about 3.5 minutes for roughly 790 Swift tests, in a macOS VM on my Mac.

The bill dropped too, and I bought no hardware.

| Label | Value | Kind |
| --- | --- | --- |
| Median GitHub queue wait, on the projects measured (Oct) | 0.00 min | measured |
| Tax plugin PR CI, GitHub vs local (its own figures) | about 17 vs about 8 min | measured |
| Road-conditions site full gate, GitHub vs local (medians) | 10:41 vs 5:30 | measured |
| GitHub runner time removed, tax plugin | about 214 h a week | measured |
| Merge-to-staging wait removed, tax plugin (added up across every merge) | about 68 h a week | measured |
| GitHub runner time removed, road-conditions site | about 14 h a week | measured |
| Local jobs in one week (2026-10-18 to 24) | 1,170, about 167 a day | measured |
| Wait for a local runner, median and slowest 5% | about 1 s; 7 to 10 min | measured |

## When it stops being true

Local CI has a queue GitHub never had: a median of about a second, but the slowest 5% waited 7 to 10 minutes for a free runner.

A local runner can lie. The tax plugin's first one accepted an overflowing shard count, skipped every shard and still printed PASS; a reviewer caught it, and the count is now range-checked.

Speed has side effects: an agent polling CI hit GitHub's API rate limit, and the tax plugin's shared changelog table conflicted on every merge (now one file per entry). If your GitHub runs are short and never duplicated, moving buys little: the road-conditions site removed 14 hours a week of GitHub runner time, against the tax plugin's 214.

None of this is specific to Swift or TypeScript; it should work for Python or any other stack.

## Agent brief

```agent-brief
You are helping me move CI off GitHub-hosted runners onto my own machines, and cut the runs I don't need.

First ask me:
- Which projects run CI where, and which never used GitHub runners.
- What each machine can give CI: memory, cores, and who else works on it.
- Which checks I want GitHub to keep.

Constraints:
- The gate is one recorded local pass at the exact commit; nothing else counts as passing.
- No change to merge rules or production deploys without my go.
- Report GitHub runner time removed as that, never as agent time saved.

Steps:
- Measure first, per project: GitHub queue wait apart from run time, and runs per change (general).
- Find duplicates: PR and push CI for one commit, deploys re-running tests, merge gates needing GitHub on top of a local pass.
- Write one local check per project: pinned toolchain, one PASS or FAIL line naming the commit. Any language works, Python included (general).
- Record the pass on the PR as a commit status or check (general); make it the merge condition; then take GitHub CI off PRs; then stop deploys re-running the gate.
- Keep GitHub as a fallback merge gate when no local job starts within 30 minutes; wire it to your merge rules (general).
- Keep CI on main after every merge for an independent check of main, or only a nightly once the local gate is trusted.
- Size slots by memory (see the CI-that-yields entry); run each job in its own container, network namespace and memory cap; block the host's public address.
- Count only the gate's run; allow one recorded retry per version.
- Watch the local queue's p95; add a runner when it grows.

Pitfalls:
- If GitHub's queue is near zero, the gain is fewer, shorter runs.
- A runner can print PASS after skipping everything: range-check shard counts, and compare test totals with the last run (general).
- A sandbox can fail good tests (ports, loopback): fix the runner, not the tests.
- Laptops can hang under CI load: keep them out.
- Agents polling CI can hit GitHub's API limit: poll less often.
- More merges conflict on a shared changelog table: one file per entry.

Done when:
- Each moved project's gate is one local pass at the exact commit, recorded on the PR.
- GitHub runs only what I chose to keep.
- A week's numbers show runs per change, local wait p95, and GitHub runner minutes before and after.
```

## Related

- [CI that shares agent boxes and yields to their owner](https://eng.vaisman.me/entries/ci-that-yields-to-its-owner.md)
- [One project's lesson becomes every project's rule](https://eng.vaisman.me/entries/fleet-effect.md)