# CI race — round 2: the real-world suite (2026-09-13)

Eleven interleaved rounds per platform of the full suite: the synthetic micro
tier plus four notorious OSS projects built cold at pinned tags inside the
timed window. Protocol: one commit per round pushed to both mirrors,
alternating which platform receives it first, ~5 min between same-platform
pushes, timestamps in `push-times.jsonl`, joined by
`scripts/join_push_times.py`. 22/22 runs succeeded; every step in every run
recorded `status: ok` — no skips, no reruns, nothing excluded.

## Environment (as disclosed, not as matched)

| | rickub | GitHub Actions |
|---|---|---|
| runner class | `large` — 4 vCPU / 8 GiB / 32 GiB scratch | ubuntu-latest — 4 vCPU / 16 GiB |
| silicon | 1× AMD Ryzen 9700X bare metal (EU) | Azure standard fleet (shared) |
| isolation | Firecracker microVM per job | VM per job |
| measured single-core (probe) | 594 kops/s | 220 kops/s (2.7×) |

vCPU COUNT is matched; silicon and RAM are not — the probe row quantifies the
CPU gap so readers can attribute. This is a platform-vs-platform race as a
customer consumes them, not a controlled-CPU experiment.

## Real-world tier — medians, n=11 (p95 in parens)

| step | project @ pin | github | rickub | ratio |
|---|---|---|---|---|
| `rust-ripgrep` | BurntSushi/ripgrep @ 15.2.0 | 28.75 s (29.73) | **12.70 s** (13.32) | rickub 2.3× |
| `cpp-sqlite` | SQLite amalgamation @ 3.45.1 | 80.86 s (86.35) | **42.22 s** (42.87) | rickub 1.9× |
| `node-typescript` | microsoft/TypeScript @ v5.9.3 | 47.27 s (50.98) | **22.91 s** (24.70) | rickub 2.1× |
| `java-guava` | google/guava @ v33.7.1 | **38.00 s** (43.40) | 40.27 s (42.79) | github +6% |

## Micro tier — medians, n=11 (p95 in parens)

| step | github | rickub |
|---|---|---|
| rust cold build | 12.97 s (18.61) | **4.97 s** (5.92) |
| rust test | 1.47 s (2.04) | **0.37 s** (0.37) |
| docker build | 5.77 s (10.65) | **3.13 s** (3.54) |
| push → job start | 9.78 s (p95 **80.2 s**) | **4.32 s** (4.73) |
| end-to-end wall (run create→update) | 243 s (197–342) | **134 s** (128–138) |

## Honest readings

- **Three of four real projects run ~2× faster on rickub** (2.3× ripgrep,
  2.1× TypeScript, 1.9× SQLite). The fourth — Guava — **GitHub wins by 6%**.
  That is the most informative row in the table: a single-module Maven build
  is dominated by dependency download and single-threaded javac, the least
  CPU-bound workload here. Where the build is not CPU-bound, the gap closes
  or flips; the probe's 2.7× single-core advantage is what the other three
  rows are made of. We report the loss because it bounds the claim.
- **Variance is as lopsided as the medians.** rickub's p95 sits within 1–6%
  of its median on every step, and the whole 11-run wall-clock spans 10 s
  (128–138 s). GitHub's per-step p95s run 5–30% over median and its wall
  spans 145 s (197–342 s) — and one round's push→job-start was 80 s (p95
  row), a queue wait, not execution. Medians understate the experienced gap.
- **Storage burst numbers are round-noisy**: the 2 GiB dd medians flipped
  between rounds (GitHub 3.9 GB/s → 1.7 GB/s; rickub 3.5 → 2.5). A
  cache-warm dd burst measures as much luck as storage — read it as
  "comparable", no stronger, on both platforms.
- What this does NOT prove: fleet-scale queueing (one rickub node vs
  GitHub's planet), warm-cache paths (cold by construction — caches are a
  separate experiment), and workloads unlike these five.

## Traces

Per-step `results.jsonl` for all 22 runs is collected under `results/`
(machine-local, gitignored); the public originals:

- rickub runs 14–24: rickub.com/rickub/ci-bench/actions
- GitHub runs 34749218316, 34749658712, 34749753971, 34750047176,
  34750152332, 34750423984, 34750527899, 34750819327, 34750925882,
  34751250473, 34751355186: github.com/ogirardot/ci-bench/actions
- Push timestamps: `push-times.jsonl` → `scripts/join_push_times.py` →
  `compare.py --push-times push-times.json`; raw compare output preserved in
  the repo history of this commit's message thread (reproduce with the same
  three commands).