GitHub Actions runner benchmarks
The same repositories and the same workflows, run on GitHub-hosted ubuntu-latest and on compiler.dev. Median 1.5× faster across 18 measured stacks (15 faster, 2 the same, 1 slower). The best stack was 3.0× faster (Angular + Spring Boot).
Measured .
| Stack | GitHub-hosted | compiler.dev | Speed | Cost per run |
|---|---|---|---|---|
| Angular + Spring Boot (Gradle) + Postgres (JUnit + Playwright) | 6m 57s | 2m 18s | 3.0× | $0.18 → $0.14 (−22%) |
| Remix + Drizzle + Postgres (Vitest + Playwright) | 3m 07s | 1m 18s | 2.4× | $0.19 → $0.13 (−32%) |
| React + Go + Postgres (go test -race) | 6m 13s | 2m 42s | 2.3× | $0.37 → $0.28 (−24%) |
| React + ASP.NET Core + SQL Server (xUnit) | 3m 34s | 1m 41s | 2.1× | $0.17 → $0.20 (+18%) |
| Phoenix LiveView + Postgres (ExUnit) | 2m 07s | 1m 03s | 2.0× | $0.13 → $0.10 (−23%) |
| Turborepo + Next.js + tRPC + MySQL (Vitest + Playwright) | 3m 29s | 1m 53s | 1.8× | $0.17 → $0.17 (0%) |
| Next.js + Go + Kafka + Redis (docker compose) | 1m 16s | 0m 44s | 1.7× | $0.13 → $0.11 (−15%) |
| SvelteKit + FastAPI + Postgres + Redis (pytest + Vitest) | 3m 46s | 2m 18s | 1.6× | $0.21 → $0.22 (+5%) |
| Spring Boot (Maven) + Kafka + Postgres (Testcontainers) | 1m 54s | 1m 12s | 1.6× | $0.12 → $0.11 (−8%) |
| React + Rust (Axum) + Postgres (cargo test) | 2m 37s | 1m 45s | 1.5× | $0.16 → $0.14 (−12%) |
| Next.js + Supabase (Playwright) | 4m 03s | 2m 48s | 1.4× | $0.26 → $0.23 (−12%) |
| Nx: Angular + NestJS + Postgres (Jest + Playwright) | 3m 17s | 2m 26s | 1.3× | $0.25 → $0.23 (−8%) |
| Nuxt + Laravel + MySQL (Pest) | 2m 37s | 1m 57s | 1.3× | $0.13 → $0.15 (+15%) |
| Next.js + Express + Postgres (Vitest + Playwright) | 2m 31s | 2m 00s | 1.3× | $0.19 → $0.19 (0%) |
| React (Vite) + NestJS + Postgres (Cypress) | 3m 23s | 3m 06s | 1.1× | $0.19 → $0.16 (−16%) |
| Stack that finished about the same, 1 of 2 (about the same) | not yet compared | |||
| Stack that finished about the same, 2 of 2 (about the same) | not yet compared | |||
| Stack that was slower: single-threaded pytest (slower) | not yet compared | |||
Method
First job queued to last job done, full CI run with 10 test shards and a real database. GitHub ubuntu-latest (2-core, $0.006/min) vs compiler.dev (8 vCPU, $0.012/min). 1 GitHub run and 1-2 compiler.dev runs per stack. Each stack is a realistic application repository with its own database and test suite, not a synthetic CPU test.
Of 18 measured stacks, 15 were faster (shown above), 2 were about the same and 1 was slower (single-threaded pytest: a single-threaded job gains nothing from more cores). Their per-stack figures are not yet published, so they show as not yet compared; a further 1 stack has not been compared at all. The headline is median 1.5× faster across 18 measured stacks (15 faster, 2 the same, 1 slower). We list the ones that were not faster because they tell you when not to switch.
Why some stacks gain more
Jobs that compile (Gradle, Go with -race, Remix and Vite builds) or run tests in parallel use all 8 vCPUs and gain the most. Jobs that mostly wait on the network or run one test process at a time gain the least. Cost per run can rise when a job is short and single-threaded, because each minute costs more than a 2-core minute. The runner size picker helps you check your own jobs.
Run it on your repository
Benchmarks on other people's code only go so far. Comparison mode in the app runs a copy of your jobs on our machines while your results still come from your own runners, so you see both times on your own workflows before switching. Start with 200 free minutes or read compiler.dev vs GitHub-hosted runners.
Data: benchmarks.json (free to reuse with a link to this page).