Anthropic’s source‑control workload grew from about 2,500 merged changes per day in May 2026 to over 30,000 per day by September, driving the need for a system that could handle tens of millions of checkouts daily. Their monorepo’s history is nearly 100 GB and a compressed working copy exceeds 25 GB; naïve git checkouts would generate >600 PB of traffic per day. To address this, a seven‑person team built Scion, an in‑house version‑control system, and migrated the monorepo from git within five months while the merge rate increased tenfold. Scion’s design assumes clients are always online, uses a client‑server asymmetry where the server owns history and commit logic, and keeps only a bounded local cache on the client. Commits are expressed as changesets in a write‑ahead log that store a list of modified files and content‑addressed pointers to compressed blobs in blob storage; derived data such as blame and path history are computed recursively from changesets, allowing eager or lazy materialization. The system employs horizontally scalable tiers—scion‑daemon, scion‑cache, and scion‑servers—each maintaining >99 % cache hit rates and requiring no long‑lived state. As a result, the infrastructure footprint is 200× smaller than the previous git‑mirroring setup, CI checkouts dropped from minutes to seconds, and CI alone gained a 25 % CPU capacity increase. Claude accelerated development, rewriting the ingestion pipeline to use 20 × less memory and run 4 × faster.

Read original