Why “clone from origin every job” fails on international CI links
Each full git clone negotiates packs and downloads objects the mirror may already have. Across jittery cross-region paths, that shows up as flaky pipelines, not just slow ones. A bare mirror is a single object database you keep hot on local SSD: one scheduled git fetch (or remote update) pulls new commits from the canonical remote, and every consumer stays incremental afterward. git worktree then layers multiple checked-out trees on top of that one repository, so parallel jobs do not duplicate .git or re-hit the wire for the same blobs. That pairing—mirror for transport, worktrees for concurrency—is the smallest architecture that still feels like “plain Git” to operators. For shortening cold starts that are not only Git (images, packages), see Dev Container prebuilds & remote workspaces in 2026.
Decision matrix: clone per job, mirror + reference, or mirror + worktrees
| Pattern | Best when | Trade-offs |
|---|---|---|
Fresh git clone from origin each job |
Ephemeral cloud runners with no shared disk, or strict hermetic “nothing reused” policy. | Highest bytes over the WAN; worst tail latency on poor routes; simple mental model. |
Bare mirror + clone --reference-if-able |
Jobs spawn new working trees often but can share a local object store. | Still many working dirs to clean; reference wiring must stay consistent across agents. |
Bare mirror + multiple git worktree paths |
One beefy builder runs several branches or matrix shards with strong local disk I/O. | You must serialize or namespace worktrees; concurrent writes to the same tree still forbidden. |
Shared objects pool (alternates / NFS) |
Fleet-wide dedup with centralized storage expertise. | Path stability and permissions become outage magnets; not ideal over high-latency NFS. |
.git/config, and ref updates are visible to all linked trees. Use separate clone mirrors per tenant or per sensitivity class, not one giant pool for unrelated products.
Copy-paste playbook: mirror, fetch, then worktrees
Assume the mirror lives at /var/lib/git-mirrors/org_app.git and jobs may use /srv/ci/worktrees/$JOB_ID as a parent directory.
git clone --mirror https://git.example.com/org/app.git /var/lib/git-mirrors/org_app.gitgit -C /var/lib/git-mirrors/org_app.git remote update --prune
git -C /var/lib/git-mirrors/org_app.git worktree add /srv/ci/worktrees/$JOB_ID/app-build maingit -C /var/lib/git-mirrors/org_app.git worktree list
After the job, remove the tree so disk does not grow without bound: git worktree remove --force /srv/ci/worktrees/$JOB_ID/app-build or prune stale entries with git worktree prune. If you need a non-bare working repository instead, clone once from the mirror with --reference-if-able and attach worktrees there—the economics are the same: one incremental fetch location, many leaves.
What this does (and does not) solve
Git object deduplication stops repeated blob downloads; it does not replace a binary or container cache. Layer npm, pip, and OCI pulls behind regional mirrors or pull-through caches so CI time is not dominated by package registries—see CI dependencies & mirror acceleration in 2026. Likewise, Git LFS pointer files ride Git’s object model, but large LFS payloads still need their own cache or batch tuning.
-
Measure mirror lag: compare
HEADon the mirror to origin after each timer run; alert if older than your SLA. - Disk budget: worktrees reference the same objects but still need checkout size for large trees—watch filesystem quotas per builder.
- Concurrency: two jobs checking out different commits on the same worktree path will corrupt state; always unique directories per job ID.
FAQ
origin authoritative; add extra remotes only if your governance model allows, and document fetch policies.remote update, or fall back to fetching directly from origin for that job only—log both paths for audit.Regional mirrors and quiet CI hosts: why Mac mini fits
Running a bare mirror, periodic fetch, and several worktrees is continuous disk and CPU load that rewards stable, low-power hardware. A Mac mini with Apple Silicon delivers a native Unix environment for Git and common CI tooling alongside macOS signing workflows, with idle power draw low enough that an always-on mirror does not feel like running a space heater under your desk. macOS combines SIP, Gatekeeper, and FileVault with a track record of fewer surprise reboots than many commodity Windows build boxes—important when a regional mirror is part of your pipeline’s critical path.
Unified memory on Apple Silicon keeps large packfiles and parallel checkouts responsive under pressure, and the same machine can host macOS-specific jobs (Xcode, notarization) next to the Git tier. If you want one compact box to anchor cross-region CI—object mirror by night, green builds by day—Mac mini M4 is a practical starting point; use the homepage link below when you are ready to consolidate.