Git Infrastructure for AI-Agent Scale: Per-Reference Quorums and Independent Read Scaling

Large AI-agent fleets do not stress Git in one uniform way. They create many short-lived branches, repeatedly read the same base commits, and publish updates concurrently. The useful architectural split is to protect references with a small, explicit write quorum while serving immutable Git objects from a separately scalable read layer.

This article describes a design for Git infrastructure in which each important reference has its own durability policy. A protected release branch can require more acknowledgements than an ephemeral agent branch. Once the reference is durable, clone and fetch traffic can scale through object storage and read replicas without making every read participate in reference replication.

The problem: references and objects have different workloads

A Git repository contains two categories of state that should not be scaled identically:

• References are mutable names such as refs/heads/main. They answer “which commit is current?”
• Objects are commits, trees, blobs, and tags addressed by content. After publication, an object is normally immutable.

Agent workloads amplify both sides differently. A coordinator may create thousands of branch references, while workers repeatedly fetch a shared base commit. A merge service may update one protected branch frequently, while most consumers only need read access to already-published objects.

If every read node also owns a fully synchronous copy of every mutable reference, read capacity becomes coupled to write coordination. If references are treated as ordinary cache data, a failover can expose an old branch tip even when all required objects are present. The design target is therefore:

1. Make the durability rule for each reference explicit.
2. Publish objects before publishing the reference that points to them.
3. Scale object reads independently from reference writes.
4. Make stale reads visible and bounded rather than silently authoritative.

Architecture: a reference plane and an object plane

The proposed system has two logical planes:

• Reference plane: a strongly coordinated metadata service that stores reference values, generations, policies, and acknowledgement state.
• Object plane: durable content-addressed storage plus horizontally scalable HTTP or smart-HTTP read workers.

The reference plane is authoritative for names. The object plane is authoritative for the bytes identified by object IDs, but it is populated before a reference becomes visible. Read workers may be numerous and disposable; the reference service remains smaller and is protected by quorum rules.

agent
  │
  ├── push objects ──> object ingress ──> durable object replicas
  │                                           │
  └── update ref ───> reference coordinator ──┴── quorum acknowledgement
                              │
                              └── publish generation
                                           │
                         read workers fetch objects independently

A per-reference quorum means that the acknowledgement requirement belongs to the reference, not just to the repository. For example, an illustrative policy could be:

refs/heads/main       quorum: 3 of 5   visibility: protected
refs/tags/release-*   quorum: 3 of 5   visibility: protected
refs/heads/agent/*    quorum: 2 of 3   visibility: ephemeral
refs/merge-requests/* quorum: 2 of 3   visibility: ephemeral

These numbers are examples, not measured recommendations. Select them from the failure tolerance, latency budget, and operational topology of the deployment. A quorum must be large enough to preserve the required state across the failures the system claims to tolerate.

Write protocol: objects first, reference second

The critical ordering rule is simple: a reference update must not become readable before the objects reachable from its new value have met the object durability policy.

1. Receive the packfile or object set from the writer.
2. Validate object hashes and connectivity according to the repository policy.
3. Write objects to the required durable backends.
4. Wait for the configured object acknowledgement threshold.
5. Submit the reference update with an expected old value or generation.
6. Commit the reference only after its per-reference quorum is satisfied.
7. Publish the new reference generation to read workers.

The expected old value prevents two agents from silently overwriting one another. A failed compare-and-swap should return a conflict to the caller, who can fetch the new tip and retry using an explicit policy.

# Illustrative local validation
git fsck --full --connectivity-only

# Inspect the reference that a coordinator intends to change
git show-ref refs/heads/agent/example

# Update only if the old value is still the expected commit
git update-ref refs/heads/agent/example NEW_COMMIT OLD_COMMIT

The commands above demonstrate local Git operations; they do not implement distributed quorum coordination. In production, the coordinator must wrap the equivalent compare-and-swap operation with durable acknowledgements, fencing, retries, and audit records.

Independent read scaling

Independent read scaling works because most object reads do not mutate repository state. Read workers can be added according to clone, fetch, archive, or blob-download demand while the reference coordinator remains sized for metadata operations.

A practical read path is:

1. Resolve the requested reference through the reference plane.
2. Return its commit ID and generation to the read worker.
3. Serve the packfile or objects from a nearby cache or object backend.
4. Attach the resolved generation to internal logs and, where useful, response headers.

Do not let a cache invent reference truth. A cache may serve an object by ID, but a reference lookup should include a freshness rule. For protected references, a read worker can require the latest generation from the coordinator or a signed snapshot whose validity interval has not expired. For ephemeral references, a bounded stale-read window may be acceptable if callers understand that the branch is advisory.

Object storage should be treated as a durability and distribution layer, not as a replacement for reference consensus. Garbage collection also needs coordination: an object must not be deleted merely because it is absent from one temporarily stale reference snapshot.

A reproducible design exercise

The following exercise uses a local bare repository to make the ordering and verification steps concrete. It does not simulate network failures or prove a distributed quorum implementation.

set -eu

rm -rf /tmp/git-agent-scale-demo
mkdir -p /tmp/git-agent-scale-demo
cd /tmp/git-agent-scale-demo

git init --bare origin.git
git clone origin.git work
cd work

git config user.name "Example Agent"
git config user.email "agent@example.invalid"

printf 'base\n' > README.md
git add README.md
git commit -m "Create base"
git push origin HEAD:refs/heads/main

BASE=$(git rev-parse HEAD)
printf 'base commit: %s\n' "$BASE"

git switch -c agent/example
printf 'agent change\n' > change.txt
git add change.txt
git commit -m "Agent change"
AGENT_COMMIT=$(git rev-parse HEAD)
git push origin HEAD:refs/heads/agent/example

printf 'agent commit: %s\n' "$AGENT_COMMIT"
git ls-remote origin refs/heads/main refs/heads/agent/example

To adapt this exercise to a distributed implementation, place an object-ingress acknowledgement between the commit and the reference update. Record at least the object set, acknowledgement set, old reference value, new reference value, policy name, and committed generation. Then verify from an independent reader that the resolved commit can be fetched from the object plane.

Verification checklist

Verification should test both planes and their ordering:

• For every committed reference generation, confirm that the new commit ID exists in each required object backend.
• For a compare-and-swap conflict, confirm that the old value remains unchanged and the caller receives a retryable conflict.
• Stop one reference replica and verify that a quorum-protected update either commits safely or fails closed according to policy.
• Stop one object read worker and confirm that reads can be routed to another worker without changing reference state.
• Serve a deliberately stale snapshot and confirm that protected-reference freshness rules reject or label it.
• Run repository integrity checks on a representative restored object set.

Useful evidence includes coordinator audit logs, reference generations, object-backend acknowledgement records, fetch traces, and the exact policy applied to each update. A successful client push alone is not enough evidence that every intended durability layer has acknowledged the data.

Failure cases and recovery behavior

Object acknowledgement succeeds, reference commit fails

The objects are now candidates for reuse, but no new reference is authoritative. Retry the reference update with the original expected old value only if the operation is still valid. Otherwise, leave the objects subject to normal retention and garbage-collection rules.

Reference quorum is unavailable

Fail closed for protected references. Returning success without the required quorum creates an ambiguity that downstream agents cannot safely resolve. An ephemeral branch may use a weaker policy, but that exception must be explicit and observable.

Read worker has stale reference data

Do not silently advertise stale data as current for protected references. Refresh from the reference plane, reject the request, or return a clearly bounded stale result if the API supports that distinction.

Concurrent agent updates race

Use generation checks or expected-old-value comparisons. A retry must re-read the reference and decide whether to rebase, merge, create a new branch, or abandon the update. Blindly replaying the same write can turn a transient race into data loss.

Limitations

This architecture does not remove the cost of pack generation, repository negotiation, large-file transfer, or merge computation. It also does not make every branch equally durable. Per-reference policies add operational complexity: policy changes need versioning, auditability, and a clear migration rule.

Independent read scaling can expose consistency trade-offs. A reader may resolve a new reference generation while an edge cache still lacks the corresponding packfile. The protocol must handle that state with retries or a temporary “objects not ready” response; it must not reinterpret the missing object as a deleted commit.

Finally, quorum size is not a universal constant. Network partitions, correlated failures, storage durability, recovery objectives, and regulatory requirements all affect the correct policy. The examples here are a starting point for a testable design, not production capacity planning.

Conclusion

For large AI-agent workloads, Git infrastructure is easier to reason about when mutable references and immutable objects have separate scaling and durability controls. Per-reference quorums protect the names that coordinate work. Object storage and independent read workers absorb clone and fetch demand. The essential invariant is ordering: durable objects first, authoritative reference second, independently scalable reads third.

Continue with the Guides section or look up terms in the Glossary.