Engineering guide

Scaling Agent-First Software Delivery

Increase useful parallel delivery capacity by improving task boundaries, feedback throughput, and release evidence rather than simply adding more agents.

Published: Jul 14, 2026
Updated: Jul 14, 2026
Reading time: 13 min read

On this page

Agent-first delivery means coding agents perform bounded implementation or investigation work under engineering controls. It does not mean every ticket is autonomous or every agent can merge. The limiting capacity is often CI throughput, hot-file contention, evaluation quality, or reviewer attention, not model availability.

Overview

Outcome

Acme Shop can run independent checkout-api tasks in parallel only while task claims, isolated runners, CI slots, and reviewer capacity remain available; otherwise work queues or escalates rather than competing for the same files.

Overview

Prerequisites

Define task classes, owners, path scopes, deterministic gates, rollback expectations, a lease-based task queue, CI telemetry, and a human review service-level objective before raising concurrency.

Claim small, independently verifiable work

In the Acme Shop scenario, ACME-1842 adds missing-country validation to checkout-api. A second agent may add a tax-provider timeout test because it changes a different test seam. Neither should simultaneously modify the shared cart contract without a prior design task and one owner.

Acme Shop capacity-aware delivery queue

Classify task
Claim with lease
Isolated runner
CI capacity
Review capacity
Merge or release lease

A leased task claim prevents duplicate edits, while runner and review quotas limit admitted work. A successful local patch still waits for independent CI and human merge controls.

Task claim and capacity policy

task_class: checkout-validation
claim:
key: "acme-shop:services/checkout-api:ACME-1842"
lease: "30m"
max_active_per_path: 1
capacity:
agent_runs: 4
ci_slots: 2
review_slots: 2
gate:
required: ["test:checkout", "typecheck", "ci:checkout", "human-review"]
on_capacity_exhausted: queue

Task claims coordinate work; they are not merge authority. Repository protections, required checks, review, and deployment approval remain independent controls. Prompt text can encourage an agent to honor a lease, while the queue, branch rules, and CI admission policy enforce the practical boundary.

Plan CI and review capacity together

Instrument queue time, provisioning, dependency install, focused tests, authoritative tests, artifact upload, reruns, and reviewer wait. A ten-minute agent task waiting forty minutes for a runner is not a fast delivery path. More importantly, generating reviewable pull requests faster than owners can assess them increases risk and rework.

Keep fast iteration checks separate from the merge gate. For ACME-1842, test:checkout -- tax gives fast feedback, while the clean ci:checkout gate and owner review decide acceptance. Cache immutable dependencies by lockfile and platform, never mutable workspaces or credentials. Quarantine flaky checks with an owner and expiry rather than retrying until green.

Representative queue output

[ACME-1842] claim acquired lease=30m path=checkout-api
[ACME-1842] runner admitted slot=1/4
[ACME-1842] focused check: PASS in 38s
[ACME-1842] CI queued: position=1 ci_slots=2/2
[ACME-1842] review queued: commerce-platform position=1
[ACME-1842] merge: blocked until CI pass and human approval

Budget the complete delivery loop

Set task-class budgets for queue wait, execution time, model inference, tool calls, context, CI minutes, retries, and review wait. Track p50 and p95 by task type. Account for model, runner, storage, network, evaluator, and human-review cost; a cheap run that creates a long review is not a cheap delivered change.

When a run exceeds a budget without new deterministic evidence, stop and preserve the branch, diff, test output, and blocker classification. An operator can split the task, add a fixture, increase measured capacity, or reject the approach. Do not silently increase concurrency, retry a flaky test indefinitely, or bypass review to drain a queue.

Validate scaling behavior and recover

Positive validation: two independent checkout tasks acquire distinct claims, use separate workspaces, pass their focused checks, and receive CI and reviewer slots without touching the same files.
Negative validation: a second claim against ACME-1842's path is refused or queued; a fifth run is queued when the agent quota is four; an agent cannot merge merely because its claim succeeded.
Failure validation: simulate a stalled CI runner or expired lease. The run is marked blocked, artifacts are retained, and the task returns to the queue only after an owner decides whether to renew, split, or cancel it.

Recovery starts with the limiting resource. Repair CI provisioning, reassign the owner, or reduce task overlap before adding agents. If a claim owner disappears, expire the lease with an audit record; do not let another agent overwrite an unknown partial branch.

Troubleshooting

Symptom	Likely cause	Evidence to collect	Safe recovery
Duplicate checkout patches	No path-aware claim or lease	Claim keys, branches, changed paths	Cancel one run; add a shared-path claim before requeueing.
CI queue dominates latency	Runners or setup are saturated	Queue, provision, and test durations	Add measured CI capacity or reduce admitted work, not gates.
Review backlog grows	Agent admission exceeds owner capacity	Open PR count and review wait p95	Lower concurrency or assign review capacity before scaling.
Lease expires mid-run	Budget is too short or run is stalled	Lease events and last evidence	Stop or renew through the queue; inspect why progress stopped.
Flaky test consumes retries	Test defect is hidden by retries	Failure signature and retry history	Quarantine with owner and expiry; repair before promotion.

Authoritative references

Reduce delivery-path latency for distributed CI

When profiling identifies artifact delivery or global network latency as a CI bottleneck, talk to Optimi about performance architecture for reliable software delivery paths.

Discuss delivery performance architecture