Engineering guide

Scaling Agent-First Software Delivery

Increase useful parallel delivery capacity by improving task boundaries, feedback throughput, and release evidence rather than simply adding more agents.

Published: Jul 14, 2026
Updated: Jul 15, 2026
Reading time: 9 min read

On this page

Agent-first software delivery means coding agents perform bounded implementation or investigation work under engineering controls. It does not mean every ticket is autonomous or every agent can merge. The limiting capacity is often CI throughput, hot-file contention, evaluation quality, or reviewer attention, not model availability — the same discipline a managed orchestration layer like Optimi applies when admitting traffic across providers by measured capacity rather than best effort.

Overview

Outcome

Acme Shop can run independent checkout-api tasks in parallel only while task claims, isolated runners, CI slots, and reviewer capacity remain available; otherwise work queues or escalates rather than competing for the same files.

Overview

Prerequisites

Define task classes, owners, path scopes, deterministic gates, rollback expectations, a lease-based task queue, CI telemetry, and a human review service-level objective before raising concurrency.

Model agent-first software delivery as a shared capacity budget

Adding more agents does not add throughput once another stage is saturated. Acme Shop's policy exposes four independent pools — the per-path claim lock, agent_runs, ci_slots, and review_slots — and throughput is set by whichever fills first, so raising agent_runs from 4 to 12 without touching ci_slots or review_slots only lengthens the queue in front of the same bottlenecks. Little's Law is the underlying check (tasks in flight equal arrival rate times average time in the system); measure where tasks actually wait, and grow only the pool with measured slack — low queue time and idle capacity, not a guess — one increment at a time.

Claim small, independently verifiable work

In the Acme Shop scenario, ACME-1842 adds missing-country validation to checkout-api. A second agent may add a tax-provider timeout test because it changes a different test seam. Neither should simultaneously modify the shared cart contract without a prior design task and one owner.

Acme Shop capacity-aware delivery queue

Classify task
Claim with lease
Isolated runner
CI capacity
Review capacity
Merge or release lease

A leased task claim prevents duplicate edits, while runner and review quotas limit admitted work. A successful local patch still waits for independent CI and human merge controls.

Task claim and capacity policy

task_class: checkout-validation
claim:
key: "acme-shop:services/checkout-api:ACME-1842"
lease: "30m"
max_active_per_path: 1
capacity:
agent_runs: 4
ci_slots: 2
review_slots: 2
gate:
required: ["test:checkout", "typecheck", "ci:checkout", "human-review"]
on_capacity_exhausted: queue

Task claims coordinate work; they are not merge authority. Repository protections, required checks, review, and deployment approval remain independent controls. Prompt text can encourage an agent to honor a lease, while the queue, branch rules, and CI admission policy enforce the practical boundary.

Task claims release work safely when capacity or evidence runs out

Leases coordinate scarce paths and capacity, while CI and human review remain independent merge controls.

Download:PNG SVG

Plan CI and review capacity together

Instrument queue time, provisioning, dependency install, focused tests, authoritative tests, artifact upload, reruns, and reviewer wait. A ten-minute agent task waiting forty minutes for a runner is not a fast delivery path. More importantly, generating reviewable pull requests faster than owners can assess them increases risk and rework.

Keep fast iteration checks separate from the merge gate. For ACME-1842, test:checkout -- tax gives fast feedback, while the clean ci:checkout gate and owner review decide acceptance. Cache immutable dependencies by lockfile and platform, never mutable workspaces or credentials. Quarantine flaky checks with an owner and expiry rather than retrying until green.

Representative queue output

[ACME-1842] claim acquired lease=30m path=checkout-api
[ACME-1842] runner admitted slot=1/4
[ACME-1842] focused check: PASS in 38s
[ACME-1842] CI queued: position=1 ci_slots=2/2
[ACME-1842] review queued: commerce-platform position=1
[ACME-1842] merge: blocked until CI pass and human approval

Capacity evidence belongs in one timeline

Queue time, CI telemetry, and reviewer wait usually live in three separate tools, which is fine until a release stalls and looks like three unrelated dashboards. The same correlation gap shows up across any estate with several specialized providers behind one entry point; a supervision layer such as Optimi's MYO folds telemetry like this into one timeline, so a CI shortage and a reviewer backlog read as one attributable event, not two.

Review capacity is usually the tighter constraint once you are scaling coding agents: agents open pull requests faster than owners can assess them. Require each agent-authored PR to carry a short receipt — scope, checks run, and the decisions a reviewer should inspect — so review time goes to judgment, not re-deriving context, and route task classes deliberately: a shared-contract change needs the owning engineer; a narrow addition like ACME-1842's tax-provider test can take a lighter pass. Track bounce rate per task class; a rise is review debt, fixed by slowing admission, not by adding reviewers who rubber-stamp.

Batch and speculate without overrunning agent CI capacity

Once ci_slots is the measured bottleneck, spend existing agent CI capacity more efficiently before adding runners. Merge-queue systems — GitHub's merge queue, and the older Zuul-style gating used at OpenStack and Google — combine several queued, independent changes into one temporary branch, run CI once against the batch, and pay for a smaller bisection round only if it goes red.

Applied to Acme Shop's two ci_slots: run three independent, path-scoped checkout tasks as one combined CI pass instead of three serial ones, and split and bisect on failure instead of discarding all three. Batching only helps when changes are genuinely independent — an unresolved shared-contract task like the cart-schema example reintroduces the contention a path-aware claim was meant to prevent, so exclude it from any batch. The trade is a little merge latency for materially more throughput from a fixed CI pool.

Budget the complete delivery loop

Set task-class budgets for queue wait, execution time, model inference, tool calls, context, CI minutes, retries, and review wait. Track p50 and p95 by task type. Account for model, runner, storage, network, evaluator, and human-review cost; a cheap run that creates a long review is not a cheap delivered change.

When a run exceeds a budget without new deterministic evidence, stop and preserve the branch, diff, test output, and blocker classification. An operator can split the task, add a fixture, increase measured capacity, or reject the approach. Do not silently increase concurrency, retry a flaky test indefinitely, or bypass review to drain a queue.

Validate scaling behavior and recover

Positive validation: two independent checkout tasks acquire distinct claims, use separate workspaces, pass their focused checks, and receive CI and reviewer slots without touching the same files.
Negative validation: a second claim against ACME-1842's path is refused or queued; a fifth run is queued when the agent quota is four; an agent cannot merge merely because its claim succeeded.
Failure validation: simulate a stalled CI runner or expired lease. The run is marked blocked, artifacts are retained, and the task returns to the queue only after an owner decides whether to renew, split, or cancel it.

Recovery starts with the limiting resource. Repair CI provisioning, reassign the owner, or reduce task overlap before adding agents. If a claim owner disappears, expire the lease with an audit record; do not let another agent overwrite an unknown partial branch.

Coherence risk is a fourth failure mode worth validating on purpose. Independent CI passes and reviews do not guarantee compatible designs: two agents can add a different retry helper for the same tax-provider call and both pass review, unseen by each other. Run a periodic duplicate-symbol scan, separate from per-PR CI, and assign one owner to consolidate before implementations drift apart.

Troubleshooting

Symptom	Likely cause	Evidence to collect	Safe recovery
Duplicate checkout patches	No path-aware claim or lease	Claim keys, branches, changed paths	Cancel one run; add a shared-path claim before requeueing.
CI queue dominates latency	Runners or setup are saturated	Queue, provision, and test durations	Add measured CI capacity or reduce admitted work, not gates.
Review backlog grows	Agent admission exceeds owner capacity	Open PR count and review wait p95	Lower concurrency or assign review capacity before scaling.
Lease expires mid-run	Budget is too short or run is stalled	Lease events and last evidence	Stop or renew through the queue; inspect why progress stopped.
Flaky test consumes retries	Test defect is hidden by retries	Failure signature and retry history	Quarantine with owner and expiry; repair before promotion.
Passes locally, fails in CI	Toolchain or dependency drift	Image digest, lockfile hash, environment diff	Pin the runner image and lockfile; reproduce in that image.
Batch run goes red, no clear owner	Independent-looking tasks batched without bisection	Batch membership, bisected results	Split and re-run halves to isolate the offending change.
Merged work duplicates logic	No coherence check across individually green PRs	Duplicate-symbol scan, merge history	Run a periodic coherence scan; assign one owner to consolidate.

Authoritative references

Extend capacity discipline to the delivery path itself

Dependency mirrors, artifact registries, and CI runner egress all cross a network before a merge completes. Optimi orchestrates that network for Performance, Security, and Visibility across providers, with MYO correlating the evidence — so a CI capacity review does not stop at the runner.

Discuss delivery-path visibility with Optimi