Delivery and Experimentation
The first wins are about doing the same things faster. The next wins are about doing things you couldn’t do before.
Three figures worth tracking, in the order we saw them typically show up: delivery compression, productivity, and experimentation rate. Each tells you something different. The third distinguishes organizations that have transformed from organizations that have just gotten faster. A fourth signal, the unattended track delivering its first real work, shows up later and is treated separately at the end of the chapter.
Delivery compression
The most visible result of agentic work is that things ship faster. Migrations that used to take three sprints take two days. Bug fixes that took a week take an afternoon. Cross-platform features that required multiple teams ship from one team in a day.
This is real. It’s also easy to over-claim. The temptation is to compare against the slowest example you can find or count the win without acknowledging upfront cost. An honest framing has two numbers: a floor — what you can defend against committed milestones with recorded outcomes — and a central tendency — what teams report across many tasks, including dramatic outliers. Both are useful. They tell different audiences different things.
Compression isn’t uniform. Pattern-based work — migrations, refactors, boilerplate-heavy implementation — compresses dramatically. Features that need real product judgment compresses more modestly. A single number across all work types will mislead you. Track by category.
What we saw: Sustained delivery compression of about 2× across committed milestones — projects with formal start dates, target dates, and recorded outcomes. For high-pattern tasks (refactors, migrations, boilerplate), compression moved into the 5×–10× range. A few tasks with very strong pattern fit went higher. Those numbers describe the best cases, not the typical ones.
The pattern that mattered more than any single number: compression extended over time into work categories that initially didn’t seem to benefit. Implementation compressed first. Then validation. Then planning. Then operational work. The tools weren’t getting fundamentally more capable across that period. The organization was getting better at applying them.
Compression has a shelf life. It measures the gap between planning assumptions formed before agents and outcomes produced with them. Once your organization’s planning catches up, compression ratios collapse toward 1×. That’s not a regression, it’s the metric doing what it should. When that happens, the question shifts to throughput, the next section.
Stalled → If compression numbers look strong but the throughput of finished work isn’t moving, the gap is downstream. Code is being produced faster; the work isn’t reaching customers faster.
Productivity
Compression measures how fast individual tasks move. Productivity measures how fast the system as a whole ships. They’re related, not the same — and the gap is where most agentic transformations leak value.
Software productivity has been hard to measure for decades. There’s no clean number, which is why the industry has converged on frameworks like DORA and DX metrics that correlate with productivity even if they don’t directly capture it. The pragmatic move is to use the same proxies pre- and post-agent and look at how the trends move. That’s what we did.
The metric worth tracking is throughput-with-cycle-time. How many pull requests are merged per developer per week, and how long does each take from open to merge. Together they tell you whether the work coming out of compression is reaching production. If throughput climbs while cycle time also climbs, code is piling up in review. If both improve, the system is genuinely getting faster.
What we saw: One product team’s median PR cycle time dropped from ~5.7 hours to ~1.4 hours over 90 days — roughly 76% — while throughput grew 32%. Another team adopting structured agentic workflows mid-quarter showed cycle time drop from ~14 hours to ~5 hours over three weeks (60%) while throughput grew 47%. Both teams were leading adopters with structured operating models. These are achievable in well-run pockets, not org-wide averages.
A few caveats:
- Leading-team metrics aren’t org averages. Be careful about presenting them as if they were.
- Self-reported time-savings are softer than they look. People aren’t great at estimating time savings. Use these numbers as directional, not defensible.
- Cycle time matters more than raw throughput. A team producing twice as many PRs with three times the cycle time has moved the bottleneck, not removed it.
- PR cycle time measures code reaching the main branch, not customers. A team can be fast at merging and slow at deploying. Track both if you want the full picture.
Watch out → Productivity metrics are easier to game than compression metrics. When adoption itself becomes the target, engineers respond by producing usage that looks productive but isn’t producing matching value. If your productivity numbers are climbing without matching outcomes for the business, look at how the metrics are being incentivized.
Experimentation rate
The first two figures measure how fast you do what you were already going to do. The third measures the work you can now attempt. Work that didn’t have a path before because the cost of trying was too high.
The signal worth watching is the rate at which experiments turn into roadmap commitments. POCs from a hackathon. Prototypes from micro-sprints. Side bets engineers made because the cost of trying dropped. Some produce nothing. Some produce something interesting. A few reshape what the organization thought its roadmap should be.
What we saw: A two-day org-wide event focused on agentic delivery produced over 35 distinct projects, of which roughly a third (12) became formal roadmap commitments the following quarter. None of those 12 work items existed in any plan before the event. A few months later, eight had reached production, and eight new experiments had been initiated — these new ones not from a hackathon, but emerging from the production work itself. The experimentation pattern had shifted from event-driven to continuous. One generation of experiments produced the infrastructure and learnings that fed the next.
The conversion rate (about a third of attempts becoming formal work) isn’t a universal number. What matters more than any specific value is the existence of the conversion rate — a pipeline from attempt to roadmap that didn’t exist before.
If your organization has suffered from “we have to keep going because we’ve already invested” sunk-cost thinking, the experimentation lens gives you a way to address it. The dysfunction was always there. What’s changed is that the cost of trying something else has dropped enough that “we committed, we have to finish” is now visibly the expensive choice, not the cautious one. This isn’t a new principle. It’s the right time to put a long-standing principle into practice.
Culture → Experimentation rate goes up when the organization stops requiring a business case for every attempt. If teams need to justify a small experiment before they can run it, the cost of trying hasn’t actually dropped — you’ve just moved it from execution to approval. Make small experiments cheap to start, not just cheap to execute.
The second track: unattended work
The dual track was deliberate from the start. While the developer-steered track was the primary goal, unattended agent loops were planned as a parallel engine for work that doesn’t need a developer in the session. The unattended track came online later because it depends on the developer-steered track maturing first: the conventions, the agent-legible environment, and the reviewing capacity all have to be in place before unattended work can run safely.
The first concrete example: ten security tickets resolved in 45 minutes, no engineer in the session. The agent ran on a schedule, picked up the tickets, made the fixes, opened the pull requests. A human reviewed and merged. The same work had previously taken multiple engineer-hours per ticket of triage and remediation.
This isn’t faster execution. It’s engineer-hours eliminated entirely from a class of work. It changes what’s possible for security backlogs, dependency updates, observability tasks, infrastructure maintenance, and other domains where work is well-defined but tedious. The metric stops being how fast did the engineer do this with help and becomes did this require an engineer at all.
You still need code reviews to make sure unattended agent work doesn’t cause downstream problems. The most progressive teams in our organization are now spending real cycles on what work looks like when most of the routine work runs without them.
Trends over time
The initial stages of transformation saw wider ranges of delivery compression. This settled as the transformation became more normal four months into the process.
| Figure | First few months | Four months in | What changed |
|---|---|---|---|
| Delivery compression | 2× on committed milestones; wider variance on individual tasks | ~2–2.5×, stabilizing | Variance narrowed; floor held |
| Cycle time | Self-reported time savings, directional only | 60–76% PR cycle time reduction, measured | Better instrumentation; harder numbers |
| Experimentation rate | 35 POCs, ~12 converting (one event) | 8 in production, 8 new emerging from production work | Event-driven → continuous |
Cycle time data from team-level DX tooling; early time-savings figures from self-report surveys.
The most important shift: early metrics were mostly about implementation speed. By four months in, the metrics that mattered were about review speed (cycle time compression) and operational autonomy (unattended agents). These are the next two constraints in the agentic engineering system, and both are now showing measurable movement.
What to ask your teams
Two questions to put to engineering leadership at the end of a quarter:
- How much faster did we deliver what we’d already planned to deliver?
- What did we attempt this quarter that we couldn’t have attempted six months ago?
The first question is the one you’ll always remember to ask. The second is the one that matters. If the answer is empty, you’re getting a productivity dividend but missing the larger half of what agentic work is for. If the answer is a substantial list, you’re seeing the strategic return.
Using the numbers
Most organizations measure only the first figure. The reason isn’t that the others are hidden. It’s that compression numbers come naturally from the work, while productivity needs real instrumentation and experimentation rate needs the discipline of tracking what’s attempted, not just what ships. The figures only tell you something when read together: compression alone means individual tasks are getting faster; all three moving means the roadmap itself is being reshaped by what you’re learning.
Use the figures above as orientation, not targets. Aim for the pattern, not the specific numbers.
Stalled → If executive expectations are running ahead of what the organization can sustain, the fix usually isn’t more data. It’s reframing what the data means. Compression numbers age out as the org’s planning catches up. Productivity metrics need careful nuance. Experimentation rate is a real signal. Bringing executives along on what each metric is actually saying, and isn’t, is part of the leadership work.
The 2× compression floor describes a successful committed-milestone improvement sustained over time. If you’re seeing 1.5× across many such projects, you’re probably on track. If you’re seeing 8× on every project you can name, you’re probably comparing against your worst examples and your numbers won’t survive scrutiny.
The 60–76% cycle time reductions are what leading teams achieved with structured operating models and active investment in their review pipelines. If your leading teams are showing similar trajectories, you’re seeing the same pattern. If not, the gap is likely in the operating model adoption or the review-pipeline investment, not the tools.
The roughly one-third hackathon-to-roadmap conversion gives a reference point for whether your experimentation pipeline is real. Not all of them, not most of them, but enough to be a noticeable share of your next quarter’s plan.
The unattended track is harder to benchmark because it’s still new in most organizations. The right question is whether you’re investing in it at all yet. If you have engineers spending time on well-defined, repetitive work an unattended agent could do, the question is when you’ll start, not whether.