Measuring Productivity in an Assisted World: Why Delivery Leadership Needs New Signals (1 of 3)

Building on from a previous post, The Future of Delivery Leadership in Software Engineering. Part one of exploring leadership challenges in an AI-Augmented world.

For years, delivery leadership has lived with an uncomfortable truth: productivity has always been hard to measure.

We defaulted to proxies, Story points. Velocity. Utilisation. Hours logged. Tickets closed. Lines of code written (briefly, and usually regrettably). Even outcomes like release frequency became stand-ins for a more difficult question:

Are we actually becoming more effective at delivering value?

Now, AI-assisted work has made that question dramatically harder.

When an engineer can produce in hours what once took days, when a product manager can synthesise research in minutes, and when delivery teams increasingly operate with intelligent assistance embedded in daily workflows, many of our old productivity measures no longer make sense.

The challenge for leaders isn’t simply measuring productivity in an assisted world.

It’s learning how to measure meaningful productivity.

The productivity illusion

Most traditional delivery metrics assume a relatively stable relationship between effort and output.

More effort → more work completed.

That relationship is breaking.

A developer assisted by AI may produce 10 times as much code in a sprint. But does that mean productivity increased?

Maybe.

Or maybe we just accelerated technical debt.

A team may close more tickets than ever. But are they solving more customer problems, or simply breaking down work differently?

A delivery function may appear dramatically faster. But are people creating durable capability—or becoming dependent on tools they don’t fully understand?

The danger is obvious:

In periods of technological disruption, organisations often optimise for what becomes easiest to measure.

And in an assisted world, output is becoming dangerously easy to inflate.

Why old measures fail

Many familiar productivity indicators were already imperfect. AI simply exposes those weaknesses.

Velocity becomes unstable

Velocity was never meant to be a performance metric, yet many organisations quietly turned it into one. In assisted environments, velocity inflation becomes inevitable.

If AI reduces estimation uncertainty, automates boilerplate, or accelerates execution, story throughput rises.

But velocity tells us little about:

• Quality

• Customer impact

• System resilience

• Sustainability

• Learning

Higher velocity may indicate capability. Or noise. Or hidden future cost.

Leaders should be increasingly cautious about treating delivery speed as synonymous with productivity.

Activity metrics become almost meaningless

Commits.

Hours worked.

Messages sent.

Documents produced.

Tickets completed.

Assisted work dramatically increases visible activity while often reducing human effort.

Ironically, your highest-performing people may become less visible, not more.

The best leaders will stop rewarding visible busyness and start evaluating real, impactful progress.

The question shifts from:

“How much work did this team do?”

to:

“How much valuable progress did this team create?”

That is a fundamentally different leadership problem.

Productivity becomes multidimensional

In an assisted world, meaningful productivity sits across four dimensions.

1. Outcome productivity

The most important measure remains brutally simple:

Did we improve outcomes?

Are customers happier?

Did conversion improve?

Did incidents decrease?

Did time-to-value shrink?

Did adoption increase?

Assistance should amplify impact—not merely throughput.

A team producing more without improving outcomes is not necessarily more productive. They may simply be generating more organisational waste.

2. Flow productivity

Delivery leaders should pay increasing attention to flow efficiency.

How smoothly does work move through systems?

Questions worth asking include:

• How long does valuable work actually take?

• Where are delays occurring?

• How much rework exists?

• How often are teams blocked?

AI often improves local efficiency while exposing systemic bottlenecks.

If engineering accelerates while governance or decision-making slows, productivity gains disappear.

The limiting factor shifts. Leadership attention must shift with it.

3. Quality productivity

Faster work only matters if quality survives.

In fact, assisted environments may require higher standards, not lower ones.

Useful indicators include:

• Defect escape rate

• Reliability

• Change failure rate

• Maintainability

• Rework volume

• Cognitive load on teams

The uncomfortable truth:

If AI helps us produce low-quality work faster, we haven’t improved productivity. We’ve improved waste generation.

4. Capability productivity

This is the most overlooked dimension.

Assisted teams should become more capable over time, not less.

Leaders should ask:

• Are people improving judgment?

• Are teams learning faster?

• Is domain understanding deepening?

• Are we reducing dependency on key individuals?

• Are decisions improving?

There is a real risk that organisations mistake automation for capability. But delivery maturity has never been about output alone.

It has always been about resilience, adaptability and collective improvement.

The emerging leadership challenge: measuring team impact

Perhaps the biggest shift is this:

In assisted environments, leaders need to measure team impact, not labour.

Some individuals and teams will suddenly create disproportionate impact. Not because they work harder. Because they work differently.

They orchestrate tools well.

They make better decisions.

They synthesise information faster.

They remove friction.

They improve systems.

The best delivery leaders will become experts in recognising impact without defaulting to surveillance.

Because over-measuring activity in AI-enabled organisations creates exactly the wrong behaviour:

People optimise for metrics alone instead of outcomes.

We’ve seen this movie before. Only now the metrics are easier to game.

What should delivery leaders do now?

Three practical shifts feel important.

1. Stop treating productivity as a single number

It never was.

Balanced measures matter more than ever:

Speed + quality + outcomes + sustainability + learning

Any metric without context becomes useless.

2. Measure systems, not individuals

Most productivity problems are system problems.

Poor prioritisation.

Slow decisions.

Dependencies.

Fragmented architecture.

Approval bottlenecks.

AI may improve individual performance while exposing organisational dysfunction.

Don’t mistake local optimisation for enterprise productivity.

3. Reward judgment, not just output

As assistance increases, judgment becomes the differentiator.

Who asks better questions?

Who spots risk?

Who simplifies complexity?

Who improves decisions?

Who protects quality?

In a world where production becomes easier, the ability to judge well becomes more valuable.

Summary

Delivery leadership is entering a strange transition.

For years, we struggled to measure work because so much happened invisibly.

Now we face the opposite problem:

Too much work appears measurable.

Too much output becomes visible.

Too many signals become inflated.

The risk isn’t that we fail to measure productivity. It’s that we measure the wrong things with greater confidence.

In an assisted world, meaningful productivity may come down to a harder question:

Are we helping teams create more value, make better judgments, and experience less friction than before?

Everything else is just numbers.