Why we shouldn't use these productivity metrics for project execution

Recently, we have seen a proliferation of productivity metrics to manage agile projects.

Many of these productivity metrics are output-based and not outcome-based indicators

Let's take a look at a couple of them.

Velocity: Velocity measures story point completion per sprint in the agile world. It is commonly used to measure the productivity of a given user and for a team. Velocity, by definition, is an output-based metric.

Velocity as a metric can be manipulated easily, since Velocity is a meassure of Story points, developers can inflate the stories and show higher velocity productivity.

Story Point: Story points, by definition, are a measure to define the story's complexity.

Agile has developed various ways to define this unit of measure, Fibonacci, and other methods. Story point, by definition, is kept fussy so that developers can develop without getting stuck on the estimates. Since there is no universal standard measurement for the Story Points, it cannot be used as a basis for any productivity measure relative to other teams.

Escaped defects. The ratio of the defects found during the testing phase compared to those found during production. The theory is if we see 9 out of 10 defects before the production release, then the Escaped defect is 90%. It means 90% of the defects are captured well in advance, which correlates to customer satisfaction.

The problem with these metrics is they can be easily manipulated. Defects come in various forms and complexity. For example, the QA team and Dev team can quickly inflate the bugs by double-counting or converting one bug into multiple bugs. So, for example, if we found a bug in a module, and this particular functionality repeats in different modules or multiple scenarios, the QA team can just create new bugs for each module, thus inflating the bug count.

We should move away from these metrics, which are based on output rather than an outcome.