How to Measure KPIs in Agile

For Agile, the old saying may still work- you get what you measure. 

Agile is not about free thinking or doing without structure, Agile is the complimentary methodology of Waterfall, add the flexibility on the rigidity; accelerate the speed of the hierarchy; create cascade from the steep slope, etc. Agile does not take less discipline, but needs more engineering and management discipline, thus, how to measure KPIs in Agile is also critical in Agile project success.

First, you must be careful in establishing KPIs, as the old saying goes, you get what you measure.       1) Be sure to understand what decisions you are looking to make based on these KPIs. Since there will be an effort to collect and message the raw data, you want to be sure that there are decisions made and actions taken based on the information provided. 2) There are many lagging KPIs that often take some time to reflect issues that arise. 3) The measures should both motivate individual as well as team performance, so the approach can be splitting KPI’s into 50:50, it means setting KPI for the team as well as the individual. In the traditional PM way, each individual will get a handful of KPIs. In the agile world, team is more important than an individual, do focus on team KPI as well

Second, well take the “Ask a Question” phase for collecting feedback: Around 80% of the people in software development completely miss is the "Ask a Question" phase. Most people do what you are doing right now and leap straight into the "Conduct Research" phase or even plow right into the "Construct a Hypothesis" phase. So let's start at the beginning, what question are you asking of your agile team? Is it the right one? Remember that an agile team will be for the most part self-organizing around specific issues so the right questions for Team A working on Product X may not be the right one for Team B working on Product Y.

Third, use agile metrics as KPIs. There have been three types of metrics that ever mattered in software development: Cycle time, quality (as measured by defect rates) and customer satisfaction. So use certain agile metrics, focus on the following - Velocity Improvement, Quality Delivered, Business Value produced (Desire of Customer divided by Cost of producing functionality). You may also collect the following metrics such as Velocity history, % Change in Velocity, Stories completed (done), Sprint burn-down, Story Points completed, Bugs etc. The quality of builds - pass, fail, Code Coverage by unit tests, functional tests,  to name some, but too much focus on all would either sub-optimize your overall delivery or may lead to management overhead.

Fourth, measure the amount of working software value that reaches customers hands, such as "percentage of story points completed within a sprint". Velocity trends (not absolute values) are also helpful. Although using Story Points is a crude way of doing this, much better to also estimate the business value of each story (so each story gets two estimates, one for complexity and one for value) then you can measure the value produced by the team easily. The tough part about value is - it is not an easy thing to assign the value to a story. In some circumstances, value can not be a KPI for IT projects because (1) business value is subjective especially if business value points are used; (2) some projects have no direct way to calculate business value (think regulatory projects that must be done to comply with regulations); and (3) Business value varies too much from project to project based on the perceived business benefit making it hard to measure objectively.

Fifth, ‘SMART’ principles to measure project iron triangle can be also effective. Most organizations have some reporting around scope, schedule and cost to determine if a particular project has a variance that needs investigation. These measures are often percolated up to a simple green, yellow or red designation to provide a view of the health of the project at a quick glance. This is especially helpful when reviewing a portfolio of projects / programs &/or products. There may be a few internal KPIs that the Agile team itself may want to monitor, one that comes to mind is build up-time, which might be of value if you have issues with people frequently breaking the build and having that cause problem for the rest of the team. Though some experts suggest recipes like Specific, Measurable, Achievable, Relevant, Timely - SMART & immediately actionable, Negotiable, Valuable, Estimable, Sized to fit, Testable but most importantly, the Actionable, Understandable & Accessible KPIs would far weigh enough. 

Whatever metrics you derive to measure Agile projects, they should enable better actions and serious improvements, accurate enough, not exhaustive but simple enough and, more importantly, should motivate the team as well.

Posted in: Agile,KPIs