Friday, March 20, 2015

Can you Compare Apple-to-Apple between Different Agile Projects?

Continuous improvement (CI) is in DNA of Agile.
Agile has been broadly adopted to manage projects across verticals, from management perspective, is there any single Agile parameter used for performance comparison between different Agile projects? Can you compare apple to apple between different agile project? Who would be more interested in such comparison, senior executives or team leaders? What do you intend to do with the team that under-performs? And, does such comparison help the team improve itself or compete with others?

By definition, each project (team) is unique: If you look deeper into this there are too many variables which make projects different-team size, technology, extent of external dependencies, experience of team in working with the domain and technology, experience of team members working with each other etc. So you want to find the team's weaknesses, understand its causes, and address those causes to improve your delivery capacity. The finding of key performance indicators (KPI) to compare one team to another is not the best way to proceed. You will be able to find the weakest team (according to the chosen KPI), but it does not make sense to use the KPIs to compare one team to another, because there is just so much variation of conditions between teams. Instead think about using performance metrics as a way for teams to reflect on themselves. Use metrics that cannot be gamed. Story Points are very easy to game. Measuring elapsed times (flow times) is very hard to game.

Comparison is possible, and not necessarily meaningful; only if you are able to normalize across ALL the variables which can lead to differences between projects. How do you normalize for chemistry between team members? There are broad metric that you can track and compare across projects, like how often are team deploying into a production like environment, feedback loops, cost of change etc. One should refrain from comparing project metric like velocity or story points etc. as these are specific to each project and governed by the dynamics of individual project situation. There are methods to assess Agile Maturity, parameters like how we are collaborating, communicating. But these are not for any comparison, it is for improving as a team.

Continuous improvement (CI) is in DNA of Agile: It’s not an attempt to find weakness but to provide support. It advocates CI as key for success in Agile. Any process or people needs improvement over time, you simply can’ have stable process for longer duration. You need to nurture a mindset of continuous improvement. As long as that kind of metric becomes better and better over time, you know you are going in the right direction. Note that at a certain point that metric will stabilize; and that is indicative of maximum sustainable capacity. When you get to that point, strive to keep the teams at the same performance level. Monitor the metric to see if there are regressions or new problems. Always good to know the process, so that you can know the reason for under performance by other project, plus you can have quantitatively defined goals and objectives for each project. Try these metrics:
* Flow times
* Flow time distribution
* Flow efficiency
* Work in process
* Throughput

"The Value" is the right parameter for comparison: The query is how to assess value delivered to customer. Thinking loudly what are the comparison parameters followed in water fall methods. For example, you have more than hundreds of projects on Agile, you need to have comparison basis which accommodates team size difference, different sprint size etc. If there is a single parameter which is valid for all projects, ideally comparison between projects is not recommended, but in real terms, many businesses do comparison between projects. Senior managers were sometimes extremely impressed with huge amounts of measures being collected, analyzed and processed. More seemed to be better. Especially in an Agile environment, the development staff will take the opposite approach: keep it simple. Two or three measures, probably dealing with velocity and defect rates, should keep an Agile team busy and informed. Senior Managers and external stakeholders will probably be thrilled with excessive metrics that will eventually be tossed, since in their perception, movement is progress. From the perspective of your team, however, such metrics will be viewed as roadblocks to progress.

Use these metrics as a baseline for each individual team to improve against itself, rather than compete with each other. Here are the few basic metrics used for capturing progress:
1). Defects per sprint. All defects will add to total effort of the sprint. If there is no effort bandwidth available in current sprint then postpone to next sprint, as user stories are accepted only after defects were fixed. So all defects are tracked not through its number but effort require to fix it which also take care of complexity of defects;
2). Story points: it takes care of any complexity in User Story. When story points were allocated, it’s based on its complexity. The more complex User Story is, the more Story points are allocated to User Story
3). Refactoring: There are some projects needs refactoring effort.

Measurement is always important, however, there’s no one size fits all. It has to fit for audience, and the purpose for such comparison is to make team continually improve itself, not to compete with each other, and assess the business value delivered to the customers.


Post a Comment