Friday, March 20, 2015

Can you Compare Apple-to-Apple between Different Agile Projects?

Continuous improvement (CI) is in the DNA of Agile.

Agile has been broadly adopted to manage projects across verticals, from a management perspective, is there any single Agile parameter used for performance comparison between different Agile projects? Can you compare apple to apple between different agile project? Who would be more interested in such comparison, senior executives, or team leaders? What do you intend to do with the team that under-performs? And, does such comparison help the team improve itself or compete with others?

By definition, each project (team) is unique: If you look deeper into this there are too many variables which make projects different-team size, technology, the extent of external dependencies, the experience of the team is working with the domain and technology, the experience of team members working with each other, etc. So you want to find the team's weaknesses, understand its causes, and address those causes to improve your delivery capacity. The finding of key performance indicators (KPI) to compare one team to another is not the best way to proceed. You will be able to find the weakest team (according to the chosen KPI), but it does not make sense to use the KPIs to compare one team to another, because there is just so much variation of conditions between teams. Instead, think about using performance metrics as a way for teams to reflect on themselves. Use metrics that cannot be gamed. Story Points are very easy to game. Measuring elapsed times (flow times) is very hard to game.

Comparison is possible, and not necessarily meaningful; only if you are able to normalize across ALL the variables which can lead to differences between projects. How do you normalize for the chemistry between team members? There are broad metrics that you can track and compare across projects, like how often are team deploying into a production-like environment, feedback loops, cost of change, etc. One should refrain from comparing project metrics like velocity or story points, etc. as these are specific to each project and governed by the dynamics of individual project situations. There are methods to assess Agile Maturity, parameters like how we are collaborating, communicating. But these are not for any comparison, it is for improving as a team.

Continuous improvement (CI) is in the DNA of Agile: It’s not an attempt to find weakness but to provide support. It advocates CI as a key for success in Agile. Any process or people needs improvement over time, you simply can’ have a stable process for a longer duration. You need to nurture a mindset of continuous improvement. As long as that kind of metric becomes better and better over time, you know you are going in the right direction. Note that at a certain point that metric will stabilize; and that is indicative of maximum sustainable capacity. When you get to that point, strive to keep the teams at the same performance level. Monitor the metric to see if there are regressions or new problems. Always good to know the process, so that you can know the reason for underperformance by other projects, plus you can have quantitatively defined goals and objectives for each project. Try these metrics:
* Flow times
* Flow time distribution
* Flow efficiency
* Work in process
* Throughput

"The Value" is the right parameter for comparison: The query is how to assess value delivered to the customer. Thinking loudly what are the comparison parameters followed in waterfall methods. For example, you have more than hundreds of projects on Agile, you need to have a comparison basis which accommodates team size difference, different sprint size, etc. If there is a single parameter that is valid for all projects, ideally comparison between projects is not recommended, but in real terms, many businesses do a comparison between projects. Senior managers were sometimes extremely impressed with huge amounts of measures being collected, analyzed, and processed. More seemed to be better. Especially in an Agile environment, the development staff will take the opposite approach: keep it simple. Two or three measures, probably dealing with velocity and defect rates, should keep an Agile team busy and informed. Senior Managers and external stakeholders will probably be thrilled with excessive metrics that will eventually be tossed since, in their perception, movement is progress. From the perspective of your team, however, such metrics will be viewed as roadblocks to progress.

Use these metrics as a baseline for each individual team to improve against itself, rather than compete with each other. Here are the few basic metrics used for capturing progress:
1). Defects per sprint. All defects will add to the total effort of the sprint. If there is no effort bandwidth available in the current sprint then postpone to next sprint, as user stories are accepted only after defects were fixed. So all defects are tracked not through its number but effort require to fix it which also take care of the complexity of defects;
2). Story points: it takes care of any complexity in User Story. When story points were allocated, it’s based on its complexity. The more complex User Story is, the more Story points are allocated to User Story
3). Refactoring: There are some projects that need refactoring effort.

Measurement is always important, however, there’s no one size fits all. It has to fit for the audience, and the purpose of such comparison is to make team continually improve itself, not to compete with each other, and assess the business value delivered to the customers.


Post a Comment