Friday, January 9, 2015

How to Measure Agile Maturity

Achieve agile maturity at the 'organization' level, not just the 'team' level.

Many organizations are doing agile, and very few organizations are being agile. In order to improve agile maturity, the measuring needs to go beyond agile as "agility." It is a philosophy of managing complexity and unpredictability through empiricism. How do you measure a philosophy? There are practices that agile is enhanced with; such as engineering practices, testing practices, requirements etc. Also, many of the agile practices assume there are other project management practices in place it could leverage off; take Risk Management or Communication Management as examples. But more specifically, what are the principles and methodologies to measure agile maturity at an organizational level?

Organization maturity assessment should assess whether development teams are required to put their effort into the right project. The people who ask for functionality and the people who verify that functionality is being delivered should be part of the team. Everybody on that team should continuously collaborate and interact in the interest of delivering business value. The business value can't be determined by the development team only. The development team should deliver what the business asks (better, cheap, fast). Quality is the difference between what is delivered and what was required. An experienced dev/tester can advise but should never substitute the user. The development team always needs to keep business value and ROI in mind and as the focus. Always thinking about helping the business to deliver more business value earlier on and increase ROI, actually enable return earlier on to be reinvested, so more return value can be created faster. Achieve agile maturity at the 'organization' level, not the 'team' level. Therefore, the relevance test needs to be included:
- all approved work is completed
- all completed work adds value to the business

It's about results, not an artificial "maturity" metric. Much of Agile is a reaction of the one-size-fits-all attitude of maturity models that can be satisfied without real improvement. If it's about bragging rights, brag about your improved performance in the areas that are important to prospects and customers.  
(1). You encourage what you measure. If you measure intermediate objectives, you will encourage meeting them, which often sub-optimizes the main goal - delivering value. Therefore, the only thing you should measure publicly and continuously is delivered value.
(2). When a team root causes an issue (often, but not always triggered by not delivering sufficient and consistent value), then (and only then and only temporarily) measure the intermediate metrics needed to identify where the breakdown is and to see if remedies are addressing that breakdown. Once that issue is resolved, quit measuring those intermediates.
(3). Any effort at comparing teams or defining and collecting metrics so that management has spreadsheets to manage instead of teams to manage is worst than waste - it will always lead to dysfunctional behaviors. Put that effort into improving each individual team in its individual context. Making each team the best it can be at delivering value on a consistent basis is the only goal.

The most meaningful measure of Agile success is customer satisfaction. This won't necessarily be possible on the first release. However, if customer/end-user (and field/sales) satisfaction with the software improves after the successful implementation of Agile, then you've likely built a high-quality product that is more focused on customer needs. The survey needs to be statistically relevant and formal. It won't apply to all companies or internal projects, but large Agile transformations in large companies can achieve great value moving to Agile and have an objective measure of that value with continuous improvement. Further, to measure against your former process, you would need to have established baseline data with comparable or translatable metrics for comparison. If you don't have that, then you're probably limited to measuring the ongoing improvement of your new process approach. Financial measures are the area where you're almost guaranteed to have prior and comparable data. Performance and satisfaction measures depend on how much you focused on these things before.This should help teams and organization to:
1). Check periodically whether they are on right direction
2). Quantify the progress
3). Focus on right priorities
4). Give an idea to change leaders that where to focus and where to bridge the gap

There are a different level of agile maturity. At macro level, it is about being agile and doing the right things (projects, etc); and it means three things:
- Business IT integration
- Portfolio Management
- Portfolio governance

At the micro level, it’s about doing agile projects; and there are three things matter:
- Predictability of the team delivering to their commitments
- Continuous Improvement mindset exposed via retrospectives and how they are followed through and
- The level of Innovation ( the realm of Unknown Unknowns) the team can generate sprint on sprint.
This part is about building the product right.

Leadership maturity: C-level executives commitment can be judged in three ways as well on:
- Agile Hiring
- Agile Contracts (where vendors are engaged)
- Devops Implementation

The problem with measurement is that more often, metric is very subjective. That is, from a scientific methods perspective, there is no way to run a real experiment to try one process against another. Each organization is different, and even within the same organization, and even if they do better by "being Agile," there is no way to remove the confounding factors (such as luck entering a product space or better market conditions). But this will not stop an organization from measuring components of things that are measurable and will lead to favorable outcomes. Still, in order to improve the organizational level of agile maturity, understand and measure agility from different perspectives, focuses on customer satisfaction and ensure organization as a whole is optimal than the sum of pieces.


Post a Comment