Agile Measurement

Project success is measured against the business drivers of revenue growth; revenue retention; cost containment; risk optimization; and compliance.

Many organizations are shifting from Waterfall to Agile mode. Agile does not take less discipline but needs more engineering and management discipline. Given that business initiatives are unique endeavors, it would take a substantial number of projects using comparably skilled resources and different approaches to having unbiased metrics concerning alternative approaches. As Drucker wisely pointed out," you can only manage what you measure." But how to measure Agile success effectively?

The degree of success would be measured in terms of how much could a team deliver for their stakeholders/investors in agreed time frame and set cost for the project. If you need to define some kind of success measure, then identifying which of these, and it may be more than one, are the outcomes they are looking for and asking how the business will define success in that criteria is one approach. What are your goals in the execution of the project? In general, there are only a "benefits" that businesses value. They want a product that:

-makes money

-saves money

-saves time

-reduces risk (or increases safety)

-more convenient

-is durability/reliability

-adds prestige (or pride of ownership)

- has entertainment value

You have to make sure that the customer has defined carefully what they mean by "value," and the customer needs to make sure that those definitions still apply. The big strength of "agile" methods is that the value is delivered slice-on-slice, so that if the situation changes, the losses are minimized. If the business is "fuzzy" about the where the "value" of the project sits, or how that is to be obtained, this can create issues. Few matrices could be:

1). Number of changes delivered/ Total number of changes asked for. (It’s important because the entertaining changes is one of the key drivers for using Agile. This matrix can be used to indicate Agility of the team.)

2). Actual cost to Estimated cost (It will help in showing how effective is the Agile management in terms of managing the costs.)

3). Actual efforts to Estimated efforts. (It will help in showing how accurate the estimations have been.)

4). Quality. (Post-production defects/ Total number of defects reported including post production defects)

Organizational leadership is responsible for the performance of the organization. Organizational leadership is held accountable for that performance. Therefore, when and how an organization goes about ideating and conceptualizing is a decision that falls upon organizational leadership because it impacts performance. Organizational leadership is responsible for deciding whether an agile approach is suitable and organizational leadership is accountable when the results fall short. The agile team is responsible for delivering solutions - not for organizational strategy, not for capability gap analysis, and not for prioritizing organizational initiatives that derive from the capability gap analysis. The business capability gap analysis will lead to initiatives to buy, build, enhance, sunset, or retire systems and processes. Those initiatives will be prioritized and made into projects that are road mapped for project execution. And the project team doesn't get to decide what the project is so the team can't really be held accountable if someone else's idea doesn't work. The reason why it isn't the agile team's role to ideate and conceptualize is because the agile team isn't held accountable by investors, lenders, and regulatory authorities. The organizational leaders who are held accountable are responsible for ideating and conceptualizing.

Self-governance requires a level of maturity. Agile approaches rely on a project team's ability to self-govern - less management intervention. If you have teams mature enough to self-govern, you probably also have teams that would be efficient and effective regardless of approach. For a metric concerning success to have any validity, it has to be measured against something. In terms of an agile approach, it has to be measured against alternative approaches. The problem with that is that you have no reliable metrics on what would have happened with alternative approaches, be they different agile approaches, hybrid approaches, or waterfall approaches. Project success is measured against the business drivers of revenue growth; revenue retention; cost containment; risk optimization; and compliance; irrespective of the approach used to execute the project. This is how the business will measure the success of the project - by whether the desired result was achieved. This in most cases won't be known until after the project team has dispersed.

Agile is not and never will be a one-size-fits-all solution for what ails business. Evangelize if you want. It is just another tool in the toolkit. It is people getting work done, and people have gotten work done in many different ways using many different approaches. A good approach won't make a weak team's performance excellent. A poor approach won't make an excellent team's performance awful. Excellent people will excel despite an approach, not because of it. Weak people will perform weakly despite an approach, not because of it. Whatever metrics you derive to measure Agile projects, they should enable better actions and serious improvements, accurate enough, not exhaustive but simple enough and more importantly should motivate the team as well.

Posted in: Agile,KPIs