Three Perspectives in Agile Success

The intended and expected impact of adopting Agile methods is to improve quality not worse.

Agile as an emerging software project management methodology, has been adopted in many organizations.  The fact of the matter is that as Agile methodologies are getting more popular, everybody is getting on the Agile bandwagon. As this phenomenon is taking place, the level of discipline, experience, and understanding of Agile practitioners in the industry is getting quite diluted, even causing concerns and confusion upon quality delivery. Thus, what are the biggest challenges with Agile, and how to overcome it?

1.   The Biggest Pitfalls with Agile

The intended and expected impact of adopting Agile methods is to improve quality not worse. The problem is not with the methodology (Agile vs. Waterfall) as much as it is in the mutually exclusive sets of expectations, or the business wants more and more functionality while IT management wants it sooner and sooner. Indeed, there're many pitfalls at Agile journey:

• Top-down standards and practices are de-emphasized: The biggest problem with agile is that software development devolves from a strategic partnership of technology. Agile is touted as a bottom-up development process with just enough top-down oversight to keep it all together. However, in practice, it is more likely that agile methods are implemented in small-scope projects of shorter duration as a bottom-up process where the enforcement of top-down standards and practices is de-emphasized or optional. The larger the assembly, the more likely these independently-built components will have integration issues and require rework, extending build time and cost of the project. 

• Lack of Clear Architectural Foundation: The problems cited for Agile around product quality tend to focus on poorer design and architecture, rather than code.  Agile software development implies concurrent design and concurrent engineering -- parallelization of tasks within a larger framework. Rapid and throwaway prototyping are used to minimize the amount of time and energy spent going down blind alleys. However, without a clear architectural foundation and strong requirements analysis up front, any bottom-up development process will spend its time chasing shadows in an attempt to satisfy fickle customers articulating "needs" that may or may not be relevant, realistic, or in the best interests of the organization 

• Immature Project Management Team: Obviously due to the lack of sustainable processes, Agile has become the most preferred option for most teams ...no one likes to be constrained by "laws" and invoking Agile gives the opportunity to weak teams to sell non-quality to weak management... at the end,  management is responsible for poor project performance as they have agreed to the approach and staffed the team ... the PM is working with what is given and the team is working as per their knowledge...of course the team will always exercise Parkinson's law in the context of doing minimum necessary to fly under the radar and management will always be bashing about speed but that's the nature of the game and will not change unless management becomes more mature and articulated. They're misunderstanding about Agile, as a low mature team believe Agile = little/no for structure, documentation, reuse, adequate workflow structure, PM structure, adequate regression testing, etc. 

• It’s fault of poor practices rather than the methodology: Methodologies have always been iterative and focused on change, at least since Aristotle invented physics. At the end of the day there is no magic in IT software development, regardless of the methodology, it is a discipline and if the discipline is not followed then bad things happen. The unnecessary failures are perhaps caused by misunderstanding Agile or bad management practice, thus, further check following metrics to verify Agile/Scrum practices, which management tools they use in an Agile Project:
  1) Sprint / Iteration Plans (there should be one for each Sprint)
  2) Burndown Chart
  3) Burnup Chart
  4) Velocity Chart
  5) List of Backlog Items
  6) Defect List & at what stage / environment where they found (QA, UAT, Production)
  7) List of Test Plans / Test Results / Test Coverage.

2. High-Mature Team is Key to Agile Success

One of the main points of Agile manifesto is people over process. Software development is an engineering discipline. The process is very important in any engineering discipline Agile is enabling speed; people do not understand the mission values of Agile. Most of the time,  speed is seen as the main driver and the tendency is to achieve speed at any price. It will cause Agile failure if the team is lacking maturity, that is and is not their fault as they are the way they are,  the true root cause is lack of knowledge from those that assembled the team which most of the time is not the PM (if they use a PM and not a Team Leader) could it be the hiring managers or other role…

• The high-performing agile team (either physical or virtual), can communicate well, collaborate well cross-functionally, it's critical for project success as Agile processes generally require better and more mature developers. In waterfall, a senior developer creates a Low-Level Design document and the work that a junior programmer gets is to fill in the function bodies. Agile talks about refactoring, which basically means applying design principles to the code at development time, and a developer with the same skill just will not be able to do that.  

• Pair mentorship: Pair Programming and mentorship are an excellent way of teaching newcomers about correct Agile implementation and becoming better IT professionals in general. The correct way to start an Agile project or practice is to pair your experienced and inexperienced people together until they understand how to do Agile correctly, or train developers to understand the nuances of applications they develop and cultivate analytical capabilities., etc. This sort of training and mentorship is the “Safety Net” that Agile methodologies provide and recommend when building an Agile practice. 

• Overall Scrum Team Maturity: Maturity is not just required out of the developers. Scrum makes a blanket assumption that all team members can do one another's job. This assumption simplifies certain things, but it is not a practical one to make most of the times. There has to be someone mature in the team who keeps things moving and deals with the issues as they arise.  Overall test skills should be provided to fulfill. a) testing skills (methods, ...) b) functional domain skills (business context, processes,...) c) technology skills (platform/application technology, certifications,...) to ensure a foundation for successful testing service can be provided

Therefore, in general, agile does require more maturity on the part of "everybody". It counts on this maturity for aspects like raising issues when appropriate and arriving at a great design as code matures. Agile can be a recipe for failure without this maturity. 

3. Agile is both Engineering and Management Disciplines

Agile is the umbrella and under that,  you will find that there are many different types of Agile development. SCRUM, Crystal, XP,  just to name a couple and there are many more. It is important to know that agile processes don't guarantee a successful delivery. No SDLC process can do this.

However, what Agile can do is let you know you have a problem much earlier than traditional SW development processes.  The objective of Agile is to get incremental improvements or developments in front of stakeholders in a short time to enable the development team to obtain feedback before they spend too much time developing something that does not meet stakeholder expectations. It does not mean developing without specs or without consideration of code quality.Agile is both engineering and management disciplines.

1) Agile, in and of itself, isn't the problem. Poor practices are. Running Agile projects without following necessary Agile practices and discipline can actually be very dangerous. Sometimes Agile is used as an excuse by some for not executing discipline that has been long and hard won. If you really execute Agile well, then it has more effective control, communication, metrics than waterfall - the big difference is you treat code and configured components as an asset and nurture the ability to make the learning of the development lifecycle a critical part of the way businesses deliver change. Agile is a disciplined business approach for IT excellence and code quality - as well as a means to deliver customer service excellence in increments. Agile can be an effective management philosophy. It can also be a cover for ineffective project management. The true approach should be "as soon as possible considering all facets of best practices".

2)  The quality of the new, rapidly developed solutions leveraged the quality of the well-designed architecture and component and there was still time to do a full analysis and design The PM should be able to give a detailed gap analysis -- what was supposed to happen, what is actually happening, and the likely source of the difference. A valid choice where there is a predefined and immutable deadline is to accept a reduced feature set or to can the project. If the choice is to accept a reduced feature set then what is delivered should be of high quality. The next choice is whether there is business value in filling out that reduced feature set in subsequent iterations. 

3)  The agile process needs to have fundamental knowledge: Agile process documentation is generally moot on certain points. It either does not want to look rigorous or does not want to repeat the well-known best principles. Thus, while the scrum process documentation would go at great lengths as to how scrum is done, it would not elaborate the various activities -- regression testing being one of them. The point is, the companies that use agile processes need to have this basic knowledge rather than following some agile bible religiously. It's worth resetting collective expectations to understand what the real purpose of Agile, XP, Iterative, Paired Development, TDD, Waterfall, Staged-gate, and SDLC methodologies. 

4)  Agile includes management of stakeholders, who need to understand both incremental development and iterative development. What is unique to agile is the sprint (short delivery cycles) and the continuous stakeholder involvement. They often need to grasp that just because they can see and touch something,  it is not necessarily ready for production- it may only be ready for feedback, adjustment, and refactoring. If they insist on deploying products before they are production ready, that may be a cause of poor code quality. The most important aspect with Agile is you require the full engagement of the client so that they can give feedback and direction in a timely manner. Many customers don't have that capacity. The next step is then to have someone take on that role which is where problem starting to creep into the whole process. 

5) Agile takes both engineering discipline and management practices. Agile is not 'opposite" of Waterfall, without structure, free thinking or doing, Agile is the complimentary methodology of Waterfall, add the flexibility on the rigidity; accelerate speed from the hierarchy; or create cascade from the steep slope., etc. Agile is about discipline and not just for technical teams but for business people as well, the business should never be presented with a choice that accepts poor quality that will be resolved at some later date. Neither should they have poor quality inflicted on them with a similar excuse.

6) The fundamental problem of managing vendors and requirements. The whole point of agile development (and really, all of the iterative, rapid prototyping development frameworks) is to focus on many iterations of task-level work, operating in parallel across the whole of the product, making extensive use of feedback to rapidly bring working software (as opposed to non-working models) to completion. If the vendor is responsible for it (told your team longer development cycles and poorer quality are the norms), you're better off finding another developer. If this is feedback from your team based on observation, then your team requires training in project management, requirements, or vendor management.

In summary, don't blame the methodology, blame the target audience. Blame the people thrusting it down the throats of the target audience without proper oversight to ensure successful adoption of the process. Blame the lack of education on the process and its benefits. Blame the "cold turkey" approach. Therefore, we should not accept that Agile means poor coding or missed deliveries. We should also keep in mind that processes alone do not deliver the product. It takes people, resources, processes and values to make a successful delivery. All of these must be reviewed in context together if one is to find the root cause and apply effective countermeasures.

Read more about Agile Pillars: 

Is Agile Strategy on the Way

Main Characteristics of Agile Enterprise Architecture

Three Perspectives in Agile Success

Agile Process Management

Agile Budget Planning

Agile Talent Management

Agile Governance

Posted in: Agile,Project Management