Five "Super-Pitfalls" Why Large IT Projects Fail

Large IT Project Fails because it's Too Large. 

Major IT projects have much higher percentage of the failure rate than other business initiatives, further, large IT projects fail even more frequently than the smaller project and cause significant damage for the organization in both short term and long run.
Approximately there are at least 40 failure-factors that seem to repeat themselves in the industry studies, which can be grouped into categories like planning and control, strategic fit, scope, commitment and participation, communication, management and leadership, risk management and a few others. But,  besides main aspects listed here to lead average IT project fail, what are those “super-pitfalls” to cause the large “Mighty Project” fall?

1. Lack of Clear Business Justification

These large projects need a clear plan from conception to implementation and lifecycle management. They need people with the foresight to see what the future may hold for the industry so they are not completing a project that is already an antiquated system. Usually, project portfolio management ensures project effectiveness, so larger projects can fail due to an inadequate PPM (portfolio management) process.

• A business case, if done well, will provide a much better baseline to consider length & cost considerations for the project. First and foremost, when initiatives are in this size range (with a few exceptions), they are not IT projects....they are business projects. Both business leadership and IT leadership need to LEAD these efforts as well as be accountable for them. If you try to deliver these size efforts only as IT, you will fail almost every time. So such debate is not just for the CIO, but also for other C-level officers. 

• The review is also imperative - Especially in large projects where several departments and executives are involved, very different opinions and expectations emerge about the project and its benefits for the particular department / executive. Some are incompatible to a degree, Particularly in an environment with multiple geographic locations where additional complexity adds to the mix, it's absolutely critical to make sure the Project Plan is thoroughly reviewed by all relevant business sponsors; and then re-checked to spot any flaws,  If the benefits are there, the IT group will be able to ensure that the correct costs for the project are identified and agreed to, including spending the money for the right level of project management.  

• Define success should be in both qualitative and quantitative terms A major reason where large projects fall is - the lack of an exact definition what are the project's deliverables. The challenge with the clear definition is not so easy when to manage the multi-organizational project. Each organization tends to have its own culture, the point of view and different definition. So each organization still ends up having same wording but the different requirement. Thus, they need strong project/program managers to see projects through without outside interference causing the change to a plan. And when an organization is paying for these, they should have the budget set aside to see the project through to fruition

• Understand the true cost at the front end of the project. Appropriate planning and estimating MUST be done on the front end of projects. And you must eliminate budget politics to get the true cost. Many times when being told the real cost of a project. The response was, "It just can't cost that much," not because they had any experience or insight into the effort, but because the number provided with all kinds of quantitative backup was beyond some arbitrary budget. If the project really costs too much, then don't spend the first dime on the effort. But instead, the project starts and is under budgeted by 50% or 100% and when it starts to go over budget it's now a failure.           

Statistically, only 20% of the projects analyzed the business benefits of the projects, it can be summed up with proper project preparation. An IT project needs to be viewed as any other project in the enterprise, with the due diligence necessary to ensure good results. If you can't, you should not be tackling a project of any scale. You will not be able to define measurable goals, monitor spend, gain any metrics and determine progress.

2. Large IT Projects Fail because they are too ‘Large’

Industry study also shows that small projects will succeed 76% of the time while less than half of large project will be successful. Small waterfall projects have almost the same success rate as small agile projects. The industry researches suggest that delivering product in small doses produces positive results. So the real question is how you keep projects as small as possible, and how and when you scale them? How to break-up large projects into small deliverables? If you don’t do large projects, chances are you will not fail at doing large projects. It is that simple! The vast majority of large projects are artificially large.

• Think Big, Act Small.  So you can think big, but you need to act small by making every big project a group of small projects. Keep a short interval between milestone deliverables, and for each attained, celebrate across the team, while rethinking the next sprint. Re-size and re-estimate. Keep asking the right set of questions by looking back on collaboration sessions PRIOR to anything being executed,: What did we miss? What possible issues could come up that are a potential risk? What would be the return on the investment if we were to "push" it at present - versus waiting and being reactive?  

• Second, the scope needs to be aggressively managed by the business, along with IT, based on the business case. If the scope does not deliver tangible, measurable benefits, then it is cut. You don't include things in scope because of preferences, nice to haves, or "this is the way we have always done it." Many big initiatives that fail do so because scope snowballs. Also, once the design is complete, you lock scope period. The only way the scope is changed after the design is with executive approval and tangibly delivered business benefit and even then, in 99% of these cases, the scope should be rejected. 

• Any project needs the equivalent of an architect to define the scope of the project. It also needs the commitment of senior management to authorize and fund the project. Upon a closer look at the building/construction process, EAs realize that all medium to large scale projects needed to be ”segmented” to ensure that as the project progressed, segments could be deployed and used if there should have the need to halt the project or to change direction due to business reasons demanded it. 

• Catch the “Window of Opportunity”: However, projects never go from being well managed, on-budget and on-schedule to outright failure overnight. There is always a transition period during which time the project is “troubled”. If you can get through the noise to see the real issues, you have a window of opportunity in which the project can potentially be rescued, and is likely that this is the last chance to save the project. Execute well-defined and time-boxed patterns of work - capability patterns; establish the required cadence based on quantified output. 

3. Lack of Skilled Emotional Maturity

Another research on large project shows Eight-one (81%) of successful project form $6 million to over $10 million in labor cost (large) were managed by organizations with a highly skilled to skilled emotional maturity. Only 19% large successful projects were managed by an organization with moderate to poor emotional maturity skills. Large projects with organizations that have moderate to poor emotional maturity skills will fail 71% of the time. So you have two ways to make large projects successful: 1) make them or 2) improve your emotional maturity skills.

• Lack of Leadership Maturity:  Leadership is critical; both at the Project Lead level and the CIO level ~ along with good marketing and communication. Projects of significant magnitude involve a multitude of people and take a long time to complete. Organizations need true leaders dedicated full time to delivering the project. Who lets things get this far out of control? Projects don't run themselves - people do. People let projects go with poor and incomplete requirements. People let target dates slip, and people don't hold members on the project team members accountable. People let project scopes get so large they become unmanageable.  

• PM Expert: Organizations also need experts with emotional maturity in the delivery of large-scale projects to manage the day to day and week to week activities. And you need the discipline of a business solution methodology (not just a tech based methodology) to make sure all aspects of the solution (strategy, culture, change mgt, process, tech, and measures/metrics) are covered. Sometimes people are always looking for PMI certified resources to lead and manage projects. These are the wrong value sets to follow when structuring a large-scale effort and if you hire resources this way; you are setting yourself up for failure. As PMI methodology is too simple for large-scale projects and the PMI management approach is one of coordination and reporting, not accountability, leadership and delivery. You MUST staff senior level, highly skilled, highly experienced and accountable resources to have a chance to succeed.  

• “Un-Professionalism” or Low-Mature Attitude to Project Pressure: What is the real reason behind large projects being more prone to failure? One reason for failures is that a big project is usually a high-profile, important-to-the business project. When projects and tasks are high-profile, it encourages people to lie and everyone is a champion in his field. And there was also constant pressure of stakeholders to go faster, faster, without knowing the situation, no guidance to a technical team such as system & network admin, database expert, no training session etc. 

4. Lack of Change Management

Statistics shows that a large number of IT projects fail (30%-70%) due to the failing to prepare the IT staff and the business for the actual change to come. The problems were in all the areas such as assessment of project scope, project pace & magnitude, project guidance & administration, technology and expertise, lack of domain knowledge, insufficiency in software development & database management and stakeholders influence & their expectation. Most projects consider and plan for training, but they don't consider organizational acceptance and adoption. 

• Adding organizational change plans to the project methodology will help to maximize the opportunity for success. Consideration should be given to the end user but also the IT staff. Both the business and IT may be impacted. Experts in the existing technology or process may have a steep learning curve and this can be very unsettling for the team. There is fear of the unknown and it can produce some interesting behaviors which ultimately impact the success of the project. 

• The business (not IT) actively DRIVES the change through the organization. Once there is a consensus at the leadership level, the change WILL happen, period. You do allow either a risk or benefits based challenges to solution components to a point. However, in general, employees embrace the change (and, of course, you need a substantial part of your program dedicated to org change mgt). Politics and passive aggressive behavior create a slow, painful death for projects.  

• PM and CM (Change Management) should go hand-in-hand: Change management is one of the core disciplines which helps ensure an IT project is "ready" to go live in production. A mature project readiness review process will include several system management control points. However, change Management is often overlooked and done poorly. Change Management is the discipline of assessing for, planning, and mitigating risk. it includes human and organizational change management 

• A project is more than an internal change...it is a marketing endeavor. It has to be constantly displayed, promoted, discussed, rewarded and evolved. All project are organizational change projects. They truly are when you get down to the core of it. So what is your currently understanding of your operation...can you map it? What process mapping do you have for the operational aspect of the project you are about to launch? Is it cross-functional, you have to have a dead-solid link between process, technical concepts and user roles. If you can't define...or have not thought of...your operational dimension, fail due to lack of operational analysis and ops./tech integration.

5. Lack of Proper Governance

Successful and large are not mutually exclusive. The difference comes from governance. As many times, Project Management has succumbed to the modern fetish, the effectiveness of project delivery has suffered because project management is now horribly over-engineered. for valuing form over substance. Simply put, we focus more on how we carry out the project rather than focusing on the activities producing the product we are being paid to produce, and a significant percentage of budgeted costs are non-productive costs or red tape, governance, and bureaucracy. This does not count the more of non-budgeted costs such as executive sponsor or user time. Thus: doing governance just right (neither too much nor too little governance) is both science and art

• Distinguish between project management success and project success. Typically, those organizations would have several different ways, level, dimensions, etc. to measure success. For example, they distinguish between project management success and project success. The former is mainly about being on time and on a budget. The later measures whether the benefits used to justify the project investment were achieved. One can fail on one and be successful on another. For example, a project can exceed the original budget but end up achieving the benefits. A forth type of measurement is whether the project is a strategic success. 

• CIOs and PMOs have to know when to probe into the weeds of the project to determine the true health of a project. CIOs and PMOs need to understand what critical risks to monitor from troubled projects in their own organization as well as benchmarks from other projects in other organizations. Once identified, these issues and risks have to be able to be escalated to the appropriate governance level for discussion and resolution. Is PM in line for the strategic direction of the company? If not, the use of limited resources is considered a failure, even if the project was a success from a project management and business benefits perspective. That is why it is important to understand how success is defined, and the methodology, of the various studies, before drawing any conclusions. Check the following symptoms:

         --Inadequate or conflicted staffing assignments
         --Underestimation of future project/resource requirements
         --Non-standardized expectations around IT resources
         --Improper allocation of IT funding in the annual cycle 

• The tough part is how to govern it right. The basis of good Project Management does start with the principles of the methodologies espoused by PMI , but that is not enough to ensure the Project or Program success or that PM leadership is sufficient. So, good PMO leaders and PMs has to know how to balance the needs of the project with the needs of good governance -- to use the right methodology, apply it at the right time and in the amounts that are really needed. Nowadays it seems that the focus is more on planning how you are to do it, coupled with a determination to adopt methodologies and standards. The result is that we spend more time planning the methodology and approach to the project rather than working on the technical requirements of the solution and how we can deliver it.  

• It's more critical to develop “Dealing with complexity” for large project mechanisms. Often, projects fail for many reasons outside the control of the project team. Management support is crucial and PMO, IT management, sponsor and exec leadership have to exert the learned management finesse to balance what is just the right amount of support, bureaucracy or governance.     

         --Minimize the number of irrevocable decisions
        --Minimize the number of essential competencies for success
        --Don't make success dependent on competencies you don't have and can't get
        --If a decision is irrevocable, make it pay off quickly 

In summary, Scope Creep, Failure to Obtain Stakeholder Commitment, Inability to Assemble a Team, and Failure to plan or make proper governance are all potential pitfalls. Large IT project success has a lot to do with the clear business case justification, being agile to think big act small, the quality leadership and related experience involved, the change management ability to understand and communicate where you are in the project and the governance ability to effectively manage risk at all levels. Both Project Management and Business Governance need be done right.

Posted in: CIO Debate,Lessons from Failures,Project Management