Why Analytics Fails to Deliver its Promises

The important bit is the communication of all insights in a joined up way which inspires simple effective action. An analytics professional is a translator of sorts.

Although analytics project is at top priority list of any forward-look organization, it has very low success rate to reach customer satisfaction, what are pitfalls, why analytics fails to deliver its promises, and what are principles to follow when doing analytics project.  

1.    The Pitfalls to Analytics Success

Lack of analytics talent, immature process and technology are all causes to fail analytics, more specifically:

1)     No clear business purpose for the analytics model being built. What's the decision you're trying to make, or the problem you're trying to solve? The fatal failures always seem to result from the strategic/visionary side. Far too often, the analytical work is started without a very clear goal of the actual problem, the business objective, and most importantly, the eventual deployment of the “answer”. A great analytical solution usually proceeds in reverse order from the implementation backwards to the data collection/aggregation stage.

2)     Incomplete or ineffective sponsorship. Are you working directly with all those who have a say in the decision, and having regular reviews with them? Or project managers cut you off from the sponsors or generally don’t let you communicate with the sponsors or others in the organization. If this happens, escalate to sponsors.

3)     One size fits all. You need to focus on a few key points before starting an analytical project, which may help in finding exact results as per the client requirement.

- Predictive Analytics vs. Traditional Statistics
- Group level decision making vs. Individual evaluation
- Business Objectives vs. Analytics Metrics
- Low Incidence vs. High impact occurrences
- Effectiveness vs. Efficiency 

4)     Failure to take a hypothesis-driven, rough cut approach to the problem. Quick and dirty analyses early on can simplify the scope and focus considerably, before investing in more detailed modeling. Analytics is above all a practical discipline that should be oriented around solutions to problems; the techniques and technologies should be a function of what is needed to solve the problem, not the other way around

5)     Clients who request “parallel universe” models that could answer any question. These are tempting but unrealistic. Clients who want to build a model to affect decisions they don’t own. The “if we build the model and conduct the analysis, they will change behavior” does not usually work. 

6)     Projects that stall—slow data gathering, low project meeting attendance, etc. If a project appears to be stalling, call for a sponsor review immediately to get clarity about the project’s importance and help move things forward. If this does not work, consider communicating a clear “end date” at which the analytics team will stop working on the project—this can prompt client action

7)     Unfortunately, sometimes the best analysts tend not to be the best action translators/inspiring communicators and vice versa.  The important bit is the communication of all insights in a joined up way which inspires simple effective action. An analytics professional is a translator of sorts. Their responsibility is to align the right data with the right analytical techniques to solve problems for the end user.

2. The Analytics Principles to Follow

But most of the pitfalls can be avoided when the following principles are incorporated.

1)     All the data and models in the world will have pitfalls if proper theories and principles are not incorporated.

2)     Modeling and Statistical analysis will out-perform management judgment the majority of the time, but experience + judgment + modeling will always out perform models only or management judgment only or experience only.

3)   The best models incorporate and cross-validate proper behavioral theory + management judgment + business experience.

4)   Analytics relates to a variety of data handling techniques used to justify certain business actions - there are lots of contextual insights, competition or customer feedback that must wrap around analytics to make truly effective business decisions.

5)  Figure out “WHY”:  Let's not forget data tells you who, when, where and how but not why. Once you have your derived data and use it to identify a meaningful subset of users and talk to them to find out "why" they are doing what they are doing. Knowing why is important for growing a business.

6)   During and after the model/final analysis, a seasoned analyst should view the assumptions and methodology of the process. If time permits, the model should be shown to a gathering of diverse background people within organization (different teams) and their opinion be taken. Let the technique not decide what we make but let us decipher what fits best and gives meaningful insights.

7)     Don’t overlook simple or sometimes obvious solutions; keep in mind, at complex circumstances, the science vs.art of project success comes from tradeoffs.

If you have a great team of analysts, follow the principles, you get the objectives correct, the actual operational aspects of doing the analytical work is quite simple, and the analytics should always deliver on its promise.

Posted in: Business Intelligence,Lessons from Failures,Principle/Practice