Thursday, March 28, 2013


There is no denying that the volume, variety, velocity and complexity of data from new sources (think social data for one) is growing dramatically and affects many industries heavily. The data volume is doubled every two years, but only 1% of data get analyzed, and more than 70% of Big Data effort fails. Is Big Data becoming a big headache for many organizations?

1.   Big Data Life Cycle Management

Big Data is really all about analyzing massive amounts of unstructured data and applying intelligence to it to derive some form of value - be it greater understanding of consumer behavior, ways to shorten process chains or some other benefit. But the question is whether organizations have even tackled the basics of data management - do we really know and understand what data we have, where it is, who is responsible for that data (oh, and how long we should keep it for, what security is needed etc. ) as well as what value is there in unstructured data, that structured data engines cannot possibly deliver?  
  • The life cycle of big data is to evaluate capacity planning for Big data services through timely intervals and expected growth cycles. Every bit of data counts, however, not every bit of data is created equal. The velocity and a metric for relevant time sensitivity is important in the collection processes of what you are analyzing and for what purpose pursuant to the predictive analytics needed to support the model coupled with levels of complexity. You may wish to start with an industry leader model as a systematic approach to master volume, velocity and complexity. 
  • The idea with big data is not to push it out to the information consumer but to analyze it using Analytics tools,  with the hopes of extracting actionable business information and opportunities that you can then push out to the information consumer (those in your company, or your business partners, who can make meaningful changes based upon what the information tells them). And YES,   it is a very worthwhile project for many industries. 
  • How to Manage Big Data without uplifting cost:  If we look at Big Data as an opportunity to gain the insight,  we all want without the uplift costs of structuring the data to our pre-conceived notions of what the data can tell us, then the new value and opportunity is to learn something that we would otherwise miss point. Here is really first to find a business case where big data can provide the solution for.

2.    Big Data Analytics Starts with Business Case

Now we may ask if we can use the Big Data and if we have a problem where its solution might come from this data and that solution would give us competitive or comparative advantages. It is clearly big challenge to treat unstructured data such as behavioral response by leveraging traditional technology as well as new approach. 
  • Start with Business Case: "Big Data" will let us eventually open eyes beyond transactional value of current IT services, and it will be addressed as a form of "Decision Supports". The business value of Big Data will be derived by someone first figuring out what business problem it can solve, a problem that other options cannot. So start with questions, and explore Big Data for:
         1) Customer Insight -Understand customer with data-based empathy, and optimize customer  
             experience life cycle  
         2)  Decision Making Insight -Provide talent the right information at the right time to make right
            decision timely.  
        3)  Talent Insight --Know your talent in and out, analyze talent pipeline, develop your workforce
            analytic competency.  
  • Let Big Data tell you something unknown about unknown: Some argument point include: This is less about people asking the right questions in the context of what they already think they know about the world and letting the information tell us things we hadn't realized, but it’s having an interrelated impact. In this context, every business needs Big Data and the analysts that can use it effectively. If we analyze information quickly without having to first scrub it and index it, then we have a really powerful tool for changing our businesses to provide the products that matter. 

3. Big Data Talent, Structure, Framework, Test and Metrics

That will be the most difficult and innovative task on how to turn unstructured data into a strategic asset, an asset whose value can be measured. The problem is most organizations do not have the people who can do that. It takes talent, structure, framework, methods and metrics.
  • Assuming structure is required at a semantic level, in order to produce meaning, and given that much of the effort in Big Data is directed toward putting a structure on sources of unstructured data. Also you may want to think first about a Big Data framework before selecting a product, platform or a service.

  • Big data need embrace a predictive analytic methodology for the development of methods and metrics that will deliver the greatest impact given a desired business schematic that the front office can embrace using an array of predetermined structured and unstructured data. Without proper visualization and analytics,  gathering so much data is not going to help. Not many products come bundled with all these components. Again, integrating all these components is another challenge

  • Whatever you do there will be some smoothing or refinement through testing the results and refinements to your model and about all use cases. There is the test to learn which methodologies or methods will work in an effort ascertain an acceptable result. Then there are the use cases that must demonstrate to the stakeholders the challenges needed through metrics to solve tactical and strategic business problems

  • Last, but not least, Big Data talent is on demand: Big Data talent should have 3-‘C’ quality: Curiosity, Creativity and Concentration, "PHD" is nice to have, not must have, the analytic skills and business knowledge are both important to master Big Data
Is 80/20 Rule also applied to Big Data and soothing your headache? Some say,  taking decisions effectively & efficiently is doing it with just 20% of the information and estimating the other 80%. There lies the rub: how do we get the 20% of what we need in a timely fashion? It takes talent, strategy, methodology and measurement.


A special thanks for sharing such an informative post. I definitely learned a few new things here.

Post a Comment

Twitter Delicious Facebook Digg Stumbleupon Favorites More