Sunday, November 16, 2014

See Through the Big Data from Multiple Views

 Big Data thinking and implementation are neither linear nor single dimensional.

For the majority of organizations, Big Data is still the big puzzle, they put big investment on it, but it seems they really don’t understand it; how to see through the data and truly capture insight and foresight from the dots connections?

Top-down executive’s perception of Big Data. All their analysis begins with this data set, which generates prediction scores, and the outcome of their strategies and decisions they incorporate depend on the accuracy of the data. Bad data eventually lead to worse decisions and actions, which, can prove to be very costly for corporations. However, iis not uncommon to find organizations that refuse to start looking at data at the executive level before the data is "conditioned" and "governed". Once you expose information at the executive level, no middle manager anywhere is going to pass up an opportunity to make their data more reliable. Bottom Line, if you start building a Data Governance methodology and structure before you start exposing raw information to executives you'll never get out of the gate. Follow the logical steps 1) Initiating an analytics program at the same time you start wrapping your head around data governance. 2) Finding a way to expose the 'reliability' of information within your business intelligence environment. When it is bad data, say it is. 3) When the 'bad' information starts to be seen by the executives you'll, at least, have a strategy to reign in the clutter.

Quality & System Engineer’ view: One of the important steps needs to be taken is Data Cleansing - converting data that originates from usually different legacy data sources into a single structured source. It is only when we harmonize and integrate the data into a structured database that magic then occurs.....the ability to slice and dice the data to uncover meaning and insights. Also, there’s time taken for testing - most projects usually under-estimate the amount of time required to test the developed solution. As all systems engineers will attest - you need to test to validate the model works, and then test again with the user to verify the model does what the user wants. Yet, to test robustly, one usually requires building two models so that you can independently arrive at the desired outcome. Data quality is the fundamental issues and requirement about the data analytics.

Users’ view of Data & Analytics: Users are fighting change. It takes the time to clarify exactly what the user wants – many times the project deliverables usually are not what the client originally specifies. , when doing a BI project, do a discovery where you  meet with users, as well as everybody else critical to the project's success. Many technique issues of data processing such as data cleansing and data quality are not glamorous and sexy to the user. Yet they are perhaps the most important components to ensure the project team delivers the project on time, within cost and to user specifications.

 Data Architect’s governance Insight: Data Governance as a concept sounds logical. Yet as in all standards enforcement, one ends up building layers and structures upon existing ones. And since it impacts on users directly, the implementation of a project ends up fighting against human resistance to change. Better to seek out a solution that "translates and converts" existing data structures into an integrated data structure. And the panacea is that this seamless migration can all be done with the right disciplined and structured application.

Big Data thinking and implementation are neither linear nor single dimensional, only understand it through multiple views, from both top-down, bottom up and middle out, and outside-in, Big Data myth can be untangled and perceive it from the different angle.



Post a Comment