Big Data, Quality Data

There is quality data, but there’s no “perfect” data in Big Data world.

Big data has big potential, also face big management challenges, from understanding three 'V's: Volume, Velocity, Variety of Big Data characteristics to mastering three 'W': WHY, WHAT, WHO on how to handle Big Data.

First things first, Quality Data is the desired data status: By “quality data” –it means clean, organized, actionable data from which to extract relevant information and insight. To get this data, you must have deep domain expertise in the acquisition, collection, management and delivery of structured and unstructured data, and you are equipped to aid both in crafting a business’s content strategy and in executing against such plans. Here are a few more principles:

There is quality data, but there’s no “perfect” data in Big Data world: No matter how "clean" your data is, it suffers from the limitations of chaos theory on its accuracy and applicability to the "real world". You might have very clean data on your customer profiles, but necessarily that data is incomplete. The accuracy and compromise will continue to coexist across the span of information management. Hence, Big Data Quality efforts need to be defined more as profiling and standards versus cleansing. This is better aligned to how big data is managed and processed.

Quality data is like the Holy Grail, businesses all want to achieve it; but not sure if it’s very doable: Business operates in the real world, and the real world is muddy and chaotic. Organizations need tools that deal with muddy and chaotic data, not a focus on making the data adapt to somewhat weaker tools.

Quality data leading to quality decision: Technology obviously plays a significant role in the content practice, contextual understanding, and once you get good data, you want it delivered to the end-users, via data-feed, API, web or mobile application. Quality data means how to transform the clean data into the useful information, and deliver it to the right people at the right time and location in order to make the quality decisions as well.

Quality leadership is crucial to manage quality data: The strong leadership via cross-functional collaboration can add a lot by putting resources on this to support the business, construct a vision and hire strong resources to develop the vision and humbly take the time to understand the business, to construct the business intelligence structure the business needs; to well mix on how to execute it and for how much of the data can be provided back to the business. Many times, the discovery of this is the hard part to see not only linked to metrics and KPIs, but also back to continuous improvement and visibility of the health of the enterprise.

Big Data, quality data, quality is not equal to perfect, but good enough data to transform into business information and insight for capturing the trend or optimizing customer experience. As it is also important to leverage data quality and cost/benefit analysis. Still, quality data is means to the end, the end is how to run a high quality, high performing and high mature organization.