Thursday, May 15, 2014

Big Data, Smart Data

Putting "smarter" on your data is a team effort and directly related to the "smarter" TEAM you have.

Big Data is a big trend, but being bigger is not always equal to being smarter, having more and more data does not necessarily make any people or company “smarter”. So what are the key factors to transform Big Data to Smart Data?

Know the business objectives: One of the reasons to have big data is to cast a wider net so that people can better understand the diverse distribution of the “information” therein. Having tools that are smart or intelligent will help, but only to some extent. One can fit the best model using sophisticated tools; however, if you do not know the real objectives, most of the effort will be wasted.

Domain knowledge: Domain knowledge and expertise are key points to a better, smarter data deployment. Especially in areas like data discovery and mining. You cannot focus only on tools and how to use them. They are the means to the end, not the end. It’s very hard to find experts in mining and data discovery, also knows a lot of business. The best approach that works is for a human with domain knowledge to look at the data plotted in many different ways, through different lenses.

Accuracy of a story: Since all models are wrong, but some are useful, the usefulness of the models for big data analysis will often be determined by how accurate of a story can be told from the evidence, with the big problems of assumptions and biases being the enemy of truth in many cases. It is important of collecting information in a richer way to allow for elimination of biases and spatiotemporal location of data, helping the Volume and Veracity component of Big Data to be more "honest". Data are raw material and what to make out of it takes a team with expertise in subject matter, business needs, and excellent statistical and data analytical skills.

The smarter way to collect and categorize big data sets, even the basic intentions such as Analysis, Prediction and Prescription often require context and semantic attributes to be effective. Putting "smarter" on your data is a team effort and directly related to the "smarter" TEAM you have to manage data.knowledge and


Post a Comment