Saturday, November 2, 2013

Does Big Data need “Big Governance”

The value of big data is only multiplied by good data governance.

Big Data has big value, it also takes organization’s big effort to manage well and an effective governance discipline can fulfill its purpose, but does Big Data need “Big Governance”?

There’s misconception that "data governance" is an inherently heavy weight process. Just to clarify data governance is simply proactively decided how much and what type of effort is needed for different types of data; and then creating mechanisms to make management of the data easier and transparent to the business. It’s different from master data management, in the case of "master data", it’s used to "stitch" data together there is usually a fair amount of data management, cleansing, privacy considerations etc

Good Data Governance can never be "old solution", but a good habit: Data Governance really comes down to making a proactive decision about what data is needed, what it means to the enterprise and how to understand the quality of data (Big Data, Master Data, Reference Data, Transaction Data, etc.). Also how to improve the data quality where needed, thus, Data Governance should be seen as a good habit, not a software package or an old solution. The end result is their "big data" will be viewed as accurate and trustworthy.

Governance is also an attitude - one that gets confused with censorship and overwhelming controls. Big Data keeps the CIOs on their toes especially now with the very definition of "Data" is changing from a volume, variety and velocity perspective. Data governance was never about a structured data or an unstructured data, it was always about discipline, consistency and streamlining your processes to ensure you have right controls, and there's always a "known element" in what you do. And always be savvy about the user cases and workloads that run in these new environments. Treat these environments as an inherent part of your governance approach

Quality is only one dimension of governance. People are seeing governance as a synonym for quality; the latter can certainly be viewed as a dimension of the former but all too often seems to dominate the business case. Further, proper data governance is not a one time exercise but a constant review. Successful companies with great data governance have set up governance committees that include representatives from various parts of the business and agree on how the data will look (definition of the customer), be deployed to create value.

Data Governance is also not the same as Master Data Management: One of the problems is that DG get conflated with heavy weight processes like master data management. Effective governance doesn't always have to be defined under "building a master data management" or "data quality", otherwise you may never be able to initiate a data governance in first place. It's a proactive step, and one which will start paying its dividends as you start implementing in manageable steps. Of course MDM, Data Quality etc are relevant, and will come under the preview of Data Governance however it's also a lot dependent on organization maturity to reach those stages. 

Data Governance is to bring all stakeholders (IT, Business, Functional Departments n others) together towards a common enterprise objective, where each of the stakeholders have a role to play - you can throw various popular acronyms like Data Stewards, Shadow IT, Business Analysts, Data Champions, Governance Councils.Thing is, if you are trying to build new processes or integrate them with existing processes, the reasons you need at least some level of governance will become all too apparent.

Data governance is integral element of IT/Business Governance. That is exactly the sort of thing that you run into on a daily basis. The fear of uncovering problem areas that threaten the political capital of the management sometimes cripples the ability to really understand and leverage the data assets of an organization. A strong differentiation to effective management in today's market is the willingness to examine all aspects of the organization, whether that is established process and structures or even organization delegation and control. 

Data Governance is a Holistic Mindset: It used to be that we were seeing silos as marking the boundaries and territories of different lines of business within an organization. In those cases, the most effective transformation was the change in culture that made the data and process an organizational asset, rather than a LOB one. These days it seems that more organizations are going to the position of "its big data, therefore it is not my responsibility". So no governance is applied and equally difficult issues arise in the attempts to effectively leverage the data assets. Thus, an effective Data Governance needs to shape such a holistic mindset for business to visualize the full picture of big data and  through connecting  information dots.

Big Data does not need a big governance, but it takes agile governance practice to orchestrate organization’s information strategy, to ensure the big value can be achieved from the abundance of data and information flow.


Post a Comment