What Causes Islands of Disparate Data?

The only thing that will stop data silo is strong and effective governance

Although data is life blood of digital enterprise today, very few organizations can manage their data effectively, most of them are inundated with overwhelming data flow, from small data to large data set to monolithic data silos, from esoteric data definitions to rigid data process. In order to figure out better data management solution, the first step might be to ask: What causes islands of disparate data? The key reason can be classified into:  

1. Data Governance -Principles/Policies/Processes

What causes data isolation? The most common causes is lack of data governance disciplines which include bad planning, lack of basic data principles, disciplines, procedures, standards, inconsistent data management policies/oversight/governance. In larger corporations, the teams building solutions simply don't know that data is already governed by the enterprise. Without a comprehensive metadata strategy, how would they know what data standards exist or where identical data is stored?

What causes islands of redundant, disparate data? It’s a 'project driven' data strategy. Project Managers and project teams are under extreme pressure to implement a given solution as quickly as possible and expending the least amount of money possible. In the absence of a strong governance framework, they'll look at what data they require to implement their own project and deem it easiest/quickest to create a database with all their needed data in their own domain and write their application directly against it. So, it causes data redundancy, bad data quality, data silo or unnecessary complexity, then the islands of disparate data are formed. 

2. Data Stewardship 

Lack of data ownership is the other lead cause. In some cases, it’s the abandonment of data ownership. Usually IT plays such data steward role, but steward is not necessarily as ownership, it takes cross-functional collaboration to breakdown silo and manage data seamlessly.  

How many organizations focus on data stewardship? There are generally no attempts to gather and hold knowledge on the how-what-where-who of data in an organization. It would be nice to have a data ‘guru’ or team of data experts, well mixing data architects or other technical roles, to guide the answers to the key questions:
What data is needed?
Where is the data?
Who owns it?
How do we get it? 

… 

The tension between centralization vs. decentralization: Every large corporation has tension in it, between the forces for centralization and those for decentralization. This is as old as the hills. Those in the center can come up with the strategies, but are incapable of delivering the solutions to the business units; those attached to the business users can deliver tactical solutions, but aren't interested in the big picture. So the challenge is to meet the demands of business units while coordinating the disparate silos of development where possibly providing achievable enterprise-wide solutions. 

3. Finance

The other cause of ‘Island of data’ is finance related. Most large companies assign budgets on a departmental basis, as they set up functional goals. In such cases, departmental managers are only interested in meeting their own goals and in many cases will only finance IT projects that further those departmental goals. So there’s no unified IT budget for data management or try to maximize ROI by sub projects, it all causes the issues to manage data efficiently and effectively cross-enterprise scope..

The organizations have built up these data islands over years of effort and they aren't torn down over night. The only thing that will stop that is strong and effective governance. Data Classification and strong governance can turn the tide though and set you on the right path.

Posted in: Big Data,GRC,Information/Data Management,Lessons from Failures