Untangle the Integration Hairball

Shoot the Tangle to See from Different Angle. 

With the increasing speed of change and information growth, IT management faces the new challenge: Whose fault is the integration hairball? What's your take and how can one untangle such hairball or minimize its impact?

1. The Cause to Hairball Problem 

The hairball is an example of “creeping normalcy”, which refers to the way a major change can be accepted if it happens slowly in unnoticed increments. Point by point, year after year, you build application connections one at a time in silos. Project owners gain tactical expediency in the short term, but eventually the integration hairball grows so unwieldy and complex that it begins hurting business performance and consuming IT budget with exorbitant maintenance costs.

The alternative definitions and an explanation why the hairball problem persists:
1) Application - a fit-for-purpose collection of components designed to acquire, transform and distribute value-added digital assets in the form of structured and unstructured information.

2) Integration - the process of aligning the physical, logical and semantic aspects of two or more applications in the context of a larger design, which must also satisfy the definition of an Application, above.

3) Information - an enterprise-specific collection of individual (instances) or sets (classes) of values deemed to be important for the conduct of business.

Information Architecture (as opposed to: Interface Architecture, Infrastructure Architecture, and/or Interactions Architecture) - an intersection of structural, functional and contextual aspects of Application communication, human and otherwise.

4) Metadata - the components in an Application with the express function to act as contextual intersects to other Information components. 

The maturity of an organization is an important factor. A lot of it has to do with maturity of the organization and the rapidly increasing complexity of the business causing the IT approaches to solving them become obsolete. Another important factor is whether the company is taking the short range view only, or are they willing to consider longer range solutions. The people must understand that simple, quick solutions look good for the short run, but that they can cause tremendous problems in the middle to long run. 

2. How to Untangle the Mess 

Finding fault is very often both time consuming and accomplishes nothing. The real issue is to untangle the mess.

What it means is that the underlying architecture should provide application decoupling, broad connectivity support, shared data catalogue, support for bulk, individual processing and replication of data, real time and scheduled processing, advanced transformations and format translations, data quality control on the fly, full tracing and monitoring, and with the support of publish/subscribe and services interaction patterns.

When possible, map out the whole hair ball before taking it apart and creating a central data store of exchanged data. After mapping things out, one then goes about the actual untangling in small steps. Most of the steps tend to be straight forward, but every so often, the data is found to be managed in a very contorted way that has to be carefully analyzed before proceeding.

Starting small is a good approach, but scaling up from there has to be carefully architected. The transition from a small initiative to demonstrate value to a completely integrated solution is a journey that continues as the organization's information needs continue to evolve with the business

The difficulty might be in transitioning from the existing to the new way of doing things". Hairball prevention service" is brilliant. It should consist of two basic strategies 1) Contain and Atrophy or 2) Migrate and Retire. Most of the complexity involved in "untangling" this situation is in the eyes of the beholder. It's not as difficult as people make it sound. So shoot the tangle to see from the different angle.

Posted in: Information/Data Management