Four Aspects in Measuring Enterprise Architecture Effectively

The set of possible measurements for EA is very large. This is partially because EA is the glue between strategy & execution, Thus, the main principle is: Focus on outcomes, "beginning with the end in mind.", Generally speaking,  EA benefits indicators can be grouped into the next four categories.

1. Communication Value 

Successful EA effort will enforce cross-functional communication; on one hand, EA addresses abstract concepts like business motivation, and on the other hand, addressing how business processes are enabled in very concrete and material ways such as IT applications and infrastructures.

• Enforce stakeholders’ communication: Success criteria for successful EA organizations are improvements in KPIs for their stakeholders. Complexity is inherent in negotiating how to please many masters and quantifying their contentment quotient in hard currency. An enterprise defines objectives of its enterprise architecture practice including different priorities for the activities. To measure the success of enterprise architecture, multiple types of metrics are needed. 

• Communicate the Change: EA is about change. And you need to measure if you can manage changes. One central EA objective is to improve the improvements. So, measure your business improvement processes and check if you are successfully changing it.  

• Convey Triple Bottom Lines: There's the basic premise that EA should be measured by its impact on the bottom line. It is not just economic measurements, but environmental and social measurements as well. The latter two still being less standardized, and thus, more ambiguous and harder to actually get agreement on what exactly to measure. Still, there is potential to the idea since it means an enterprise cannot simply stop accounting once a good service passes through its doors. 

2.  Strategic Value

EA is a missing link between strategy and execution.  So one of the measures can be the effectiveness of execution. In order to measure the value of EA, one has to be able to see the enterprise holistically as an organism is struggling to survive in a hostile environment, not just as a means to an end.

• Generic Enterprise Architecture Capability Map, Metrics and Measurement capability are placed in the business perspectives; EA ROI is based on that organization's TCO models as Dollars are usually far more persuasive than logic. The Capability Portfolio ranges from Advocacy Curve measurements (how many people are actually doing what EA is influencing them to do, and how well) to regular old operational and HR metrics for the EA team as a unit.  

• Measure Strategy Alignment & Execution: Although the focus on reducing TCO is important - what is more important is the competitive edge and innovation right.  By looking at metric(s) that show integrated project planning investment in strategy alignment & execution vs.other - which by the way directly influences reducing TCO. 

• How "LEAN" is EA without losing effectiveness - the cost-effectiveness of the organization's EA. Many times it seems as if businesses focus solely on cost reduction which can be counter-productive. Most cost-centric initiatives are indeed counterproductive. It's important to look at how an initiative will contribute to or decrement other goals as well. The capability-based analysis makes sense to ensure that completeness.  

• Tracking the EA & reusability and generating a GEM (Good Enough Metric) or any other KPI, to be review with the senior management on a periodic basis. By having an asset registry- to log all enterprise "assets" or reusable components that can be further reused by various "teams" within an enterprise.  

• EA KPIs stay focus on measuring some cohesive business goals:
  -Business strategy-execution progress measurement
  -Business culture Index, productivity
  -Business Communication Effectiveness
  -Business growth opportunity
  -Cost Structure Optimization
  -GRC Management
  -Sustainability  

3.    Tactical Value

At the tactical level, the purpose of 'doing EA' is to improve the efficiency with which an organization operates. Measures of efficiency would vary based on the nature of business the organization is in

•  Processes metrics and KPIs to validate improvement: Focus on measuring the effectiveness & efficiency of EA by measuring the processes and KPIs performance within the organization, evaluating the improvements in these processes and KPIs and validating these improvements with the goals or the benchmarks. 

• Cost metrics to demonstrate the cost-saving, an increase in revenue generated by EA. This metrics is appropriate for showing value for standardization, resource-consolidation, and exploitation of opportunities. As far as the expense of EA. There are a lot of organizations that are getting into EA without a good grasp of how much capital and resources they have to devote to a minimum-functional EA 

• Time Metrics to demonstrate the improvement in the efficiency of the processes. Ensuring the reusability of components /assets is part of the employee's annual performance plan/goals so that EA and reusability can be enforced in day to day routines. Otherwise, it has a tendency to stay in the discussion boards and not get implemented. The best way to measure the success of Enterprise Architecture effectiveness, when you will start receiving calls from the LOB executives for their needs.  

• Activity Metrics to show the efficiency and output of enterprise architecture. (Example: Number of architectures reviewed, number of standards created, number of business problems solved, reduced production defects related to the architecture). This is good to keep track of the efficiency of EA, but these are not good to demonstrate the value of EA. Also, there can not a thumb rule to define the processes and KPIs within the EA purview. The processes, KPIs, measurement methods, metrics for data collection and representation would differ from one enterprise to other 

• EA Measure: Is Architecture right (freeze), the documentation complete and delivered on time
  - Has the enterprise architect delivered against all projects (PM feedback)
  - Demonstrable cost savings
  - Risks identified and solved
  - Time spent right
  - Auditor report
  - Customer and Peer 360-degree feedback
  - Architecture Maturity Matrix

4. Governance Value

Governance right is the other key area EA can deliver value. Determining costs associated with risks is as much art as it is science in many cases, but where EA can point to a mitigation a cost-benefit can sometimes be determined. Of course, many risks are not uncovered at the level of abstraction of EA efforts and are not known until detailed implementation architectures/plans are developed.

• Risk Metrics to demonstrate the impact of EA on reducing risks for the enterprise's business. Innovation and Governance related activities are good candidates for these metrics. EA should demonstrate the risk of not adopting innovative ideas, new technologies, and opportunities.  

• Governing Right: The scope of enterprise architecture is broad. To measure & monitor performance & effectiveness of the EA, organizations can use the performance monitoring framework such as a Balanced scorecard.  At a high level, it includes
  (a) Cost reduction by standardization and resource consolidation,
  (b) Improvement in efficiency by process optimization,
  (c) The exploitation of opportunities by bringing innovation and research,
  (d) Solving business problems (example: merger, divestiture, expansion of business in new market/regions, new products, etc)
  (e) Governance to ensure an operational model of the enterprise is aligned for its vision 

• Avoid pitfalls: EA makes it easier to avoid the kinds of problems that "order-taker" IT organizations run into all the time:
  -well-intentioned projects that the business wants, but don't deliver value
  -implementation of redundant, complex, and duplicative capabilities
  -ignoring root-cause problems because they are politically difficult to address
  -infeasible or conflicting business strategies that never get resolved but drive wasteful or conflicting behavior
  -un-vetted or un-reviewed business decisions and initiatives that optimize one area of the business at the (greater) expense of other areas

Usually, the organizational management changes the focus of EA based on its need, the combination of metrics for measuring EA effort would change from time to time. Accordingly, the metrics should be adjusted, but the important thing is, EA is the means to the end, always focus on business goal and strategy execution via communication well and governing right.

Posted in: EA,KPIs,Performance Master