Is IT Benchmarking Waste of Time

The benchmarking has several triggers and objectives, and often a variety of outcomes and downstream initiatives.

IT benchmarking is one of the most debatable arenas in IT management. Can benchmarking be useful? Or is it a waste of time? What’s the science and art of benchmarking?

There needs to be a strategic reason for benchmarking in the first place. Setting up benchmark is to help management measure the work, not restrict or set up barriers to the work. Before a benchmark or any other metric is applied, the proposer needs to specifically state what actions will be taken in response to variations in the value of that metric. Executive support of the initiative is critical. If an organization is not willing to change directions based on poor results from a benchmarking study, then it should be cautioned that this is a good use of time, money, and resources in the first place. The measurement studies show that benchmarking have several triggers and objectives, and often a variety of outcomes and downstream initiatives...not only for cost optimization and vendor renegotiations.  

The question should be what benchmarks should be used. Benchmarks can be a useful instrument for comparing services from one provider to another; the selection of suitable "peers" to benchmark against is best driven by comparable scale and complexity of the service towers under scrutiny; which is often not limited to industry vertical or mico-vertical. Peers from the same industry may not be suitable peers for IT benchmarking purposes for several reasons - size, spread, business model, service delivery models, legacy, and so on. Put another way, the peers similar in scale and complexity for "desktop services", may not be the most suitable for comparing against "servers" or "application support". It really depends upon who you are benchmarking against. 
1) Industry 
2) Relative Size 
3) Technology Adoption Curve / Strategic Use of Technology 

Benchmarking is not one size fitting all. Every organization is different with different requirements, technologies, and workforce. Benchmarking is like a quick health check which highlights the areas where improvement or quick fix might be required. Benchmarking often provides good data and highlights issues and its incumbent, IT leaders have to use that data in a way that motivates and improves the organization. Benchmarking is a good measurement for tangible stuff. However, value added services are usually out of the standard benchmark scope. Many organizations use it as a hammer to drive out the cost. It can become a de-motivator to the workforce and cause organizations to lose competitiveness while achieving only a short term cost improvement.

The reliable benchmark shall come through a detailed metrics analysis of variables. As opposed to comparisons against survey gathered data, which suffer from the anomalies of averaging (comparing apples with strawberries), a detailed metrics analysis of variables, comparing against carefully selected peers for each service tower, can highlight issues and provide valuable insights to the enterprise and to the service providers involved. A well-defined analysis, supported by a current database of metrics and a robust methodology is best viewed as a diagnostic tool. It can be seen as "switching on the headlights when driving through rough uncertain terrain in the dark". They need to be able to identify that for every dollar that spends on IT, you generate x dollars in profit and that is contributing to the overall company's performance relative to its peers.

Benchmark can help management understand cost variables.  On a more micro level, benchmarks can help procurement understand what the appropriate cost of goods and services in the market should be to be able to stay on top of purchasing contracts. The cost variable is one of inter-dependent and relevant variables that need consideration. The cost of a component part is one amongst several elements that comprise a service tower. The analysis needs to consider the entire stack of "widgets". Most of the buyers have made it a periodic exercise and built into the contracts signed with Service Provider.

At the senior leadership level, the CIO needs to be focused on a complete picture. Benchmarking as a management tool remains popular for its value in providing insights and metrics to help chart a course....hopefully towards better governed, efficient and effective IT as the business enabler.