KPI Integrity

  

 Email Article  

Tweet  

  

By: Brian Bohan, Vice President of Worldwide Sales Consulting

At best, a metric of dubious integrity is ignored and rendered pointless. At worst, it is interpreted on faith and sets in motion decisions and results based on faulty data and assumptions. Providing users objective indications of a metric’s veracity and accuracy is critical. To do this effectively, it is necessary to provide the following:

1. Data Source – This can be either qualitative or quantitative. It is one thing to list the data sources used and assume consumers of the metric can infer meaning from that; it is better if the rating is derived from an asset repository or CMDB where underlying systems are clearly described and rated.
2. Owner – Here we point to reputation in an organization as an indicator of trustworthiness. An owner with a solid reputation will confer on his or her metric that same level validity in the eyes of the broader organization.
3. Frequency & Method of Update AND Chain of Approval – Two indications of quality are captured here. First, there is the frequency and method of update. We want to capture the “freshness” of a metric. It is important to note that frequent updates do not necessarily mean one metric is somehow better than another, just as infrequent updates do not mean it is any worse. The frequency of the updates should be taken in context with the rest of the information here to infer whether the metric is up-to-date and relevant. The second concept speaks to transparency of the approval chain for a metric. Some metrics that are confined to small domains or even sandboxes may not have an approval chain, whereas those that apply broadly certainly should, and this chain should be made visible to users.
4. Quality Rating – This would be a crowd-sourced rating akin to those found applied to eBay sellers.

From my previous post, we learned that KPIs must be defined in a way that is understandable, meaningful, and measurable. KPIs are rarely defined in such a way that their fulfillment would be hampered by factors seen as non-controllable by the organizations or individuals responsible. It is therefore that providing users objective indications of a metric’s veracity and accuracy is critical.