Measurement Software

Many organisations are investing in software to automate their measurement and reporting systems. I’m a great fan of this sort of software, as long as it delivers real benefits and is implemented by people who really understand how to make measurement systems work.

Most solutions will enable you to view tables of data, with Red, Amber and Green “traffic lights”, showing performance vs. target, or performance vs. last year. This is probably useful in helping you to focus on underperforming measurements. It’s probably less useful in helping you to understand overall trends – are things getting better or worse, and importantly, why?

A Data Report with graphical presentations of performance over time does at least help you draw conclusions about performance over time. I’m not a great believer in Bar Charts (right) such as this though. I’d rather have a Line Graph and some analysis to see if any trend is statistically significant, or if the “trend” is actually just part of normal variation. And, I really hate those graphs with dozens of lines or bars, where it’s impossible to see anything significant at all!

One of the really useful features of the best measurement software is the ability to create Scorecards and to drill down to find out exactly where performance is good or bad. You can get a high-level traffic-light view (left), see where performance is below target and drill right down to the part of the organisation which has under-performed. Many of the best programs are “Balanced Scorecard Compliant” which means their features and functionality meet the requirements for building a robust Scorecard system. They will enable you to build Strategy Maps – the starting point for all effective Scorecards - and to include the core Scorecard components of Goals, Measurements, Targets and Plans.

This is all very good, but requires you to ask the basic question: “What are you trying to achieve?”. There are two aspects to this:

If you don’t have good answers to these, then you will happily fall into the classic software implementation trap of “garbage in, gospel out”.

What you are trying to achieve as an organisation should determine what you need to measure. If you are adopting a Balanced Scorecard approach, this will force you to ask four key questions:

How must we look to our Shareholders (or key Stakeholders, for the public sector)?	How must we look to our customers (or those to whom we provide services, for the public sector)?
What are the processes at which we must excel, if we are to satisfy our shareholders/stakeholders and customers?	What do we need to have in place in order to innovate, grow and develop our capabilities for the future?

The answers to these questions will lead to definitions of the goals you must achieve, and then the measurements required to track progress towards them. Unless you are already working in a very smart organisation, it is highly unlikely that you will currently be measuring all the right things. Taking this approach, in our experience, it’s not unusual to find 70 – 80% of the required top-level measures simply don’t exist in most organisations.

So, simply applying the latest whizzo software to your current measurement system is not likely to give you much more than a very efficient way to capture, analyse and report on the WRONG things.

The second question addresses your expectations of the measurement system. If you want more graphs and charts, most of the software will do that, but can your people make sense of the information and use it to drive improvement? If you want to be able to drill-down and find hot-spots and problems, that too should be easy enough, but do your people have the root cause analysis and process improvement skills to implement a workable, permanent solution?

My point is: you can have information that is a million times more available, visible and visually attractive, but unless your people have the improvement skills to do something with it, it won’t make much of an impact.

In my experience, to get the best value out of an investment in measurement software, you must recognise the need to develop people who are capable of interpreting the information outputs and then have the skills to do something about it. But first, you must decide what you are trying to achieve and therefore what you must measure.

None of this is very difficult, it’s just that most of us don’t leave school, college or university with these skills. And, by the time we are in management positions, it’s a bit embarrassing to have to admit that we don’t understand the difference between a Bar Chart and a Histogram, or how to read a Line Graph, or understand the difference between Special and Common Cause variations. This isn’t about learning to be a statistician, it’s being able to look at a set of data and get real meaning and value from it.