Activity ≠ Impact (and the ROI of measurement clarity)

Measuring the stuff we do, and achieve, in business is critically important – performance data and results information give us necessary insight into what’s going on inside AND outside of our organization. Many organizations invest heavily in measurement so it’s important to be sure that that time and effort is spent gathering the right information.

Unfortunately, I’m not convinced that organizations regularly remember to take the time to think this through before they get going with measurement. Sometimes they aren’t given the option when a well-meaning funder, government agency, or Board places misplaced measurement demands on them.

Whatever your story is, your organization must be laser focused on maximizing the value you are getting out of the data and results you are collecting.

So how exactly can you do this? The trick is to be sure about (1) what you are hoping to learn from measurement and/or what question(s) you are trying to answer through measurement and (2) what you intend to use the information you get from measurement for. In other words, the key to getting the greatest possible return on your measurement investment is to be crystal clear on your purpose for measurement.

Getting clear on the “why’s” and “what’s” of your measurement activities isn’t really that difficult – you just need to take the time to talk it out amongst interested parties and get agreement before you get started. Here’s a list of some of the typical purposes for measurement I see:

  • Compliance reporting – Is our performance reliably meeting regulatory standards?
  • Strategy management Q1 – Are we doing what we said we were going to do?
  • Strategy management Q2 – Is our strategy/are our efforts producing the expected results?
  • Strategy management Q3 – Are our results still delivering value to our customers/stakeholders AND differentiating us in the marketplace?
  • External benchmarking – How does our performance compare to others in our industry? Where do we stand relative to our competitors?
  • Learning and improvement – Do we have any performance gaps and how much do we need to improve? What are we doing well, why, and how can we transfer successes to other parts of our organization?
  • Operational management – Are we doing things right and what are the results?
  • Impact assessment – Are the results we’re producing/enabling having the reach we’ve planned for? Are we creating the community/societal outcomes we’re aiming for?      Etc.

It turns out that you’ll probably have multiple purposes for measurement across your organization. The key, however, is to realize that different purposes for measurement mean that you’ve got to have different types of metrics in your toolkit.

Being sure to carefully consider the unique purpose for measurement for any given measurement situation will help prevent a misplaced measurement scenario.

Let me show you what I mean with an example of a frequent purpose for measurement – metrics mismatch:

Let’s say we are running an English as a Second Language program for newcomers and we want to assess the impact our program is having in helping newcomers become contributing members of the community. Since we wish to assess impact, we are probably looking to answer the following questions:

  • Are we helping as many people as we’d planned leave our program with the standard of English literacy we’d established for grads of our program? (i.e. Are the results we’re enabling having the reach we’ve planned for?) and/or
  • Are we helping to make it possible for newcomers to participate in and contribute to the community socially and economically? (i.e. Are we creating the community/societal outcomes we’re aiming for?)

Beginning with these questions in mind makes it pretty obvious that it will take very specific metrics to give us the answers we need and help us assess the impact of our program.

But let’s say that we decide to jump into measurement, sidestepping a discussion about our purpose for measurement and the questions we’d like to answer. Not unlike many organizations, we might find that we end up measuring the completion of a project to update the ESL curriculum. Or, we might decide to measure the growth in the number of ESL classes we offer to newcomers. And we might even invest in a process to track and measure the average number of newcomers actually participating in a certain percent of their ESL classes.

And while these may be important things to measure (they do give us critical information about our ESL training activities), the problem with these metrics is that they help answer a strategy management question (“Are we doing what we said we were going to do?”), not our impact assessment questions.

Eh violà! If we try to apply these metrics to our attempt to assess impact we will end up with a measurement mismatch!

Unfortunately, measurement mismatches like this happen every day in organizations. The result is sub-optimization of the time and energy organizations invest in their measurement activities and the feeling that measurement isn’t delivering the expected return on the effort.

So here’s a quick recap of the steps you can take to minimize the risk of measurement misalignment and maximize the value you are getting out of the data and results you are collecting.

For any scenario where you are considering measurement:

1 Begin by developing consensus on the purpose(s) for measurement

2 Develop the key questions you’d like to be able to answer through measurement

3 Identify a short list of metrics that will give you the data, information, and insight you need to answer your key questions with confidence

4 Test out these selected metrics and see whether they do the job (if they don’t change them out and try something new), and

5 Regularly review and validate your purpose for measurement and the associated key questions – be open to change and evolution (and remember to change your metrics accordingly!).

Measurement is an important activity for all organizations but no one wants measurement to become an effort and money black hole. Your goal should be to ensure that you are getting the most possible from your investment in measurement. Putting your purpose for measurement at the center of your metric selection and measurement activities will go a long way to ensuring that you receive the maximum value for your measurement investments.