Academic references to outcomes theory

The principle

*Indicators of any type are best thought of as being measures of boxes from an underlying (implicit or explicit) outcomes model. The outcomes model sets out your underlying theory of change containing the high-level outcomes you are seeking and the lower-level steps it is believed will make them happen. Indicator sets should never be developed without surfacing the underlying outcomes (strategy) model. model. *

Dr Paul Duignan, Three Minute Outcomes Video 7 discussing this principle

The problem

People often attempt to construct indicator sets - also called indicator framework, outcomes sets, outcome frameworks, monitoring frameworks, Key Performance Indicators, KPIs, performance metrics - from, as it were, a blank piece of paper. They ask themselves 'what should we be measuring in regard to this organization, initiative, sector, community or country?'

The best way to answer this question is to answer an earlier question ‘ what is it that we are trying to do here?' In other words, 'what exactly is the indicator set attempting to measure?'

Where there are multiple stakeholders involved in constructing an indicator set, they may hold in their heads somewhat different versions of the outcomes model underlying what it is that is being worked on.

This can create confusion as you can end up with what appears to be a discussion about indicators but which is actually a discussion about outcomes and strategy. This arises in indicator set construction when you do not have a way of representing the underlying outcomes and high-level aspects of strategy which the indicator set attempting to measure.

The solution

The solution is to always surface the underlying outcomes model of what it is that you are trying to do in regard to the organization, initiative, sector, community or country. Once this has been draw, discussed and agreed on, it makes it much easier to quickly identify the indicators that best measure the underlying boxes in the outcomes model. This outcomes model does not need to drill-down into the details of strategy. But it needs to be comprehensive enough to provide boxes onto which you can map all of the indicators that are likey to appear in the outcomes model.

The outcomes modeling approach also has the advantage that it allows you to: identify the ‘level’ within the outcomes model of a particular indicator; how low down the outcomes model any proxy indicators are located; and the extent to which the indicator set covers the boxes within the outcomes model.

Once the indicator set has been built, if you have developed the underlying outcomes (strategy) model at the same time it is invaluable in the implementation phase. This is because you can immediately use it to start guiding the activity which is going to be undertaken to improve the outcomes measured in the indicator set.

Example

The example below shows an indicator set which has not been modeled against its underlying outcomes model.

The outcomes (strategy) model below shows the above indicators mapped onto the underlying outcomes model. This mapping shows you immedaitely which boxes are, and which are not, being measured; the level at which the indicators are being measured; and which parts of the outcomes model have been captured within the indicator set.