Be wary of overly ’tidy’ indicator sets

principlesbutton_med

Academic references to outcomes theory


The principle

Indicator sets that are 'tidy '- in the sense that there might include, for instance, 5 outcomes with 5 indicators under each outcome - should be treated with caution. This is because there is no theoretical reason to believe that there will be the same number of indicators at a similar level within the outcomes model underlying any particular indicator set.




Dr Paul Duignan, Three Minute Outcomes Video 7 including discussion of this principle


The problem

There is a natural tendency to created ’tidy’indicator sets. This is because they appear tidy and organized to those who build them and to decision-makers and stakeholders. However, they often provide a false sense of the tidiness of the world in which one is operating. There there is no theoretical reason to believe that in the case of any particular indicator set that there will be a fixed under of inidators at similar levels under a set of indicator areas (e.g. what may be described as outcome areas).


The solution

As is stated in the principle above, when doing any indicator work, it is always best to develop the underlying visual outcomes (strategy) model for any initiative, organization, sector, community or country before, or at the same time as you are developinmg your indicator set. 


Example

Below shows a ’tidy’ indicator set with five outcome areas and five indicators under each of them.



The example below shows outcomes models for just two outcome areas - X and Y. It shows where the indicators map onto each of these outcomes models. The outcomes models illustrate the fact that the five indicators under Outcome X are much higher-level than the five indicators under Outcome Y. Just looking at the tidy outcome indicator set above does not provide any information on where the the indicators are located within the relevant outcomes model.