Impact evaluation only option for high-level outcome attribution if no controllable indicators at top of outcomes model principle

principlesbutton_med

Academic references to outcomes theory

screenshot_1915_med

The principle

Impact evaluation (building-block five from the Outcomes System Diagram) is the only option for high-level outcome attribution if no controllable indicators (building-block two) reach to the top of the outcomes model (building-block one). 

The problem

Being able to attribute changes in high-level outcomes to a program or organization is very useful. However it cannot be assumed that this will be possible in all cases. It is often believed by those using performance management systems that such attribution will be possible merely by measuring indicators (building-blocks two and three in the Outcomes System Diagram). It is the case that merely measuring controllable indicator does establish attribution of changes in those indicators to the program or organization in question. However if controllable indicators do not reach to the top of the relevant outcomes model, then they cannot be used to attribute changes in high-level outcomes to the program or organization. While there may be measures of high-level indicators in the outcomes model (building-block three). Just measuring high-level not-necessarily attributable indicators does not prove that the program or organization has changed them.

The solution

The solution is to always consider the use of impact evaluation (building-block five in the Outcomes System Diagram) in cases where controllable indicators do not reach to the top of the relevant outcomes model. However in doing this one should not assume that impact evaluation will be able to be done in all, or even many, cases. It may be not appropriate, feasible, affordable or credible. More information on the seven possible impact evaluation designs here.