Dr Paul Duignan on Outcomes:
High up on the central plateau
We had crossed the snow covered South Crater and we were standing beneath the flank of a steeply rising ridge covered in snow. It was a perfect day in the mountains of the New Zealand central plateau. We only needed to climb up the final ridge to Red Crater and we would easily complete our crossing. (For those who want to know, the ridge I am talking about lies between the two mountains in the photo above).
The four of us looked at each other and decided to continue up the ridge despite the snow. There were hot pools on the other side of the mountain near the hut where we were planning to stay. These have recently been destroyed by an eruption, but this was some time ago and at that time you could still access them. There's no doubt that our decision-making was influenced by the thought of sinking into the warm water of the pools and watching the sunset if we made it to the other side of the mountain.
We climbed up the flank of the ridge and then started making our way up it towards the Red Crater. We didn't have crampons as we'd not expected this much snow and at the bottom of the ridge the snow was soft. Our party included an inexperienced 14-year-old - the nephew of one of my hiking companions. At one stage I stopped and took a drink from my water bottle. When I was about to put it back into my pack, I dropped it. It slithered off down the steep snow slope. We all just stood and watched in silence.
The seriousness of the situation dawned on us. If the 14-year-old slipped - well if any of us slipped - would we go the way of the water bottle? Fortunately, we were already close to the top of the ridge and at that moment a climber appeared on the ridge-line carrying an ice ax. I scrambled up to him, borrowed his ax, returned back down and cut steps for the 14-year-old, who was now tiring. We all eventually made it to Red Crater in one piece.
It was sunset now and we’d made our destination hut. I leaned back in the hot pools, surrounded by snow. In this perfectly relaxed setting I started reflecting on our decision to climb the ridge without crampons and ice axes.
Were we right or wrong? It was certainly true that we'd been 'successful'. We had got over the track as we'd planned and we were now enjoying the fruits of our adventurousness here in the hot pools. If we'd turned back we would have had none of this.
However, were we right to go on as we had done without crampons and ice axes and with an inexperienced 14-year-old in our party? The answer is certainly no. It was a mistake, regardless of the fact that the outcome turned out to be positive. We'd taken too much of a risk and we could all have been sitting around a hospital bed at this moment, or worse. The thought of the hot pools had seduced us and we'd carried on regardless of the risk.
To the US Federal Reserve
It's a long way from a ridge on a mountain in New Zealand’s central plateau to a meeting of the US Federal Reserve trying to decide whether to raise interest rates or not. Recently the Fed made the call to raise them.
Economist Paul Krugman criticized the decision because he believes that current economic indicators don’t justify an interest rate hike. In his criticism of the Fed, entitled Fed Follies, he touched on this issue of whether success on its own is enough. He writes: '. . . it will be quite some time before we have evidence about whether the Fed's judgment of the economy's trajectory was right. (I think this was an ex ante mistake even if it turns out OK ex post . . .').
Even after cutting through the economic jargon of ex ante (before the fact) and ex post (after the fact), what Krugman's saying here seems somewhat paradoxical. Even if the Fed's judgment is 'right', it's still a 'mistake', and presumably 'wrong' in some sense for them to decide to raise interest rates at this point in time.
How can we make sense of Krugman's comments and translate them into a generalizable principle which speaks to a broad range of other decision-making situations? I'm interested in identifying such principles so as to guide strategic thinking and decision-making on any topic.
Outcomes theory and the When success is not enough principle
I’m an outcomes theorist and my job is, obviously, to study outcomes theory. Outcomes theory attempts to formally state the set of decision-making and other principles we should use when we're trying to take action in any area to achieve outcomes. These principles need to be general enough to be relevant whether you’re climbing a mountain or trying to manage an economy. The theory’s focus is on how we identify, prioritize, measure, intervene in, attribute and hold people to account for outcomes of any kind in any field.
The operative outcomes theory principle here is called: When decisions are not always vindicated by their outcomes, or more colloquially, the When success is not enough principle.
This principle clarifies the issue of whether a decision can be vindicated (i.e. 'made right') by whether its desired outcomes are achieved or not. There are really two definitions of 'right' in operation in such situations. The first is whether a decision was 'right' at the time it was made - whether it took into account all of the information and factors that should have been taken into account. Our decision did not take into account the risk we were exposing ourselves to and therefore it was 'wrong' at the time. Krugman's also arguing that the Fed's recent decision is similarly wrong regardless of whether it turns out to be 'right' in the end. Based on his reading of the current economic indictors, he thinks that interest rates should not be raised. Therefore he's arguing that the Fed’s decision to raise them at the current time is not consistent with this data and is therefore 'wrong' regardless of the ultimate outcome.
Outcomes theory’s When success is not enough principle states that in a number of situations (specifically risk management situations), the 'rightness' or 'wrongness' of a decision should, in the first instance, only be based on the information the decision-maker has at the time they make the decision. A decision which might have been regarded as 'wrong' on the basis of what was known at the time cannot suddenly change to 'right' on the basis of later information regarding whether desired outcomes were achieved.
The principle in action – never leave safety to luck
The Success is not enough principle is most appropriate in situations where the cost of a one-off failure is so high that you want to make sure that people take very serious steps to avoid it. You don't want to reward them just because they got lucky.
The principle lies behind effective risk management systems, for instance, in airline safety. Pilots are punished for deviating from routines and procedures that they're required to follow when flying. It doesn't matter if no crash results from their failure to follow the correct procedure. The point is that at the time when they were making their decisions they should have followed the standard safety procedures. Their decision to deviate from safe practice was still wrong regardless of the outcome and in the airline business they are disciplined accordingly. The When success is not enough principle highlights the fact that relying on luck and ‘things just working out’ is not enough in situations where major risks are being managed.
‘Seeing’ the principle in action by using a visual model
If we look at the principle in action in the form of a visual outcomes model (an approach used extensively in outcomes theory), this principle is all about the level at which accountability is set. I've illustrated this with a DoView outcomes model of our adventure in the mountains. The DoView model shows the steps involved in us having a safe hiking trip. I've traffic-lighted the DoView with how I think we performed on the various steps - red is hopeless and green is good. Note that I’ve traffic-lighted Appropriate gear for the conditions likely to be encountered green. We did have sufficient gear if we had kept our hike at an appropriate technical level. Our errors were all around the risk management boxes shown within the DoView model below.
The point is that our level of accountability was not at the extreme right box in the outcomes model - Planned trip completed. In this case, the appropriate level to set accountability is at the Safe hiking trip box where safety is operationally defined in terms of following the right risk management procedures. Because we failed at this level (the number of red traffic-lights leading up to this clearly shows our failure) we failed in our accountability, despite the green traffic-light on the Planned trip completed.
In business decision-making, a visual model like this is a very powerful way of clarifying the level at which parties should be held accountable. In practice, people sometimes print out a PDF of the relevant DoView model showing the level of accountability and attach it to their contracts with providers. This ensures that everyone is clear about the level at which accountability kicks in and it avoids any confusion on the part of any of the parties involved. Without a visual model there often can be a lack of clarity about the level at which accountability is being set.
If you’re interested in downloading this model to play around with it, or to build your own model, get a DoView free trial here and get the DoView file of the model here. You can find the rules for drawing this type of outcomes model here.
It should be noted that the When success is not enough principle is not the only principle that is concerned with the relationship between acting and the final outcomes that an action is seeking to achieve. The final outcomes of actions can be relevant in the wider context of working out what level of risk management is required over a number of similar decision-making situations.
In addition, other principles come into play in situations where people are willing to tolerate some failures in decision-making within an overall larger system. For instance in evolutionary, experimental, or entrepreneurial systems, which consist of many attempts to solve problems. These attempts are likely to include some failures as well as some successes - that is how such systems are set up to operate. In such situations, final outcomes and 'success' are more relevant to incentivization than in the type of risk management cases where the When success is not enough principle is more relevant. I'll be discussing these other outcomes theory principles at a later stage.
As with all outcomes theory principles, the When success is not enough principle's purpose is to help us clarity the conceptual basis of how we think about, and work with outcomes. The more we can identify and communicate general principles across different topics, the more sophisticated we will be at setting up the strategy and accountability systems that we rely upon throughout society.
If you want to comment on this article you can do so on the Linkedin version of the article.
Back to the DoView Blog.