The review team then attempts to capture 'what the program is trying to do' in some form; analyze if the program seems to be doing it; and then it makes recommendations for how the initiative can be improved.
The
DoView Program/Organizational Review Process provides a more efficient way of doing a review.
In many situations, an external agency, or an independent review team in a larger organizations, has to review or monitor another program, initiative or organization.
This is usually done by the review team going through a large number of documents and having a number of discussions with the initiative's staff and stakeholders.
It does this by the review team building a DoView visual strategy model setting out 'what the program is trying to do'.
This visual strategy model is then used as the framework for answering a number of the key questions any review team will be wanting to ask. These questions are set out below.
1. Does the initiative know what it is trying to do?
The Solution
Below is an example of a DoView visual strategy model for a mental health service. Click on the boxes with gray triangles in them to navigate through the model. You will be able to see the detail that is provided on drill-down pages.
More on how to draw a program/initiative/organizational DoView.
The Problem
The review team needs a methodology to quickly work out whether the initiative being reviewed has a clear concept of its outcomes and how it is going to achieve these.
The Benefits
The visual strategy model can then be critiqued by the review/monitoring team itself; it can be checked against previous research and evaluation findings; and/or sent to external peer reviewers for critique.
It can also be used in discussions with stakeholders to see if they think that the initiative is clear in its thinking about what it is trying to do.
2. What is the initiative’s priorities and is its activity focused on them?
The Solution
First, the visual strategy model is marked up with the priorities the initiative is currently focusing on.
Second, the actual pieces of work the initiative itself is undertaking are identified (at the outputs-type level) and these are then visually mapped onto the boxes within the strategy model. This can be done by the initiative itself and reviewed by the review team or done jointly with the review team.
The extent to which there is line-of-sight alignment can be easily identified by looking at the number of pieces of work focused on each box within the strategy model. This will reveal if more work is focused on priority boxes and quickly provide a clear view of whether the initiative has achieved strategic alignment.
See here for how DoView Line-of-Sight alignment works
The Problem
Using the traditional approach of studying documentation and just talking to the initiative's management and staff it can be hard for the review team to quickly work out whether or not the initiative is actually fully focused on its priorities.
The Benefits
The strategy model marked-up with priorities can be used as the basis for discussions with stakeholders. This is to check whether stakeholder's views of what the initiative's priorities should be match up with what the initiative thinks its current priorities are.
The second stage, where the initiative's current work (at the output or project-type level) is mapped onto the boxes in the visual strategy model, is by far the most powerful and transparent way of quickly analyzing whether an initiative's work is actually focused on its priorities.
The DoView strategy model drill-down page below shows how a page can be marked-up with priorities.
3. What is the initiative currently measuring and how is it tracking?
The Solution
The review team can then very quickly assess whether the indicators are measuring what is in the box they have been put next to and, secondly, whether the indicators that are being measured are measuring important boxes or not.
Examples of indicators being mapped onto a DoView model can be seen here.
The Problem
However, it is often hard for the review team to quickly work out whether this set of indicators are actually measuring the 'important' rather than just the 'easy to measure'.
The Benefits
The traditional approach to doing this work in reviews is to attempt to do this by looking at indicators within table formats of one type of another - this is much less efficient than using a visual strategy modeling approach.
4. What is the initiative doing to prove its impact?
The Solution
More on putting evaluation questions onto a DoView
More on Duignan's Impact Evaluation Feasibility Check
The Problem
Impact evaluation is a complex technical area which is often hard for a review team to rapidly assess.
The Benefits
It makes it crystal clear as to which impact evaluation questions have, and have not, been answered regarding the initiative. It immediately clarifies the level at which impact has been established. Sometimes an initiative will claim that it has established 'impact' but on further inspection it becomes clear that it has only established impact at a relatively low level.
Reviewers should not make unrealistic demands as to the level at which impact is established, but it is essential that they are clear as to where that level lies for the initiative they are reviewing.
5. What is the best way to quickly summarize the strengths and weaknesses of the initiative that is being reviewed?
The Solution
A review/monitoring team can 'traffic-light' the initiative's DoView strategy model. This shows the boxes where there are problems and the boxes which the review team thinks are going well. This 'marked-up' DoView can then be used by the review/monitoring team as a summary graphic in their report.
A related technique - a Performance Improvement DoView is used in the same way to show both areas of strength and weakness and how these are being addressed by an initiative going forward.
More on how to use a Performance Improvement DoView
The Problem
The Benefits
If the review report includes a strategy model marked-up with traffic lights, the report reader can get an overview of the findings of the review within a minute or two. They can then look at the review text to get further information.
The model below shows how a DoView strategy model can be used to summarize the results of a review. You can see that for this initiative there were problems with: risk management; service structure matching its function; and monitoring and evaluation. In addition there could be improvements in: financial management; client information systems; their brand; client satisfaction; relationships with families; funders and their financial viability - but these were not major problems. Lastly, the initiative was doing well in regard to all of the boxes with green traffic-lights.
