The Snyder evaluation process: an
overview
This is a resource file which supports the regular public program "areol" (action research and evaluation on line) offered twice a year beginning in mid-February and mid-July. For details email Bob Dick bdick@scu.edu.au or bd@uq.net.au
... in which a very brief description of the three phases of the Snyder evaluation process is given
Contents
- Introduction
- The underlying systems model
- Preliminaries
- Process evaluation
- Outcome evaluation
- Short cycle evaluation
- Notes
This document briefly describes the Snyder evaluation process. It is a flexible process which can be used to evaluate an activity (perhaps a program or project) or a unit (for example an organisation or team.
The Snyder process usually:
- is participative, seeking to involve all of the stakeholders 1 or their representatives in the process; it is usually possible to involve the stakeholders or their representatives as actual co-evaluators in the evaluation;
- is conducted using the methods of action research, for the combination of rigour and flexibility which that allows; this also provides both action outcomes in the form of improvement, and research outcomes in the form of increased understanding;
- uses a combination of qualitative and quantitative data; and
- encourages critical reflection by all stakeholders within a systematic but flexible process.
The Snyder process uses action research methods to develop an understanding of how the activity or unit operates. It is at its most effective when used in a highly participative manner, and is especially suited to such use. High participation also increases the likelihood that the results of the evaluation will be applied to program improvement.
The following description assumes a participative version will be used; however, an independent evaluator may use the same overall process to conduct a non-participative evaluation if this is desired.
The detailed description of the process may seem complex. In practice it is logical and flexible, and relatively easy to use in moderately experienced hands. As you read the detailed description you will find it useful to remember that the process has three main components. Each of them provides a different form of evaluation. Each of them builds on the previous stages:
- Process evaluation. 2 This stage enables the evaluator and stakeholders to develop a better understanding of the functioning of the unit. In particular, it allows the stakeholders to understand the links between resource use, unit activities, the intended and unintended immediate effects of those activity, the predetermined goals which are pursued, and the contribution of the unit to some overall and long-term vision.
- Outcome evaluation. 3 This stage enables the evaluator and stakeholders to apply the understanding which they develop in the process evaluation. They are able to assess which of their goals are achieved, and how well this is done. It also allows the development of performance indicators which can be used to set up ongoing feedback and monitoring.
- Short cycle evaluation. This stage uses the understanding from phase 1, and the performance indicators from phase 2, to be applied. The result is a self-improving system which provides regular feedback to the unit about its performance.
The underlying systems model
Each of the three phases also draws upon a systems model of how a unit or project operates. Resources are consumed by activities which produce intended and unintended immediate effects in the pursuit of goals and objectives, which in turn are intended to contribute to some vision of a better world. Briefly:
resources -> activities -> effects -> objectives -> visionResources consist of anything which is required if the activities are to be conducted. They include the obvious: money and equipment and other physical materials. They also include the often-overlooked: skills, time, goodwill, and the like.
Activities are the moment-by-moment and day-by-day actions carried out by unit members.
Immediate effects are those outcomes which occur as a result of the activities, usually during or immediately after those activities. They include those immediate outcomes which are intended, and those which are not.
Objectives are the goals, usually pre-set, which unit members pursue over intermediate time frames. They are usually developed through a planning process, explicitly or implicitly. They are usually set or revised each planning cycle. Thus their timeframe is typically a year, though sometimes more or less than this. (In some descriptions the objectives are labelled "targets".)
The vision consists of the ideals held by the unit stakeholders: the views of a better world in general, and of the contribution of the unit to that better world.
A somewhat more detailed description of the process now follows. (A step-by-step description is available as a separate document.)
Preliminaries
Before the actual evaluation begins there are some crucial preparatory tasks to be done. The primary objective of this phase is to create open and effective relationships between all parties, and a clear understanding of their collective and individual roles.
The most important early tasks are:
- Building an effective and flexible working relationship between the evaluator and key stakeholders.
- Identifying and involving other stakeholders in the process, and negotiating their role and that of the evaluator and key stakeholders.
- If working with a representative group of stakeholders, setting up the means by which they can stay in touch through two-way communication with the stakeholders they are drawn from.
Process evaluation
The purpose of the process evaluation is to develop a better understanding of how the unit operates. This is done by identifying the components corresponding to the elements (resources, activities, and so on) of the systems model, and analysing how well they are linked together.
The main feature of this phase, therefore, is a focus on the links between the elements -- the way resources, activities, immediate effects, objectives and vision are interconnected, and how the early elements contribute to the later elements.
So, for example, there earliest steps proceed as follows:
- identify the vision;
- identify the objectives;
- analyse which objectives contribute most to the vision;
- develop plans to adjust objectives or vision (or both) in the light of this analysis.
In practice, stakeholders often begin to reorder their priorities and change their behaviour almost from the first step of such a process. This tends to occur whether or not formal plans are developed.
The Snyder process encourages and enables improvement almost from the outset. As people come to understand the process of their work better, they change their behaviour to reflect that improved understanding.
By the completion of the process evaluation, stakeholders understand the links between each element and the adjoining elements:
resources -> (activities + effects) -> objective -> vision
Outcome evaluation
The outcome evaluation uses the understanding from the process evaluation to identify performance indicators. These can be used to provide an outcome evaluation, and to set up the feedback systems for ongoing monitoring and improvement.
In practice this is done by taking the important elements of the vision in turn. Measurable indicators of each are then sought in the earlier elements of the system model:
vision -> objectives -> effects -> activities -> resourcesIndicators typically contain a mix of qualitative and quantitative measures. Effective indicators are most often to be found within the "resources" and "effects" elements. Desirably, the package of indicators can be viewed as an appropriate sample of the important elements of the vision, and to be drawn primarily from resources, intended immediate effects, and unintended immediate effects.
Short-cycle evaluation
The short-cycle evaluation then seeks to create systems to allow feedback of results and monitoring of effectiveness. This phase contains the following important steps:
- check that the indicators are an adequate sample of vision, resource use, and intended and unintended immediate effects;
- identify the sources of feedback information for each of the indicators;
- set up a process or mechanism to generate that feedback and communicate it to the people most able to make use of it for system improvement.
- Simply put, a "stakeholder" is anyone who has a "stake" in what they unit does, or how it does it, or who can affect the operation of the unit. [ back ]
- The evaluation literature mostly uses the terms formative evaluation (simply: evaluation of a project or program in progress) and "summative evaluation" (simply: after-the-event evaluation). These are intended, I think, to describe the function rather than the process used. I've chosen terms which are more descriptive of the process used. Having said this, "process evaluation" as used here is almost equivalent to "formative evaluation"... [ back ]
- ... and "outcome evaluation" as used here is almost equivalent to "summative evaluation" [ back ].
_____
Copyright (c) Bob Dick 1997-2000. This document may be copied if it is not included in documents sold at a profit, and this and the following notice are included.
This document can be cited as follows:
Dick, B. (1997) The Snyder evaluation process: an overview [On line].
Available at http://www.uq.net.au/action_research/arp/snyder-b.html
Maintained by Bob
Dick;
this version 2.03w last revised 20000105
A text version is available at URL ftp://ftp.scu.edu.au/www/arr/snyder-b.txt