What approaches do we use for evaluations
We use different approaches for evaluations, adjusted to the situation and the evaluation question. Here you can find some practical examples
Validation of an evaluation: a learning experience
During the first quarter of 2009, in collaboration with ETC-Crystal, MDF has executed a programme evaluation for Woord en Daad, a Dutch co-financing organisation. The programmes evaluated were Basic Needs and Job and Income. Woord en Daad's Education programme had earlier been evaluated by COMMON. A first particularity of these evaluations was that they were done on the basis of documents about projects executed by partner organisations (54) in three continents, complemented by telephonic interviews with partners. A second particularity was that the three programme evaluations were complemented by a socalled Meta-analysis, covering all three evaluations. A third particularity was that the primary aim of the evaluations was 'learning'.
The first contact between evaluators and partner organisations was during a Woord en Daad partner conference in Nairobi, where the conclusions of the draft-evaluation-reports were presented by the evaluators, first in plenary, then in specialised workshops (5) in the presence of Woord en Daad staff and its partner organisations. In these workshops the first round of leading questions was: 1. Are the conclusions and recommendations clear?, 2. Do they need amendments to be clearer? Could you explain them to the group? The next round of leading questions was: 1. Are the conclusions valid in your specific situation or organisation? If not, please explain in narrative form why not. The answers were scored (yes-no-not applicable). The third round of discussions in the specialised workshops covered the question: which conclusions/recommendations should be an issue for the whole organisation to discuss? These questions were later discussed with all members of each partner organisation and their Director. Final assessments by all specialised workshops were presented in plenary the next day. It appeared that, with minor modifications (except for the Job and Income programme) the outcomes of the evaluations were validated as relevant, recognizable, often applicable, and in any case useful to be exploited and further discussed during the further conference debates. Out of sight of the evaluators the lessons learnt from the evaluations were applied in new programme proposals for the coming period (2010-2015).
We (MDF evaluators) experienced this "validation" process as a pure learning experience, whereby the participants (Woord en Daad staff and its partners) passed jointly through a learning cycle, ending with the application of single, double and triple stroke learning in their future programmes and organisation. Woord and Daad is a good example of a learning organisation. Their triple stroke learning concerns the transformation of a typical Dutch co-financing organisation into a future network organisation linked together by global, regional and thematic alliances. Wait and see!
Sjoerd Zanen, senior consultant
