It will be unwise only to ‘fix’ in an ad hoc manner the weaknesses in the composition and application of the current set of DAC criteria. We need to consider carefully on what conceptual basis we select and define our criteria for the evaluation of interventions, or portfolios of interventions such as projects, programmes, policies, events, and so on (departing from a system rather than intervention requires a somewhat different approach).
Evaluation criteria can differ from organisation to organisation and from one stakeholder group to others, although the DAC criteria have inadvertently almost become close to becoming a ‘recipe’. In his thoughtful book, Evaluation Foundations Revisited, Thomas Schwandt points out that the differences in evaluation criteria can be based on (i) stakeholders’ perspectives of the importance of a given social problem; (ii) stakeholders’ vested interests in solutions to these problems; (iii) the norms and values of the organisational and political system in which the intervention is developed, located and administered; (iv) cultural understandings, and (v) evaluators’ own values and perspectives on criteria they believe are important.
I have an additional argument: For evaluation to address development effectively - and sustainable development even more so - there are certain imperatives that have to be met in terms of the evaluation criteria that we use. In other words, if such a set of ‘imperative’ criteria is not used to guide an evaluation, we might in actual fact not be evaluating – at least not with sufficient credibility - contributions to sustainable development or to positive development trajectories at national or regional level.
So, from the perspective that our evaluation criteria have to reflect the very nature of development as well as stakeholder interests, I propose that the identification and prioritisation of our evaluation criteria should be based on the following three principles:
One. Development as complex adaptive system (CAS). Development viewed as complex adaptive system demands that our criteria help us to reflect this when we evaluate in that context. This means that we have a subset of evaluation criteria that is imperative to include in such an evaluation – otherwise an important part of what makes for development might be neglected. The following criteria can arguably be considered for such a critical subset.
Two. Important organisational, societal and/or global norms and mandates. These are flexible and will change from time to time. At least some of the most important current global or societal norms or expectations will have to be included (for example, resulting from the 2030 Agenda, the Paris Agreement or Africa’s Agenda 2063), depending of course on stakeholder mandates and priorities.
Three. Additional stakeholder interests and concerns. These are flexible and may vary from evaluation to evaluation, and from context to context. None of these are imperatives that have to be addressed during an evaluation, but they are “good to haves” that will add value from an evaluation use perspective:
Does this argument and the three principles with examples of relevant evaluation criteria resonate with you?
In an upcoming post I will detail the rationale and proposed criteria in each case.
Zenda Ofir is an independent South African evaluator at present based near Geneva. She works primarily in Africa and Asia, and advises organisations around the world. She is a former AfrEA President, IOCE and IDEAS Vice-President, AEA Board member, Honorary Professor at Stellenbosch University, Richard von Weizsäcker Fellow, and at present Interim Council Chair of the new International Evaluation Academy.