We need “Evaluation with African Characteristics” - in how we conceptualise evaluation and in how we lead and shape it in the interest of development on the continent.
Instead of thinking about what this means, we tend to slavishly teach and use evaluation theory and practices imported from outside the continent. Our evaluation terms of reference specify the use of (DAC) criteria without sufficient tailoring for what matters to us. The African Evaluation Guidelines reflect some of our context, but we seldom use them. Our evaluation priorities are still driven primarily by international programming rather than by what best serves the interests of our continent. Our universities and short courses prepare emerging young evaluators by espousing almost exclusively imported knowledge, at best with a few tailored examples from Africa.
Instead, we should lead and shape evaluation on the continent by being deeply thoughtful and innovative about what it actually should be, do and achieve for Africa.
Of course, we should master and apply that which comes from outside the continent. This is obvious. But Africa-centric evaluation should also be ‘Africa-led’ – led by Africans for Africa’s purposes: It should not only be rooted in African philosophies, values, knowledge systems and experiences. It should also be fully informed of issues and trends that affect our continent, whether from international (see also here and here) or local analyses (see also here) - and focus on evaluation priorities and ways of doing that can help us to deal better with what is critical for the development of our societies and ecosystems.
Among others it is not enough just to focus on (often aid-funded) programmes and projects, and occasionally on policies; we increasingly need to engage with the larger systems, contexts, relationships and strategies within which the continent develops and within which we conduct our work. We need to understand, not just judge, what is taking place, what influences success and failure as we work towards development.
Most importantly, we should design monitoring, evaluation and learning systems, and commission and lead evaluations, by taking on what comes from outside - yet also creating and tailoring deliberately for our purposes, with a deep understanding of what characterizes our societies, and of what is different about our continent, its development priorities, its philosophies and values.
We have only just started attending to how evaluation in Africa can and should be led by and for Africa. Fred Carden and Marvin Alkin highlighted in 2012 the rather meagre collection of practices that have emanated from the Global South. The African Peer Review Mechanism is a notable (although not always very successful) example; so is part of the CLEAR-AA DETPA winter school and Twende Mbele, which is leading the way with significant potential for the future. The African Evaluation Guidelines developed in 2002 were the first to take the lead in contextualizing such guidance. Today the African Evaluation Journal highlights what is being done. Progress is being made. But we still need to cultivate a notion of leadership in this space, with breakthroughs and deep engagement by our own - similar to the contributions to evaluation theory and practice on the global stage by pioneers such as Michael Scriven, Michael Patton, Ernie House, Thomas Schwandt, Ray Pawson, Donna Mertens, Karin Kirkhart, Kate McKegg, Fiona Cram, Nan Wehipeihana and many others.
There is a lot to do. In recent years South African evaluation and governance scholar Fanie Cloete wrote eloquently about what can and should be done – see for example here and here. It is well worthwhile to consider his suggestions. The 2012 Bellagio Report highlighted significant implications and recommendations for evaluation from this “let’s lead” perspective. It sketched the context for African evaluation for development in the coming years and addressed a host of things to do to ensure leadership in evaluation on the continent. Among others, the following stand out:
One, advance capacities and standards more innovatively - not only for useful and high quality evaluations, but for thought leadership in evaluation that can cross boundaries and contribute to theory and practice in the field globally.
Two, synthesise our evaluation knowledge and contributions, and make them more visible.
Three, engage with African value systems, and with considerations of what is of value for Africa.
Four, work on what matters at macro and not only micro level - and with a lens on changing contexts and complex systems (linear, simplistic notions of RCTs were still very popular at the time).
Five, move from “impact” to “managing for impact” through experimenting, learning and in-time adjustments while ensuring a very strong focus on identifying and understanding negative impacts and tradeoffs.
Six, cultivate evidence-informed evaluative thinking in the broader society, as well as a demand for evaluation in a way that gives it more power.
Seven, let universities and think-tanks develop into a network of – ideally independent - centres of expertise in evaluation.
Eight, ensure dynamic and continuous dialogues within a liberal thinking space in order to inform policies and support sustainable, effective systems in government, including for monitoring, evaluation and learning.
The African Evaluation Association has recently teamed up with four other regional evaluation associations from the Global South to initiate the South-South Collaboration in Evaluation (S2SE) initiative to give life to these ideas in all the regions of the world that are relative newcomers on the global stage of evaluation. I will highlight this initiative as part of a new series of posts I will write on evaluation for transformation.
Zenda Ofir is an independent South African evaluator at present based near Geneva. She works primarily in Africa and Asia, and advises organisations around the world. She is a former AfrEA President, IOCE and IDEAS Vice-President, AEA Board member, Honorary Professor at Stellenbosch University, Richard von Weizsäcker Fellow, and at present Interim Council Chair of the new International Evaluation Academy.
This is extremely exciting and the direction African Evaluators should be going. Evaluation should be utility and learning focused. My concerns are that we are not in charge and in control especially as independent evaluation consultants, even as trainers were are not independent as the demand agenda is created by other interest and not necessarily for the real benefit of Africa.
1. As a consultant how does one get out of the slavery of doing evaluation as prescribed by the client in the TORs and conditions? The client is interested in seeing a beautiful picture of their organization or programme in the mirrow. The client wants the donor to be happy and so they can get more funding and not necessarily lessons from evaluation which may lead to him/her avoiding re-inventing failure.
2. When, where and how do we train evaluators this new approach and these values? Institutions that train evaluators merely respond to the market demands that want hear good news and not the truth or indeed the much needed lessons.
3. How do we make sure that those we train in these new values and philosophy and approaches get evaluation jobs in a market that is differently oriented and externally controlled? recently, I trained a young evaluator in some of these approaches when she went back to work the employer was not interested but infuriated by this approach and she lost her job. In other words what other things need to go with these new approaches if we are to succeed?
I find your point Seven of suggestions as being extremely useful in trying to deal with my point 3 here. But we need to explore that further and see which these institutions are and find a way of creating a conducive environment.
Great ideas I am encouraged.
Backson, you correctly highlight the need for real change across the evaluation system if we are going to have maximum benefit from evaluation. It will have to be a concerted effort. Not easy to achieve at all, but something to strive towards. We need commissioners of evaluation as well as political masters to acknowledge the importance of these concepts. As usual, some smart ones will, but they will be a minority – for a while still. We would like to establish a platform of funders and commissioners interested in experimenting with, and promoting these approaches. In places like New Zealand new approaches are already being applied; it will spread.
I have found all three inputs on Made in Africa Evaluation very interesting reading. However, like Backson, I have experienced strong resistance from many funders and NGO managers when one suggests the importance of working with African family and community values in mind. The field workers are generally very positive, but their managers just want what is required by funders. A very gifted field worker that I worked with on an evaluation summed up the situation by pointing out that technically the outcomes for the project were achieved but the changed behaviour is not embedded in the community. On the other hand, I have noted that programmes that coincidently resonate with African family and community values seem to be very successful and have greater reach and depth in terms of their impact. For me, we need to consider carefully what sort of change we want our project/program to bring about and what we believe our starting point should be. Miriam Were, an esteemed Kenyan proponent of Primary Health Care put it this way in her acceptance speech for the 2005 Gates Award:
“We are convinced that if the development agenda had recognised the existing strengths in the African people and had built on them, instead of treating African people as if all they have is ignorance to be gotten rid of and presenting them solutions with no bridges to their reality”
This remains a critical challenge. I think “Made in Africa” evaluation will need to be linked to “Made in Africa” approaches to development planning particularly in remote rural areas and communities that embrace traditional lifestyles.
Exactly Liz. Your last paragraph highlights the crux of the matter very well. That is why it is crucial that we evaluate for these approaches. To prove their value we need a cohort of funders and commissioners – our own and those working in the international development environment – who are willing to collaborate and innovate so that evaluative activities can play a more constructive role. There are examples. But most of all, our own governments who work with evaluation should promote this among our own societies. This is what is not done; the well educated are often too ingrained with convention to move out of comfort zones – and tend to stick with the worldviews and models that they were educated in, rather than nurture and evolve their own.