**4 min read**
Apollo is one of Africa’s most knowledgeable evaluation specialists, and also one of its nicest. I can attest to this, as I worked with him for several years while I was advisor (and at times de facto internal evaluator) to AWARD, a very interesting BMGF funded programme based in Nairobi that supports women scientists in (now) 28 African countries. Apollo is as far as I know the only African to have received the AEA's Marcia Guttentag Promising New Evaluator Award (in 2017) and is very likely the only African evaluator with two exceptionally useful evaluation books to his credit, namely Working with Assumptions in International Development Program Evaluation, and Credibility, Validity, and Assumptions in Program Evaluation Methodology. (It is a great shame that the publishers decided to charge astronomical prices for such important contributions to evaluation practice; an author has no control over this.) Apollo is from Uganda but has since moved back to the US where he worked before joining AWARD. I am keeping my fingers crossed that he will return - or in any case continue to advance evaluation practice for the sake of Africa and the evaluation world in general. Here he provides three great tips for all of us.
Top Tip 1. Be very aware of the ‘assumptions’ underlying what we do.
Assumptions are what we believe to be true. They may be explicit or implicit. Unexamined or tacit assumptions can be a risk to program success or evaluation quality because they are not explicitly voiced and not necessarily understood by others.
We can’t avoid making assumptions because we can’t avoid simplification. Reality – as seen in programs and their contexts - is complex. In order to do something – anything - about reality, such as measure success or implement a strategy, we inevitably have to simplify. So, we use a framework or model, such as a theory of change, to prioritize the issues that matter.
In focusing on what is in the model, we may easily ignore what is not explicit. Those are the implicit assumptions. As Jonathan Morell appropriately stated recently, “All models are wrong, but some are useful”. What is outside the model needs to be explicated and examined. Otherwise, it can be detrimental to programs and evaluations.
Top Tip 2. Practice Assumptions Aware Evaluation.
Agency is one’s ability to act on what one values. Strengthening Assumptions Aware Evaluation practice requires agency at three levels: (i) agency for ourselves, (ii) agency for others, and (iii) agency for institutions and policy.
One, agency for ourselves as evaluators. We need to recognize to appreciate our vulnerability and examine our own assumptions. Psychologists talk of the power and limitations of the conscious mind -that we human beings not only claim responsibility but also intention for actions over which we had no control. We are prone to implicit bias and assumptions, and the impact of unconscious biases on behavior is given much less credit than it deserves. If we acknowledge that our thoughts and feelings may operate outside of the purview of conscious awareness, control and intentions, we remember to examine our own assumptions.
Two, agency for others. Evaluators can draw on a wide range of program design tools to facilitate the unearthing and critique of assumptions of stakeholders and evaluators. For more on tools that could be used to examine different kinds of program and evaluation assumptions, please take a look at this article.
Three, agency for institutions and policies. Whenever it is within evaluators’ means, we need to encourage institutions to promote a culture of reflection and assumptions analysis. Integrating assumptions analysis in policies is a key step in the right direction. USAID’s evaluation policy (2013) promotes the articulation of assumptions in program theories. Another example is the American Evaluation Association’s new guiding principles for Evaluators, where it states (in A5) that evaluators should “discuss in contextually appropriate ways the values, assumptions, theories, methods, results, and analyses that significantly affect the evaluator's interpretations of the findings”.
Top Tip 3. Commissioners have to lead on assumptions.
Commissioners of evaluations hold a lot of power. If they are more normative in outlining assumptions that need to be examined along with evaluation questions, the practice in this regard will improve very significantly.
These are all great entry points for Assumptions Aware Evaluation.
Apollo M Nkwake is a M&E Technical Advisor at the Education Development Center in Washington DC. He previously served as associate research professor for M&E at The George Washington University and at Tulane University. He holds a PhD from the University of Cape Town and is a designated Credentialed Evaluator. Dr. Nkwake is a recipient of American Evaluation Association’s 2017 Marcia Guttentag Promising New Evaluator Award. He is the author of: Credibility, Validity, and Assumptions in Program Evaluation Methodology (2015, Springer), and Working with Assumptions in International Development Program Evaluation (2013, Springer).