**4 min read**
“We won’t get back to normal because normal was the problem.” - Graffiti in Santiago, Chile
It is great to see evaluators leaping into action with advice on how to adjust practice to cope with the impact of the coronavirus. This could be a transformative moment also for evaluation. As deftly stated by IEG’s Joss Vaessen and Estelle Raimondo, “Evaluators should not be tone-deaf and rely on well-oiled institutional mechanisms that regulate the production of evaluations to preserve ‘business as usual’”.
The coming months and years will be more uncertain, volatile and turbulent. This is not a health crisis. It is a series of intertwined health, social, (in some countries) political, and especially financial and economic shocks that is very likely to lead to a global depression possibly worse than 1929. The preferred immediate remedy of printing trillions of dollars and issuing long-term bonds will not sustain; governments will try to claw back losses from already-suffering citizens. Societies will be under strain as poverty extends its grip on economically rich and poor countries. Tensions between people and between nations are set to increase. Many leaders around the world are weak and, together with the manipulations of media monopolies, will be tempted to launch self-centred efforts aimed at causing havoc and conflict as old powers wane and new ones become more assertive about their place in the sun.
All systems within which evaluators work will be under pressure, and we will need to be very sensitive to stresses that result in new patterns of institutional and societal behaviour. But the new situations will also bring new opportunities where evaluation can support the healing of societies and ecosystems, and help sustain a good balance between the interests of nature, humanity, and the social-ecological ecosystems on which we all depend.
A useful list of resources with advice from evaluation specialists for the time of the coronavirus can be found on this blog of Tom Archibald, "free-range evaluation" (a really good name for a blog, btw).
I used for the summary below the insights provided by Michael Quinn Patton, Rockefeller Foundation (i.a. by Michael Bamberger), World Bank IEG, UNDP IEO, and comments from webinars held by the UNEG, OECD Evalnet and Blue Marble Evaluation.
I made my own additions in blue, including the three categories which are my personal priorities and on which I will focus in upcoming posts.
Please click on the image to view the larger version.
I found the advice provided by Michael Quinn Patton particularly thoughtful as it focuses to a more detailed extent on the implications for different dimensions of evaluation in the period after COVID-19: Emphases such as the importance of thinking and working both systemically and systematically. Engaging with complexity and practicing interconnections. Connecting the local to the global in terms of patterns and actions – promoting for example understanding of the links between health and climate. Working for global, longer term sustainability transformation. Recognising that the South and North are intertwined but that societies are very different. Making the case for evaluation’s value. Being informed knowledge workers and evaluation scientists. Using what the virus is doing now to strengthen evaluation thinking capacities.
We need to think more deeply about how we should position evaluation as practice at a critical time like this. It is invisible amidst the slew of verbal and written reports and papers discussing ‘evidence’ and analyses of what is taking place. This displays the dire need for us to connect better with research, and to move our practice with a sense of urgency beyond "projects" and "programmes" to (global) "systems".
I close with two of Michael's most pertinent quotes:
“Make evaluation all the more useful and real-time data essential so that the evaluation value proposition reframes evaluation as an essential activity, not as a mundane bureaucratic or luxurious function when times are good. Define, conceptualise, articulate and demonstrate the essential utility of evaluation. Lay the groundwork now.”
And:
“Balancing long-term threats with urgent demands for short-term, crisis-generated interventions demands in-depth transformative evaluation thinking. Evaluators need to be prepared to contribute to finding and following pathways and trajectories toward transformations for a more sustainable future. Keep learning. Support each other. Commitment to evidence-based decision-making, evaluative thinking and using evaluation to make a better world. That’s our niche.
Zenda Ofir is an independent South African evaluator at present based near Geneva. She works primarily in Africa and Asia, and advises organisations around the world. She is a former AfrEA President, IOCE and IDEAS Vice-President, AEA Board member, Honorary Professor at Stellenbosch University, Richard von Weizsäcker Fellow, and at present Interim Council Chair of the new International Evaluation Academy.
Nice post Zenda. One thing I would add which cuts across your three added fields I think, is that we need to get better at promoting the use of locally based consulting firms and individuals, not as subs and interviewers but as leads. They are on-site; they need to be in the driver’s seat – that has always been something to push for but now more than ever the opportunity is there to push commissioners to get to know and work with local evaluators.
Fred, I fully agree with you that this is a good time to push for this important change in practice. As caveat, there is some discussion about whether this might be the way to go for a less desirable/honorable reason – expecting local evaluators to face the additional risks that come with COVID-19 in the foreseeable future. But the positive side of this might well outweigh the negative in the long term – especially if countries in the South get the virus more quickly under control than expected.
Thank you, Zenda, for another timely and relevant post, and that is a useful image breaking down key consideration for evaluation in the new-norm. Caveat: I hesitate to call it a ‘new norm’ because that can imply a degree of stability or regularity, whereas I think the ‘norm’ will be even more of a moving target in the decades to come.
With crisis comes opportunity, and I do hope the evaluation community can jump on this rather than BAU. I am reminded of the 2018 NDE (Nielsen et. al.) on the importance to recognize the influence of the evaluation market place and how the very political economy we operate in can shape (handicap) the potential of evaluations to transform and meaningfully help address the complex challenges we face today:
“If evaluation is intrinsically embedded within these market dynamics of commissioned – and in effect commercial – knowledge production, then examining the extent to which and how these dynamics influence evaluation practice becomes topical. By not awarding attention to the ways in which the scope, design, methodology, and deliverables of evaluations are shaped by the push and pull of market actors, commissioners as well as providers, we fail to appreciate the fundamental conditions for the development of evaluation theory and practice.”
Covid-19 is a wake-up call to a systemic imbalance that cuts across human systems and how we relate to each other and the larger ecosystem we are part of. However, just as entrenched political interests capitalized on the fear of the 911 wake-up call that greeted the start of this millennium (who are the real weapons of mass destruction?), I fear it will likewise capitalize on the current pandemic, (i.e. see link to Guardian article below), and that any progress made towards address the super wicked problems, like climate change, may be derailed (see link to Times commentary below).
Given the magnitude of problems like pandemics, climate change, and escalating social injustice, we could do more as an evaluation community. This is not a time to be a bystander. As professor Jem Bendall expressed in his influential paper, Deep Adaptation: A Map for Navigating Climate Tragedy (see link to full paper below):
“We no longer have time for the career games of aiming to publish in top-ranked journals to impress our line managers or improve our CV for if we enter the job market. Nor do we have a need for the narrow specialisms that are required to publish in narrow specialisms that are required to publish in such journals.”
I appreciate having forums such as this to help shed light on these important issues for the evaluation community.
—————–
https://www.theguardian.com/commentisfree/2020/apr/13/with-working-americans-survival-at-stake-the-us-is-bailing-out-the-richest?utm_term=RWRpdG9yaWFsX0Jlc3RPZkd1YXJkaWFuT3BpbmlvblVTLTIwMDQxMw%3D%3D&utm_source=esp&utm_medium=Email&CMP=opinionus_email&utm_campaign=BestOfGuardianOpinionUS
https://time.com/5795150/coronavirus-climate-change/
https://mahb.stanford.edu/wp-content/uploads/2018/08/deepadaptation.pdf
Thanks Scott for your confirmation of the urgency with which evaluation has to re-position itself at this time, as I also noted in the first two posts in this series. Unfortunately both the bail-out of large companies and higher unemployment levels are likely to work against the transformational change that humanity and the planet need at this time. But there are also other forces at work, with many organisations working together to prevent business as usual and provide alternatives. I hope it inspires all of us to play our part, each in our sphere of influence.
An excellent overview — as always… but have policy makers in any country reached out to evaluation practitioners to manage the crisis or plan for its aftermath? If not, what does this state of affairs mean for the future of our occupation? In other words, when will the evaluation community face up to the need for more and smarter evaluation advocacy and more systematic evaluation professionalization?
Bob, this is the exact point I made in a webinar yesterday. It highlights two sides of the same coin: (i) the role of research vs evaluation – with researchers quoted by everyone and evaluators quoted nowhere, and (ii) the role of evaluation as complement to research, where the two could be working hand in hand. Both independent evaluation and self-evaluation (using e.g. developmental evaluation or blue marble evaluation principles) can be very useful for COVID-19 work. So we have the instruments but as you indicate, it is our positioning of evaluation, the image it has in some instances, and how we formulate its value proposition for situations like these that are woefully inadequate.