Guest Post: The resilience of evaluation in turbulent times

Share this post

**8 min read**

Apart from being a leader in evaluation in different capacities, Weronika is one of the most thoughtful evaluation specialists I have ever met. Her perspectives on evaluation strongly resonate and align with mine – including the content of this blog post. She first caught my eye as a very interesting person during an evaluation conference when she remarked that the discussion was somewhat ignorant because the people did not appear to read the literature. And she quoted the words of a song to illustrate another important point.

In this post she writes about a matter very close to my heart. She is one of the only evaluators I have seen who is familiar with “doughnut economics” (a topic I have talked and written about since Kate Raworth developed this important concept), who engages with “transformation”, the important systems work of Donatella Meadows, and so on – issues that no evaluator can ignore any more. Weronika also connects what we do with the “big picture”, something that is essential during a time when the “toolification” of our profession leads us to an overly technocratic, superficial approach to our craft. I share her concern about the poor state of awareness and education for evaluation in this era in which we live. This is also why I am now engaged in the SDG Transformations Forum (you are welcome to join!). Please read her post with the serious attention it deserves.

Resilience of evaluationWhat dominates the world we live in today is the highly unsustainable fossil fuels addiction that is inducing upon us natural catastrophes that are abnormally speeding up migration crises, food and water shortages, wildfires, military conflicts and more. Indeed, infinite growth is impossible on a finite planet. The natural and social collapse that we start to observe is not something distant that we can leave for development evaluation in other distant countries in the “Global South”. Our generation already starts to experience anthropogenic (human caused) negative changes around the globe.

In order for evaluation to be part of any solution, and not part of the problem, we must embrace the radical transformation needed to become a field that truly analyses the particular problems and programs from the ontologically most up-to-date state of the art - and we need to be ethically and methodologically prepared to strengthen the resilience of local communities.

What we need now in the world is an evaluation portfolio that is truly transdisciplinary, radical in speaking truth to power, and critically analysing the systems in which we live. Otherwise we do not stand a chance to have our voices heard.

Basic environmental and systems dynamics readings are nowhere to be seen in our discourse or academic syllabuses. Yet certain texts have become essential to understand our planetary predicament in a wider scientific community – for example Planetary Boundaries: Exploring the Safe Operating Space for Humanity, A roadmap for rapid decarbonization, Doughnut Economics : Seven Ways to Think Like a 21st-Century Economist by Kate Raworth. Her book is being widely discussed among climate scientists and economists, yet it is rarely referred in the framework of evaluation discipline. What Raworth essentially proposes is based on the planetary boundaries idea of Rockstrom et al., adding on the social dimension to safe living on the planet as a result of her long experience of working in Oxfam.

The analysis should not stop here. It should be further critically assessed. For example, in 2017 an article in BioScience called World Scientists’ Warning to Humanity: A Second Notice, had 15,364 signatories from scientists in 184 countries. A year later 50 climate scientists emphasised that “The most critical question for climate research is no longer about the problem, but about how to facilitate the transformative changes necessary to avoid catastrophic climate-induced change” (Ten essentials for action-oriented and second order energy transitions, transformations and climate change research, Fazey et al, 2018). Or take a real-life look of doughnut economics in the current world (A good life for all within planetary boundaries, O’Neill et al, 2018) – clearly indicating the absurdity of SDG target 8.2.

Doughnut of social and planetary boundaries

The Doughnut of social and planetary boundaries.
Source: Kate Raworth, Doughnut Economics, 2017
Please click on the image to see the larger version.

To show one example of their data: in quantified, statistical terms they prove that no country around the globe is ready for rapid decarbonization and fit in socially just operating space which Kate Raworth proposes (see the figure below). Engaging with experts like Clive Spash explaining ecological limits to growth, Jem Bendell explaining the need for deep adaptation, or Alf Hornborg dwelling on alternative economics proposals appears to be just too much to ask from evaluators forever complaining about the scarcity of time.

If you are not already dizzy from all these cited materials, or are a bit climate-aware and masochist at heart with all the fears of what future brings, you might have come across Guy McPherson videos on near-term human extinction. Even when they do not radicalise the idea so much, many of climate scientists admit there is not much left hope for significant policy reforms, yet not much time to lose if we are to avoid human-induced catastrophes.

Even when knowing that we cannot assume the linearity of the future, it is unlikely that our students of evaluation across Europe (and around the world) will any time soon have a chance to reach for these sort of readings and discuss them in class. This is why I do not think formal evaluation training is putting the tools they would need into the hands of emerging evaluators to contribute to positive transformations in the era of the Anthropocene.

Social Thresholds

Please click on the image to see the larger version.

Yet if we do not stand on the right side of history, it could mean that evaluation runs the risk of being highly corrupted by governments and municipalities rarely prepared to consider any transformations or decarbonization.

I believe that evaluators in these turbulent times have an ethical obligation to be personally involved outside of contracting schemes. The world we live in and the communities we are part of need visionary leadership based on critical assessment and systemic analysis. We risk losing the relevance of our field if we remain only responsive, when instead we must become agents of the most up-to-date knowledge brokering!

The discipline of economics is on the verge of ground-breaking changes, when it will become impossible to ridicule ecological economics, steady-state economics and degrowth movement. Likewise, evaluation must break out from being the technocratic tool of ever marginalised states in the framework of (neo)liberal market economies if we want to claim it plays a vital role in improving the systems in which we operate.

In his keynote speech at the Conference on Transformations in 2017 at the Centre for Environmental Change and Human Resilience (CECHR) at University of Dundee in Scotland, Michael Quinn Patton was convincing the audience that even though we provide all of the necessary data about climate change, it is now so scary and civilization-threatening that the reports get ignored for decades. As this renowned evaluator suggested, all required data already exists. The problem is that neither are we as evaluators radical enough to facilitate its use, nor are politicians eager to do so.

So how to bring about desired transformations?

A crucial point is to change our “project mentality”. One thing that evaluation has proven over the last 50 years is that “projects don’t change the system”. So, what are our leverage points (as Donatella Meadows describe in “Thinking in Systems”)? Certainly not goals and targets which interfere with learning and complex systems, because they create narrow-mindedness. Also, not the best practices which implicate that the context does not matter at all. Leverage points are context-dependent and require our personal involvement.

My favourite quote from this talk is: “Rigour resides in thinking, not in method which is just a procedure”.

** If you are interested in how to perform a conference evaluation report in the form of a song, check Patton’s keynote from a link above. I certainly would like to see EES 2018 conference evaluation report in a similar form, maybe by Aretha Franklin whom we lost this year. Or at least to honour the new anthem of evaluation by singing this new version of Leonard Cohen’s song in Thessaloniki’s venue foyer on the Welcome Cocktail with you😉. **

These will be the questions and worries I am going with to Greece at the biannual conference of European Evaluation Society, hoping to meet at least a few not only well-intentioned, but also well-informed and radically thinking evaluators who could raise my hope for this profession.

If not, then at least this blog post will serve as a long list of good references for Young and Emerging Evaluators to read in upcoming months.

The quote which leads me in my everyday life in an intentional community in Latvia and personal involvement with the local community and professional work originates from Raymond Williams book Resources for Hope” (1989): “To be truly radical is to make hope possible rather than despair convincing.”

And so I wish to all of us that the efforts we make, the hours we spend on evaluation reports, and the change we actually bring, will be worthy in the eyes of our own children and the next generations on whom we keep on putting such incredible ecological and social pressures.

Share this post

Leave a Comment

Your email address will not be published.

Scroll to Top