Evaluation for Transformation. The one big mindset change we need in 2019

**5 min read**

In its very essence, the concept of evaluation is beautiful, I argued long ago in one of my first blog posts. Evaluation is also in principle powerful, more so than scientific research. Just like science, it aims to produce evidence and useful insights that can improve aspects of the world in which we live. But unlike most research, it has a sense of ‘immediate action’ about it. It is very explicitly intended to be used, if not immediately, then in the very near future. Evaluation has this very significant advantage over research.

Yet we do not yet use this advantage effectively. We have to do more to conceptualise, define, articulate, do, sell or use evaluation effectively. We still do not have a clear concept of its value proposition, especially in significant swathes of the (richer) world now confronted by the chaotic realities of self-interested capitalist politics and ill-informed leaders presiding over increasingly ignorant and evidence-averse societies. We also have a global evaluation system with little power; and where power is concentrated, it is in large part stuck with ideas that should die.

How can we change this to be more sure that we can make truly useful contributions in an era defined by the urgent need for transformative change?

We know we cannot solve problems based on the same thinking that created them in the first place.

Evaluation for TransformationAmong many others, continuing to use projects and programmes as our point of departure or our frame of reference is not going to move us forward, neither as nation-states nor as planet. We know that evaluation as a systematic practice arose from within (relatively) economically rich countries. In the past, in such contexts, fiddling with one policy or programme or project, trying to improve it, probably made sense. Today, evaluation should not be about ‘projects’ and ‘programmes’, yet we still find it challenging to think in an integrated manner, and move outside the “evaluands = interventions = programmes and projects” box when dealing with evaluation questions, criteria, judgments and recommendations.

This remains a major issue where we can least afford it: in the development cooperation (aid/ODA) evaluation space, which often simplifies the conceptualisation and framing of evaluation assignments in order to serve the needs of democratically elected politicians who have to live in the short term (next elections) or who have to maintain soft power – providing financial or technical support to get easily defensible and thus largely quantitative results to economically poor countries.

Thomas Schwandt clearly articulated the need for what he called post-modern evaluation in his brilliant EES keynote in October 2018 in Thessaloniki.

So, what is the most important mindset change we need in 2019?

This is the era of the Anthropocene; of the Sustainable Development Goals; of the Fourth Industrial Revolution and the digital economy; of Globalisation 4.0; of tremendous battles for power among nation-states as hegemony is challenged. In all of these, we are mere pawns in much larger, mostly invisible games between the powerful.

This is the world in which we have to make a difference. This is the point of departure from which we now have to craft evaluation as profession and practice.

This means that the grand narratives around evaluation have to change. Evaluation will still be a practice revolving around systematic values-informed judgment of value, merit, worth or significance. But it has to be conceptualised in a more nuanced manner, differently articulated and taught, and differently reflected in our practices. See for instance my DAC criteria series of posts which discusses the practical implications of thinking differently about this important aspect of our practice.

We have to make evaluation more powerful through our narratives as well as through what we actually do. We have to embed evaluative practice conceptually and methodologically in something more useful.

The following are some of the simplest, most obvious – although by no means the only – concepts that, if appropriately woven together, can bring us closer to new and more desirable narratives and practices around the value of evaluation in this amazing era in which we live:

  1. ‘Sustainable development’. Following the challenge posed to all countries by the 2030 Agenda, we have to contribute to development that endures – whether of an economically rich or poor society, or of a country, ecosystem or the planet. Even a humanitarian intervention should not be considered in isolation of this larger context. Of course, we should not blindly accept all that is written into the SDGs (especially given the conflicts between some of them), but together they provide us with a useful compass for action.
  2. ‘Well-being’. We have to contribute to individual, societal, ecosystems and planetary ‘well-being’ – a multidimensional concept that takes us beyond GDP towards the distant realm of happiness. This notion is intertwined with how ‘development’ can be understood. Like development, it remains a fuzzy, largely context-specific yet very important concept.
  3. ‘Responsibility’. We are good at focusing on human rights. We are much less good at focusing on the responsibility that goes with what we consider as our rights on this planet – as New Zealand academic John Schumaker points out very eloquently in his excellent article The Personality Crisis.
  4. Complex systems. Without exception, we have to think about everything from a complex systems perspective – even if we never use the term; we can instead just talk about “the systems of relationships between things”. I have already written extensively about this in my DAC criteria series, and there are loads of materials about this issue in the fields of evaluation, systems and complexity theory, and transformative change. The sooner evaluation funders, commissioners and specialists fully engage with this issue, the better.
  5. Managing risk. Evaluation can be a very valuable part of risk management, yet it is seldom articulated as such. We need to adjust our elevator speeches about evaluation to include this aspect, and do our work accordingly – for example by ensuring that we plan according to new insights, not old models, and fully identify and where possible, address the unintended negative consequences of plans and actions. Our organisations, companies, societies, countries, ecosystems and planet are in dire need of it. 
  6. Transformation. It is not a buzzword. It is an imperative.

This is going to be much more difficult than our current practice, but change is essential. Old thinking and ossified systems have to get out of the way if evaluation is to achieve its potential in this era.

We need smart thinking around the world, and a collective effort

Changing the evaluation system that drives our notions of what is evaluated, and how, for whom, when and with which trade-offs, will require a concerted collective effort by people with power. It also has to be crafted and driven by people fully aware of, sensitive to, and smart about – the demands of this era
– the multiple value systems and contexts around world
– the potential of evaluative action to help bring about essential transformative change
– the field of evaluation as it has evolved in all its diversity and dimensions
– the real meaning of capacity strengthening for evaluative thinking and action.

It will require thinkers and doers from around the world to drive new thinking in the evaluation system – people of the quality of pioneers such as Michael Scriven, Michael Quinn Patton, Robert Picciotto, Thomas Schwandt, Ray Pawson, Jennifer Greene, Donna Mertens, Karen Kirkhart, Rodney Hopson, Jonny Morell and many others.

We need next generation as well as indigenous thinkers and practitioners who can complement the ideas of (some also ground-breaking) inspiring evaluation specialists such as Weronika Felcis, Barbara Befani, Donna Podems, Kate McKegg, Nicole Bowman, Fiona Cram, Nan Wehipeihana, Adeline Sibanda, Bagele Chilisa, Apollo Nkwake, and many others.

We need to engage with the sharp insights of maverick thinkers across multiple relevant fields – systems thinking, complexity science, philosophy, new economics (‘evonomics’), behavioural science, cognitive science and more. People such as Dave Snowden, Dambisa Moyo, Steve Waddell, Kate Raworth, Ha-Joon Chang, Nassim Taleb, Slavoj Žižek, Jeremy Lent ….

We also need the people who hold the real power in our global evaluation system.

A nice challenge for all of us to tackle in 2019.

For inspiration …

If you want to be inspired in the new year, and understand the dire need for transformative change, watch the movie Living in the Future’s Past.

Or more poignantly, one of the brilliant Al Jazeera Witness documentaries (and readily accessible), Kisulu: the Climate Diaries. You have to be very cold-hearted not to be touched by his story.

Let all of this inspire you to be the best you can be in 2019 – for the good of your society, and for our planet, its ecosystems and all its beings.

Share this article

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *