Mission-oriented Evaluation

Share this post

Mission Orientated Evaluation

**8 min read**

We need Mission Thinking, Moonshot Thinking, and Mission-Oriented Evaluation if we are to ensure evaluation is fit for purpose for this era.

The thought occurred to me as I was reading the latest book by visionary and outspoken UK economist Mariana Mazzucato, who is on a mission to establish a global mission-oriented economy.

Her ultimate goal is to change capitalism to better serve the whole of society.

We know we need to do the same with evaluation.

A Mission Economy

Mariana argues for bold thinking, bold action and bold experimentation by organisations and groups from different sectors with one shared mission: collaborating and investing together to solve one (or more) of the ‘grand challenges’ or ‘wicked problems’ of our time.

With her roots in part from the Anglo-Western world, she uses the 1969 moon landing as key example of bold and aspirational thinking, coupled to effective collaborative action and resource mobilisation across many sectors. She argues for a much greater role by government than neoliberal dogma - propagated and taken to the extreme by the US - has allowed. She points out that what gave the moon landing direction and made it work was the integration brought about by the sense of being part of a mission “led by government and achieved by many” (ironically, and likely inadvertently, confirming the merit of the approach taken by China during its recent decades of fantastically rapid development).

She argues that what she terms a ‘mission-oriented approach’ provides directions for the economy, and then puts ‘grand challenges’ - the wicked problems that need urgent solving - at the centre of how the economic system is designed.

It shifts thinking from “what problem in the markets need to be fixed”, to “what kind of markets do we want”.

It means imagining a better future, rethinking the capacities and role of government in the economy and society, and recovering a sense of public purpose. It requires society-wide innovation, and a process inclusive enough to mobilise many value creators to work towards a more sustainable and just society.

She also argues for an outcomes orientation, and for targets that are ambitious and inspirational, and able to catalyse innovation across sectors and actors. She also wants the bland notion of “partnerships” replaced by clearer metrics for a “symbiotic and mutualistic ecosystem” – an excellent suggestion.

Mission-Oriented Evaluation?

What is mission thinking in evaluation? What would an evaluation moonshot look like? What are our grand challenges in evaluation? How do we create mission-oriented evaluation? Should we?

For this we do not take as starting point the problems that exist in the field, as we often do. Instead, we will re-imagine evaluation as professional field fit for purpose to meet the urgent and serious demands of this era – an era defined by the Anthropocene, the Sustainable Development Goals, geopolitical power shifts and tensions, COVID-19 and its aftermath, and rampant capitalism accelerating already spectacular inequalities.

We know evaluation practice has to change in significant ways, as many evaluators have recently pointed out, including but not limited to Michael Patton, Scott Chaplowe, Andy Rowe (see also here), Bob Picciotto, Thomas Schwandt, Bob Williams, Beverly Parsons and co-editors, as well as Deborah Rugg and I, also in this 2018 blog post. And of course, the many Indigenous and African-American evaluators, with some of the most prominent Bagele Chilisa, Fiona Cram, Nan Wehipeihana, Kataraina Pipi, Nicky Bowman, Vidhya Shanker, Stafford Hood, Rodney Hopson and many others who have consistently hammered home how Indigenous and Global South approaches have the potential to improve what we do.

The problem is not really the methods at our disposal, although they can always improve. We already have many innovations that can lead to much better practice – if they are widely used, which they generally are not. But we did add particularly useful and visible approaches over the last decade, such as Developmental Evaluation, Principles-Focused Evaluation, Blue Marble Evaluation, and increasingly visible contributions by Indigenous evaluators. We are also now more seriously grappling with the values that need to underpin what we do, as exemplified by Footprint Evaluation.

Ironically, given the theme of this post, we need to learn from considering the obstacles to quality and change in the evaluation field. We will have to embed, even in our mission thinking, recognition of the damage they cause. Here are six persistent hindrances to progress that stand out for me:

Meekly accepting the political economy of evaluation. There are very few incentives for those with power – the evaluation users, financiers and commissioners – to change practice.

Wallowing in comfort zones. Then there are those on all sides, including evaluators, who do not want to change and experiment beyond RBM and similar comfort zones. Money flows anyway, and time is a scarce commodity.

Vague about our value proposition and image. Many who have dealt with evaluation hate it because it is threatening, exposing or downright useless. We have been unable to formulate and display – for this time – an appropriate value proposition for evaluation.

Ignoring power. We do not focus on power (asymmetries) in systems when we evaluate – neither in processes nor in content; not in how we deal with our own global evaluation system; and especially not in how we deal with those who hold real power outside the evaluation space. This greatly weakens our influence.

Ignoring our ongoing innovations. We don’t know what we know, and should know. In reading books, articles and blog posts I come across many excellent practical contributions that can improve practice. But they tend to disappear without a trace, especially when they are not linked to big names or visible organisations.

Eyes inward. We talk mainly to ourselves – North-South aid actors, Northern and Southern governments primarily following the aid examples with the exception of a few who try to focus on the principles of South-South cooperation, parliaments, consultants, a few academics. Those who most need evaluation do not even know such a practice and tens of thousands of evaluation professionals exist. These include -

  • major international networks and increasingly prevalent networks of networks that are striving to be drivers of transformational change in the world;
  • academics dabbling in development and in evaluation, but also seriously using science to address local to global problems; and
  • the private sector that relies mostly on audit and management consulting firms, as well as some very good evaluation-oriented contributions such as GIIN and Impact Management Project.

The vast majority of these operate outside our conventional global evaluation community – which I define as those of us who go to evaluation conferences, read evaluation books and articles, belong to VOPEs and/or explicitly call ourselves ‘evaluators’.

We have great potential

The point is that we need some passionate people from all over the field of evaluation, and beyond, who want to work on changing the system - often a thankless task, as yet to make a real difference, but there is much hope and reason for optimism as long as energies hold. Here are some, while also acknowledging the many wonderful individuals who are fully committed to bringing about change.

Blue Marble Evaluation in itself represents many elements of mission-oriented evaluation. Among its thoughtful and innovative set of principles and evaluation criteria different to the DAC norm, it also prefers to consider ‘thought kits’ rather than the mechanistic ‘toolkits’ that drove Thomas Schwandt to lament the growing notion of evaluators acting as technicians through the ‘toolification’ of evaluation.

Footprint Evaluation is mission-oriented too.

And potentially also the transformation-focused work of other truly forefront initiatives like CIF’s Transformational Change Learning Partnership, the transformational project approach of GIZ (see also here) and that of MOTION - all set to inspire those with power and good intentions in the evaluation field to think and work differently.

The International Evaluation Academy (IEAc) – based on the concept of national or more precisely, global science, medical, engineering, etc. academies - is not yet formally in place, but nevertheless already bubbling with energy (website soon to emerge). In essence (and I am interpreting its brief here) it has the task to focus on something akin to mission-oriented evaluation. In the Academy Council we have been thinking about what might amount to grand challenges in evaluation, supported by emergent plans for a possible system-wide Three Horizons futures initiative done in collaboration with many others, and some essential and well-crafted ‘mapping’ of the field. The potential for moonshots as well as quick wins and less risky ventures are taking shape in the eight Council Working Groups.

EvalTransform is a just-emerging initiative based on the notion of collective action towards transforming evaluation in order to evaluate transformation. I personally would like to see become the point where all innovations in evaluation and efforts to work with and inspire transformational change come together: To share innovations as well as mission-oriented and moonshot thinking that can be transformative. To demonstrate what has been done and/or simulate what might be possible. To work collectively in innovative ways on an issue across disciplinary, geographic, cultural, sector and generational boundaries to improve practice in support of large systems change and transformation. And to grapple with, as Bob Williams pointed out in a recent email, how much we understand the change dynamic within the evaluation system and its environment. In other words - how do we, or can we better, influence or effect change?

This is an exciting time for evaluation! Together with the VOPEs and many other diverse efforts noted above and elsewhere, within and outside the evaluation field, we can craft streams of action that may just become a flood of collective power that will help ensure that evaluation becomes the best it can be for this time.

But first, let’s draw some systems maps. And mission maps. Let’s dream of moonshots as well as small beginnings. Then let’s collaborate or at least learn from one another to make them a reality.

Share this post

1 thought on “Mission-oriented Evaluation”

Leave a Comment

Your email address will not be published.

Scroll to Top