Guest Post – The etcetera of transformative evaluation

With the hiatus in my posts due to a fiendishly busy schedule, I am pleased to have this pertinent guest post by old friends – evaluation luminary Donna Mertens and Stephen Porter who has contributed much to evaluation in Africa. Donna has pioneered transformative evaluation more than a decade ago through a lens on evaluation that has social justice at its core, focusing on the lives and experiences of socially marginalised groups such as women, racial and ethnic minorities, people with disabilities and the poor. ‘Transformative evaluation’ is clearly not only about the evaluation of societal, ecosystems or large systems transformation from local to global level, which is my main focus. But it also has the important dual purpose of using evaluation both to transform individuals and to transform evaluation itself. 

 “Homer was able to construct (imagine) a closed form because he had a clear idea of the agricultural and warrior culture of his own day. He knew the world he talked about, he knew its laws, causes and effects, and this is why he was able to give it a form.
 There is, however, another mode of artistic representation, i.e., when we do not know the boundaries of what we wish to portray…we list its properties – and… the accidental properties of something… are thought to be infinite ” – Umberto Eco The Infinity of Lists

Umberto Eco, in the opening pages to his book The Infinity of Lists provides evaluators with a ticklish challenge: stability supports the definitions of a finite form of list, while instability requires a reforming of an idea with the recognition that the list of its attributes is likely to contain an etcetera. Providing an etcetera is not the laziness of an Evaluator, but a signal that a list needs to be constituted by many hands to make sense of an emerging reality.

Grappling with the instability of our current time is represented in our evaluation literature and indeed Zenda’s blogs, signaling that our lists and checklists need reformulation to guide and shift our practice. The transformative challenge is all around the evaluator. Importantly IDEAS has chosen Evaluation for Transformative Change as the theme for their 2019 Global Assembly. IDEAS did this based on the rationale that:

“transformational change is needed for our societies, economies and our relationship with the environment to become sustainable. On many fronts the world needs transformational change to be able to reach the aspirations expressed in the SDGs and the Paris Agreement: a world free of poverty; leaving no one behind; and ensuring a prosperous and equitable future in diverse and inclusive societies, with economies that increase wealth but not while undermining our food, clean air and living circumstances, with a climate and biodiversity that safeguards the future of humankind. A transformational change is one that reshapes models, policies, structures, practices, culture and management.”

The goal of societal transformation brings with it a challenge for evaluators. How does the role of evaluation need to adapt for the values of the SDGs? What is the list or lists we need? Are lists the appropriate response for evaluators who work in contexts of complexity?

The shift will involve the use of a transformative lens for the evaluation design, implementation and use. Zenda Ofir, in a post in this blog space voiced this challenge as follows: “One can expect that ‘transformation’ is the point of departure for development planning as well as for evaluation in a development context, which now includes both rich and poor countries. Yet when I look for literature on evaluation for transformation it appears to be meagre.”

The response to this challenge in the international development community has focused on systems and complexity theory (see, for example, a recent Webinar recording and Complexity Checklist), without explicitly bringing a transformation lens to evaluation planning, implementation, and use.  As a rejoinder we propose that we start to form two lists that challenge the international development evaluation community to configure transformation with an evaluative perspective. These include works by one of the authors of this post (Donna Mertens), and distinguished thinkers who have written about transformation from the perspective of feminist, culturally responsive and Indigenous evaluation.

List 1: What is the form of Transformative Evaluation?

Donna has previously proposed four common themes to transformative evaluation approaches:

  • underlying assumptions that rely on ethical stances of inclusion and challenging oppressive structures;
  • an entry process into the community that is designed to build trust and make goals and strategies transparent;
  • dissemination of findings in ways that encourage the use of result to enhance social justice and human rights; and
  • addressing intersectionality when culturally responsive, feminist, equity-focused, and Indigenous theories are relevant in the evaluation context.

Work on evaluation in climate change and sustainability challenge us, for example, a recent New Directions special issue, on adaptive management and from the climate investment fund,  to think about how evaluation relates to systemic change, accountability relationships, scale, sustainability…etcetera.

List 2: What resource on transformative forms of evaluation are there?

Existing resources on transformative forms of evaluation include:

These two lists are incomplete and can be elaborated to help achieve the SDGs in complex contexts. We accept the challenge, and hope that others in the international development evaluation community will as well, engage in the etcetera, shifting our practices so that evaluation can be substantive in achieving transformation, beyond its instrumental support to projects.

Share this article

Stephen Porter & Donna Mertens

Stephen Porter wrote this blog in his capacity as Research Associate at Wits School of Governance at the University of Witwatersrand. He has worked as an evaluation advisor at the World Bank, Department for International Development, Oxfam Great Britain, and as a Director with Oxfam America. He holds an MPhil in Public Policy from the University of Cape Town and has published nine peer reviews articles on evaluation. Donna Mertens is Professor Emeritus, Gallaudet University, specializing in research and evaluation methodologies designed to support social, economic, and environmental transformation. Her many books on research, evaluation methods and human rights include Mixed Methods Design in Evaluation; Indigenous Pathways into Social Research, and Transformative Research and Evaluation. She is a former AEA President and founding IOCE Board member as well as the Mixed Methods International Research Association.

Leave a Reply

Your email address will not be published. Required fields are marked *