Evaluation – a Beautiful Practice

“All the diversity, all the charm, and all the beauty of life are made up of light and shade.”
– Leo Tolstoy, Anna Karenina

I believe that evaluation is a beautiful practice.

Of course, I base this assessment on my own definition and sense of beauty. The nature of “beauty” has been the focus of many philosophical discourses over millennia. You may want to challenge such an assessment of our profession and practice (let us set aside for the moment the notion of evaluation as a transdiscipline), and the criteria on which I have based my assessment. You may want to use another adjective – perhaps “useful”, or “noble”. But I want to continue to consider it as “beautiful”.

I cherish beauty wherever I find it, and not only in what we can see. Poetry can be beautiful. Or music, a mind, the truth (sort of) – even an idea. It is important that we nurture beauty, and strive to create it, while remembering that some well-known philosophers have argued that things are beautiful only in relation to the uses for which they are intended, or when properly applied.

I raise this issue because in my view we should spend more time and intellectual effort appreciating our profession in a way that can show the world its true value proposition, and at the same time slow down the headlong rush towards its bureaucratisation and “toolification”. We are a profession increasingly inhabited by technicians, as Thomas Schwandt points out in his recent book on the need to cultivate “a life of the mind for practice” – an outstanding contribution that should be compulsory reading for every evaluator, evaluation commissioner and educator in evaluation.

In a delightful book, A Beautiful Question, Frank Wilczek, a winner of the Nobel Prize in physics, meditates in 430 pages on whether the world is a successful and beautiful work of art. His central question “Does the world embody beautiful ideas?” is answered with a resounding “yes”. He argues that the innate human appreciation for beauty lies in the deep design of nature itself. He finds beauty in symmetry and geometry; in conceptual purity, mathematical order and harmony; in “All Things are Number”; in complementarity; and in the truths, possibilities and almost-truths of “deep reality”.

Does evaluation contribute to the beauty in the world? (Or to the beauty of the world?). Can it do so?

My response is that evaluation as profession and practice is intrinsically beautiful – but how we practice it, frequently not.

The beauty in evaluation

That which is striking and beautiful is not always good, but that which is good is
always beautiful.
– Ninon de L’Enclos

So for me, evaluation as a profession and practice is beautiful only if we do our work well. Then I find beauty in the following:

It is rooted in something that comes naturally, and that has been there for millennia.

In a though-provoking recent analysis, Ernest House draws on Daniel Kahneman’s famous book, Thinking, Fast and Slow, to conclude that both the complementary systems 1 and 2 thought processes are fundamentally evaluative; that evaluative thinking constitutes the core of our thought processes; and that professional evaluation can be considered “an institutional embodiment of our thinking processes”. Evaluating is thus a natural thought process, and a primary strategy by which humans adjust to their world.

As noted by Michael Scriven in his provocative analysis of roadblocks to its recognition and revolution, evaluation may well have existed for several million years in the form of “a simple commonsense-level practice, a proto-applied science”, long before the emergence of the species Homo sapiens.

It is in its very essence aimed at doing good.

Evaluation is not done to report, or to understand (only). It is done to trigger positive action. We use it to make incremental advances or, where possible, to help transform the lives of others (often those most disadvantaged and vulnerable), heal ecosystems, and pull the planet out of distress. We do this by supporting the agency of people and societies to help themselves and the world around them. This helps both them and us to live skilfully and mindfully in the modern world.

In his book Schwandt provides a fine list of some of the most valuable purposes of evaluation: It determines programme effectiveness; demonstrates return on investment; guides resource allocation; improves ongoing programme operations; assists in creating a climate of critical reflection; informs the citizenry; demonstrates agency, accountability and transparency; and provides information needed by decision-makers. We can add that it can also direct planning and strategy, support advocacy, and contribute to our pool of useful knowledge about issues of importance. Perhaps most importantly, it has the charge to strive to speak “truth” to power whenever feasible and desirable.

Such a profession can add a lot of value to the world. It carries great responsibility. It also gives life meaning. And it has to be done well.

In the long term, it will be too fundamental to ignore.

As Schwandt notes, evaluation is “a professional practice committed to promoting the public good through judging the value of human activity.” It is also still a highly contested arena, with great diversity, inherent tensions, and challenges by other disciplines. Scriven argues that evaluation is a busy, constantly evolving transdiscipline and, in spite of a persistent denial of this fact by other disciplines, it is actually deeply involved or embedded in all disciplines. It is an essential tool in the toolbox of every other discipline. It just still has to be recognised as such.

I agree.

It compels us to have multiple perspectives on a matter and to integrate, acknowledging the inherent complexity yet underlying orderliness of the world.

Mars Rover

A different perspective: Earth as seen from Mars. Source: Picture taken by the Mars Rover, NASA”.

We are moving from almost 500 years of dominance of Cartesian-Newtonian science to an era of holism and a systems perspective on life. Although much has been lost over past centuries, much has been gained; perhaps one had to precede the other to bring us to this important juncture.

But now we have to be systems thinkers. I believe that a good evaluator will invariably view matters from multiple perspectives, respect the ecological interdependence of things, and tell a story that connects that which is being evaluated with the wider picture.

Good evaluation searches for patterns rather than “best practices”, and integrates wherever this makes sense. It analyses and judges in context and with respect for context. It connects the “micro” with the “macro”, and recognises order within the apparent chaos.

It forces us to engage with, and strive to understand “reality”.

Evaluators are immersed in the real world. Real lives are at stake, and when societies and ecosystems are vulnerable or fragile, the stakes are high. Evaluators have to understand past and current realities, and for good advice, do their best to predict what reality may look like in future.

Plato mused that everyday life is a shadow of reality. Even though I do not interpret this as Plato has done, I like to believe that evaluation is one aspect of the “adventures of the mind” that can bring us a step closer to the essence of reality, which is “clearer and more beautiful than its shadow”.

It recognises the value of complementarity and balance.

Wilczek embraces the “wisdom of complementarity”. In the East, this wisdom has been reflected for thousands of years in the fundamental principle of yin and yang. In many respects, good evaluation honours this wisdom. Sometimes we observe opposing or contradictory forces in an organisation or system as “tensions”; sometimes we should rather view them as matters of balance and complementarity.

Good evaluation considers positive and negative influences on interventions. It looks out for positive and negative impacts. It strives towards objectivity while recognising subjectivity. It uses and integrates quantitative and qualitative designs and methods. It values independence as well as engagement.

We also strive for balance – in our judgments, and in what defines or directs our judgment. We balance the need for precision and rigour with what is practical and feasible. We have been steered by phases of reductionism, and are now being enriched by integration and holism. Even these are complementary, and beautiful when seen in balance.

And finally, as evaluators we have to engage with some of the most positive aspects of life, as well as some of the most distressing.

Evaluation engages some of our deepest, most valued capabilities as humans.

Evaluation is an art and a science. Schwandt points out that evaluators – like historians, archaeologists, investigative journalists, scientists and detectives – have to display respect for multiple types of evidence, be persistent in seeking it out, and take care in weighing it. Evaluation requires technical acumen. It also demands critical thinking and reasoned judgment which need human interpretation that is based not only on evidence, but on intuition, deep understanding, lived experience (perhaps transferred over centuries), consideration of values, engagement, responsiveness, a certain measure of empathy, and more.

Studies argue that there is a good chance that complex problem solving, social and systems skills will be far more in demand in future than physical abilities and content skills. Futurists estimate that there will be major job losses over the next two decades, driven by disruptive technologies and increasing automation. Who knows what will happen? But despite the recent surprising defeat of a champion go player by Google’s AlphaGo, it remains unlikely that the evaluation of complex matters will be done by machines in the near to medium term future.

In the end, all that matters is that we should strive to display, and enhance, the promise of beauty that evaluation holds – for the benefit of humankind, and everything around us.

What is just, also always agrees with truth, knowledge, and – in the deepest understanding of the term – it is beautiful. – Plato

 

Share this article

Zenda Ofir

Zenda Ofir is an independent South African evaluator. Based near Geneva, she works across Africa and Asia. A former AfrEA President, IOCE Vice-President and AEA Board member, she is at present IDEAS Vice-President, Lead Steward of the SDG Transformations Forum A&E Working Group and Honorary Professor at Stellenbosch University.

8 Comments

  1. Thank you Juliana. I know not everyone will share my perspectives, but we should in my view more frequently take pleasure in considering our craft from outside purely technical boxes.

  2. Zenda
    Evaluation is indeed a beautiful practice. You have demonstrated this beautifully. But evaluation is not yet a profession. Achieving the prestige and status of a profession will not come easy. Indeed, the history and sociology of professions confirm that being recognized as a profession is always the result of a highly competitive process. It is, in your own words, one of the “grand challenges” we face. There is no mystery about what needs to be done: compliance with internationally agreed ethical standards, availability of high quality education at all levels, professional autonomy and credentialing. These are prerequisites for accessing the higher echelons of the knowledge occupation ladder. They are implicit in the Global Evaluation Agenda and beyond the setting of conferences and the sponsorship of evaluation publications, they should be at the core of all evaluation associations’ agendas… See my article on: The Logic of Evaluation Professionalism in Evaluation 17(2) 165-180, 2011. (http://journals.sagepub.com/doi/abs/10.1177/1356389011403362)

    • Bob, you are indeed right. I have emphasised “practice”, so we can best interpret my use of the term “profession” as aspirational. Should we assume we are at present a practice and an “occupation”? Discussions about whether we are a discipline or transdiscipline also continue. Tom Schwandt has a very relevant article in the latest AJE http://journals.sagepub.com/doi/full/10.1177/1098214017728578 that highlights eloquently the need to focus as part of our professionalisation in addition also on the professional ethos of evaluation, to ensure that we have clarity about evaluation’s role in society and the social good we aim to serve. Thanks also for your very useful reference. Both articles are well worth keeping in mind on this important path.

  3. As Scriven has decisively argued we are a discipline as well as a trans-discipline… but we are at best a “fledgling” profession. As to the central role of ethics in evaluation please see Scriven’s authoritative chapters in Evaluation for an Equitable Society (Evaluation and Society, Information Age Publishing) April 1, 2016 by Stewart I. Donaldson and Robert Picciotto (Editors)

    • Bob
      Thanks for this very useful reference – the book seems well worth reading. Note that Tom Schwandt’s AJE article argues for stronger engagement with what we can consider our professional ethos, not ethics in evaluation per se. Both are very important topics at this time in the evolution of our practice.

      • Zenda
        The Global Evaluation Agenda framers took for granted that evaluation is mandated to work in the public interest. All professions claim devotion to this transcendent value and all subscribe to a “professional ideology of service (that) goes beyond serving others’ choices (Elliot Friedson, 2001, Professionalism: The Third Logic, Polity, p. 122) “. Thomas Schwandt’s call for a public debate about the professional ethos of evaluation has merit but it has already been heeded — in his own writings as well as in major contributions by Michael Scriven, Ernest House, Jennifer Greene, Donna Mertens and many others. Nor is Tom’s recommendation to ground democratic professionalism in evaluation by involving citizens in the evaluative process controversial. By contrast, what Tom calls the narrow considerations of the Global Evaluation Agenda of (promotion of independent evaluation in all sectors of society, increased access to high quality tertiary education in evaluation, professional development through reflective practice, credentialing and certification) have been neglected even though they have long been and remain critical not only to help propel evaluation to the top of the occupational ladder but also to help implement Tom’s vision of a re-invigorated, progressive, democratic evaluation discipline.
        Bob
        PS The EES capabilities framework includes “Upholding ethical standards and democratic values in the conduct of evaluations” .

  4. So as long as some so called evaluation professionals conduct evaluations in an unethical manner, evaluation looses its beauty. We need to cherish ethics while conducting evaluation.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.