Competencies for post-normal evaluation, Part 1

Share this post

**8 min read**

Are modernity’s blueprints for evaluation theory and practice exhausted?” Only now and again one finds provocative questions and insights during an evaluation conference keynote. A keynote that is Innovative. Thought-provoking. Visionary. In 2018, renowned evaluation scholar Thomas Schwandt delivered one of these rare speeches at the European Evaluation Society Conference in Thessaloniki in Greece - now published in the July 2019 volume of the journal Evaluation.

He invokes several key global trends that relate to evaluating society in the challenging era in which we now live, and then invites us all to consider whether our customary understanding of evaluation - what he terms “normal” evaluation, which he carefully delineates - is evolving into “post-normal” evaluation that can better reflect our zeitgeist. Among others, this requires thinking in systems, and considering complexity, emergence and plurality. It means thinking about, and working in new ways with the social, political, technological, economic and ecological complexities societies face. He concludes by wondering whether evaluation will indeed be resilient enough to adapt to the realities of complexity, uncertainty, and contradiction in ways that go beyond being only methodologically innovative.

Do read his seminal article in Evaluation if you ever think about the appropriateness of evaluation in today’s world – as all of us should do.

With this in mind, a while ago a very interesting blog post by Stowe Boyd caught my eye: 10 work skills for the post-normal era.

These ten skills offer a much more enticing and inspiring view than the lists normally bandied about, even by the World Economic Forum.

And they made me wonder: If these are the skills for the future of work, what are the implications for our evaluation profession and practice? To what extent would these skills reflect our notion of the qualities and competencies of a good professional evaluator?

Here are some of my observations on the relevance of the first five skills for evaluation this new era. How should they influence our framing of “evaluator competencies”?

Skill 1. Boundless Curiosity

Stowe Boyd: The most creative people are insatiably curious. They want to know what works and why. In a world that is constantly in flux, dominated by a cascade of technological, sociological, and economic change, the temptation may be to shut our eyes and close our ears. However, the appropriate response is to remain flexible, adaptable, and responsive: and the only hope for that is a boundless curiosity.

I cannot understand how anyone can be a good evaluator if s/he is not “boundlessly curious”. If we are to assess communities, societies, ecosystems and so forth, we need to do our best to understand how the world – around us and in its widest sense - actually works, how it evolves, and why. Narrow-mindedness, reductionist thinking, and indoctrinated, inflexible mindsets have no place in evaluation today.

More serious are the problems we cause when we buy into dominant (often wrong) narratives about how the world works (or not), yet ignore at our peril the more hidden, less visible narratives that might reflect better ways of doing. And as Boyd notes, what is now sought after is learning ability - the ability to “process on the fly” and pull together disparate bits of information. This is an important part not only of what evaluators (have to) do, but also of what we are increasingly encouraging others to do in order to be responsive and adaptive to their ever-changing contexts.

Skill 2. Freestyling

Stowe Boyd: We have to learn to dance with robots, not run away. … Today, the human-plus-machine teams are better than machines by themselves…. However, we still need to make sure that AI is limited enough that it will still be dance-withable, and not-runnable-away-from.

I love this one, probably because I am still at heart a natural scientist who also invests in companies that advance artificial intelligence. In an evaluation context, we had better learn fast how to use the advantages of new technologies for the benefit of evaluation practice, while being very aware that we are all on a highway towards severe abuses of technology that are already emerging across a good number of our rich, so-called “democratic” societies (indeed frogs in boiling water; I refer instead to “manipulated” societies). Thanks to thoughtful specialists like Linda Raftree with her MerlTech events we are dipping our toes into the subject. But so much more needs to be done in the light of the accelerating pace at which new technologies are evolving and becoming part of life.

Most importantly, we have yet to engage with what the pervasiveness of artificial intelligence and the accelerating integration of the technological and biological worlds mean for the ethics that guide evaluation and, even more challenging, for the values with which we evaluate interventions, innovations and systems.

Skill 3. Emergent leadership

Stowe Boyd: The ability to steer things in the right direction without the authority to do so, through social competence. … do you, at the appropriate time, step in and lead? And just as critically, do you step back and stop leading, do you let someone else? Because what’s critical to be an effective leader in this environment is you have to be willing to relinquish power.

I initially wondered whether this skill is relevant to evaluation professionals. We normally do not lead anything of substance. But we need “social competence” to know when to lead and when to step back, for example, when dealing with relationships in our highly politicised evaluation contexts. And we hold a lot of power when we develop our “tools” or “instruments”, putting our values into our survey questionnaires and data analyses. Or when we go on assignment - depending on the school of thought about evaluation - we act on a spectrum from “policing” to “guiding”, from “holding” to “co-creating” insights, from lightly steering when we facilitate sessions with stakeholders, to crossing the line by steering others into how we believe they should think.

But perhaps the most important challenge for our practice lies in how we are now going to evaluate governance, leadership, leadership programmes and the like. Whose values and ideologies about what leadership for this era should look like are going to count?

Skill 4. Constructive Uncertainty

Stowe Boyd: The idea of constructive uncertainty is not predicated on eliminating our biases. They are built into our minds as deeply as language and lust. … We have to accept at a fundamental level that we are inherently wired to be biased. ... Constructive uncertainty is based on the notion that we are confronted with the need to make decisions based on incomplete information. More than ever before, learning trumps ‘knowing’, since we are learning from the cognitive scientists that a lot of what we ‘know’ isn’t so: it’s just biased decision-making acting like a short circuit, and blocking real learning from taking place.

The need for this skill has to be emphasised. Not only in our normal technical efforts to reduce bias, but especially when dealing with those often rather arrogant researchers and evaluators who believe in “objectivity” and “certainty” when working with data, presenting “statistically rigorous” results, and value-laden evaluation "tools" as well as judgments. “Plausible cause", "sound evaluative reasoning”, "adaptive management", "reflection" and "triple loop learning" should be much more prominent terms and actions in our “objective facts” driven evaluation lexicons.

Skill 5. Complex Ethics

Stowe Boyd: All thinking touches on our sense of morality and justice. Knowledge is justified belief, so our perspective of the world and our place in it is rooted in our ethical system, whether examined or not. … Complex ethics are needed to jump-start ourselves, and to consciously embrace pragmatic ethical tools.

This is a critical issue for evaluation in a post-normal era. As Boyd notes, simplistic ethical systems will have to be rejected and displaced by complex ethical systems. For evaluators this means in order to make informed and useful assessments for this era, we will have to understand very well the issues that are at play as well as the trade-offs involved. Yet how many evaluation reports do you read that discuss or make judgments about trade-offs? Or thoughtfully question dominant narratives, including and especially those with ethical implications?

Boyd refers as example to the notion that the very important premises that underlie the theory of (the tragedy of) the Commons should in all likelihood serve as a foundation for future political order in both the geopolitical and workplace contexts. In my view, we will among many other issues have to reconsider ethics around our tendency to prioritise glibly and unthinkingly “democracy”, “human rights”, “individual rights” and “no-one left behind” irrespective of the societal order or the development challenges societies face. Our norms and ethics will have to reflect appreciation of the consequences of our many “aid” interventions that tear the fabric of societies. We will have to consider the fact that “good governance” that improves the lives of society in tandem with the environment is much more important than the political system through which it is done. In this era the world will need a far stronger focus on harmony in society (especially in multicultural societies), on collective interests and action, and on the rights of our natural ecosystems and the planet. In many instances this will mean trading off the rights of people - going against our conventional way of dealing with these issues.

How far removed is our current thinking and write-ups about evaluation competencies from these five skills? Do these five resonate with us at all?

The next five, about which I will write in a next post, might challenge us even more.

Share this post

Leave a Comment

Your email address will not be published.

Scroll to Top