Competencies for post-normal evaluation, Part 2

Share this post

**6 min read**

Do we as evaluation professionals need a dramatic shift in our thinking about evaluation competencies if we are to contribute powerfully to the changes the world needs now? Where do we start to think about the competencies we need if we don't want to be trapped by convention? Here is my take, based on Stowe Boyd’s very interesting analysis of work competencies for a post-normal era. This is the second set of five. See the first set here.

Skill 6. Deep Generalists

Stowe Boyd: Deep generalists can ferret out the connections that build the complexity into complex systems, and grasp their interplay.

In order to be a good evaluation professional, it is essential to see connections. We have to be able to analyse, integrate and synthesise – what we know from past research and evaluation literature, what we derive from different sources and methods used during the evaluation, and what our own experience and intuition tell us. We need to understand as much as possible about which connections matter, the interplay between them, and how this might influence the evaluation findings, conclusions and recommendations.

Studying ‘what works’ is too limiting. We need to understand much more about ‘why’ and ‘under what circumstances’ changes have been or are taking place. This will help us to see connections that matter.

CompetenceStowe goes further. He argues that in today's uncertain world, for effective work we also need broad expertise. We need ‘deep generalists’ who can “learn a lot about a lot of things, and … get a real understanding of how they are connected.” He reminds us that in nature, species that adapt best to radically changing environments are generalists - yet most generalist species are too shallow, “living on the peripheries of more specialised ecosystems”. Therefore, he argues, to cope today increasingly needs people who are both specialists, deeply connected to the contexts in which they live, and generalists, able to thrive in many contexts. He concludes: “We can’t be defined just by what we know already, what we have already learned. We need a deep intellectual and emotional resilience if we are to survive in a time of unstable instability”.

The 2030 Agenda for Sustainable Development reinforces the importance of the many connections between agents, or processes, or goals, or outcomes. Evaluation professionals today most certainly need to operate as ‘deep generalists’. We need to be specialists without being confined to our area of specialisation. We have to understand matters beyond our own narrow field, and deeply consider connections in whatever and however we evaluate.

Skill 7. Design Logic

Stowe Boyd: It’s not only about imagining things we desire, but also undesirable things – cautionary tales that highlight what might happen if we carelessly introduce [for example] new technologies into society.

His main point is that those of us in the world of work also need to focus on the implications, not just on ‘application’ or ‘implementation’ of what they are doing -  implications at the design stage, during implementation, and once done. Tony Dunne and Fiona Raby’s argument in Design for Debate about the cultural, social and ethical implications of emerging technologies is equally valid for social contexts: This shift from thinking about applications to implications creates a need for new design roles, contexts and methods ... It's not only about designing products that can be consumed and used today, but also imaginary ones that might exist in years to come. And, it's not only about imagining things we desire, but also undesirable things -- cautionary tales that highlight what might happen if we carelessly introduce new technologies into society.”

My evaluation reports often have a chapter or section about the strategic and/or practical implications of chosen pathways, findings or conclusions. We do not do that often enough. So much of what I see in evaluation reports are descriptive rather than analytical, and stop short of considering the implications of what we find. Social innovations bring new ideas, yes, but their implications need to be explored and methods to do this advanced. There are now swathes of books and articles that urge evaluation professionals to study especially unexpected negative consequences or outcomes of actions and interventions (i.e., externalities). More than that, we need to consider whether trade-offs are worthwhile. Yet the number of evaluation professionals who do this systematically and with some rigor remains extraordinarily low. This means that many assessments are essentially useless, and certainly not credible.

Skill 8. Post-normal Creativity

Stowe Boyd: Creativity itself has changed, and in post-normal times creativity may paradoxically become normal in the sense that it will not be the province of lone tortured geniuses any longer (which it was not anyway), but an everyone, everyday, everywhere, process.

Innovation or creativity is today the foundation of business, and essential for finding solutions to major challenges. It is also essential for advancing the field of evaluation. Should it be the responsibility of evaluation professionals to understand better not only how to evaluate innovation, but how to be innovative in every evaluation where this makes sense – not just mindlessly following methodological recipes, but advancing the field of evaluation through small and large innovations, and sharing them with our peers?

Skill 9. Posterity - not History, nor the Future

Stowe Boyd: George Santayana famously said, “Those who cannot remember the past are condemned to repeat it.” However, while we need to learn from history, we must not be constrained by it, especially in a time where much of what is going on is unprecedented.

In my view, this is the most important skill to add to our competency frameworks. According to Stowe, “‘posterity’ implies continuity of society and the obligations of those living now to future inheritors, a living commitment, while ‘the future’ is a distant land peopled by strangers to whom we have no ties”. Instead of being selfish, it is therefore essential to identify “not only with a community in space but also with a community extending over time from the past into the future”. Greta Thunberg has highlighted the importance of commitment to future generations, and to the world we will leave them.

This is part of a new ethics for this era. It also requires imagining and ‘designing’ futures fully cognisant of the people who will inhabit them. Evaluation professionals use the past to help inform the future. But do our evaluation questions and criteria reflect a clear notion of the responsibility of this generation to future generations, the need to break from the past - and hence the stewardship that is needed to ensure sustainable development ‘for posterity’? To my knowledge only the Canadian Evaluation Society has included this explicitly in their guidelines. If this is so, we most certainly need updates of our competency sets across the global evaluation community.

Skill 10. Sensemaking

Stowe Boyd: Skills that help us create unique insights critical to decision-making. Or twisted around a bit: We need to nurture the ability to create flexible models to derive meaning from a set of information, events, or the output of our AIs, and determine a course of action.

Sensemaking is a higher-level thinking skill that cannot be codified, ‘the ability to determine the deeper meaning or significance of what is being expressed’, creating unique insights critical to decision-making. It is in essence the combination of the skills of analysis, integration and synthesis – the actions at the core of evaluation. I often do quality assurance for evaluation offices. A significant portion of evaluation reports that I get to assess are descriptive, devoid of clear evaluative reasoning that shows analytical abilities, or the synthesis that is needed to arrive at valid evaluative findings and conclusions.

In my third and final post I will summarise my thoughts about evaluation competencies for this era.

There’s a small cadre of agitators (I include myself) shouting out that the times are a-changin’, but I don’t know how far our voices carry, or if others can understand our words.  

For last year’s words belong to last year’s language

And next year’s words await another voice.
— TS Eliot, Little Gidding

Perhaps this is proof, once again that we need new ways to think about — and talk about — this rapidly changing world: we will have to find another voice. Perhaps that is the 11th skill.

Stowe Boyd, Medium, 2017

Share this post

Leave a Comment

Your email address will not be published.

Scroll to Top