**6 min read**
What would evaluation have looked like if invented “here”? How can it be re-imagined to help enrich our theories and practice worldwide? This question has been on my mind since 2006, yet only in 2016 I took the time to sit down to structure a reasoned framework around this idea. Since then I have occasionally used in my own work and in presentations what I have come to think of as the NICE – Navigating the Intersection of Culture and Evaluation – Framework for Research and Innovation in Evaluation.
It is in the West, primarily in the US, where systematic evaluation as we practice it today originated sometime in the early part of the last century. (Of course, we should remember that the Chinese were the first to develop systematic evaluative action for decision-making already during the Han dynasty sometime between 206 BC and 220 AD. This led to the initiation of such examinations in the West in the 19th century. Evaluative activities were already recorded during the Zhou dynasty around 1,000 BC.)
We are now in a good period in the evolution of our craft through the increasing focus on complexity as well as the excellent work of indigenous evaluators and others from different parts of the world, in particular New Zealand and North America. This is adding frameworks and new ways of thinking about what we do, how and why.
Use of the NICE Framework
I structured the NICE Framework while thinking about our own Made in Africa (MAE) Evaluation initiative here in Africa, grappling with what it means at the nation-state rather than at ‘community’ level.
The intent is for the framework to serve as an instrument that can help us in research and innovation in evaluation, and in advancing evaluation theory and practice – including identifying key elements that we need to attend to when fundamentally re-imagining evaluation in societal cultures different from those in the West.
It should force us to think deeply about where real differences between societies or nation-states might lie, and experiment with those in our practice.
Of course, the idea is not to diminish what we already have, but to enrich current evaluation theory and practice with new approaches and ways of working and thinking.
I tend to focus my attention about cultural differences at the level of a country or large parts of its society, rather than at community level where the focus is usually on one culture or on tribal customs. This is only to some extent useful when working with national development in multicultural societies. It is at national or transnational level that we most often ignore culture because we think it is too difficult to consider. Yet this is the level at which our governments have to think and work.
This is where I believe the NICE framework can be quite effective, and the reason why I refer to societal or national dispositions rather than one single culture. I touched on this in 2016 in a short article in EES Connections.
So, does the NICE Framework have aspects worthy of our attention?
I provide the framework here because it can be further developed. I am interested in your insights and suggestions. I also intend to write an article or working paper on the topic. (As independent international evaluator running around the planet my time and incentives to do work normally the ambit of universities and think-tanks are extremely limited. Yet as practitioners working across many boundaries – whether disciplinary, geographic, sector or ideological – we can analyse and integrate diverse experiences across many cultures with empathy and acute awareness of context. We can help to provide rich insights that have value for others.)
In putting together the framework I tried to steer away from conventional concepts and terminology in the social sciences that might divert us from key arguments or create divisions along disciplinary or ideological boundaries (ontology, epistemology, phenomenology, anyone?). I acknowledge that this might be a problem for some, but of course, the NICE Framework is just one possible approach aimed at making us think deeply about where how we can approach research and innovation in evaluation.
Although the framework highlights many elements, it is simple to work with once the underlying reasoning and rationale for the elements are clear.
Outline of the NICE Framework
The NICE Framework was developed on the basis that there are at least three important interrelated issues that have to be considered when working on the interface of societal culture and evaluation (i.e., where the culture of a society and evaluation meet/connect), and when we start to think about developing evaluation theories and practices from non-Western perspectives.
This is of course my own interpretation of what matters; it is possible that social scientists (I am a natural scientist by education) will have more discipline-informed classifications and ways of articulating these elements. Still, I trust the basic idea is sound.
First, there are those critical elements in society that we need to understand and interrogate because of the nature of ‘evaluation’. These are the elements #1-10 in the yellow box in the framework diagram. They are directed by what we perceive to be deeply embedded ways in which society functions and that will impact on evaluation (especially elements #1-3), as well as core aspects of evaluative practice that also relate to the dispositions and operations of society (especially elements #4-10).
Second, there is also society’s disposition towards evaluative thinking and practice that will influence their relationship with ‘evaluation’ as we currently conceptualise it. The framework captures – also through #1 to #10 in the yellow box – those elements that are likely to influence society’s understanding of, and attitudes towards evaluative thinking and evaluative practice as we know it.
Third, over time, the co-evolution of culture and context will influence societal dispositions (attitudes) to, as well as our conceptualisations of evaluation. These are the elements noted in the red and blue boxes in the diagram, which are also reflected in this diagram (see also here on page 15) which I derived from other sources in the literature to give a more visual picture of this well-known example of co-evolution.
Just to say a bit more about the last point: a nation-state or society’s shared history influences its culture, and vice versa. Shared history such as, say – thinking about the recent past only – growing peace and prosperity resulting from specific sources of wealth such as industrialisation, often coupled to colonisation (much of the West); or of apartheid and deliberate creation of inequality (my country South Africa); or of colonisation and slavery (much of Africa); or of the level of openness to influences from other parts of the world (e.g. Bhutan, which opened up to the world only a few decades ago).
At the same time, the culture of that society will also shape its history and contexts as it responds to external and internal forces. Take war, for example. Some nations forgive atrocities more easily than others. Honour and retribution are to some much more important than to others. Some value certain lives more than others so will hardly appreciate the suffering of many in places different to their own. Some are much more resilient in the face of adversity. And so on.
All this and much more will affect the interplay between history, culture and context. And this interplay will influence how evaluation is perceived, and how it should and could be conceptualised to best serve different societies across the world.
(Note that I prefer not to collapse ‘culture’ and ‘context’, as many do, also in development or evaluation frameworks. I believe this draws attention away from the important role that culture plays in the evolution of humankind and of different societies.)
Interpretations will, of course, differ per context and ideology. As noted, I also tend to link ‘societal culture’ to the boundaries of a nation-state that has some shared history and heritage, although it can also be used trans-boundary or for a particular part of a nation-state, such as those who share a language or religion. It can refer to a tribe or a ‘community’. Or to the largest distinct group of people who share important characteristics, such as (Sub-Saharan) African, East/South Asian or Western civilizations.
In my next post I will explain a bit more about the framework and it potential utility, with examples, to encourage some conversation around the approach and main elements.