In one of my recent posts I wrote about the pervasive influence of power on development, and the imperative to deal with it in our evaluations. Here I want to address the evaluation of “empowerment” – the support given to individuals in their efforts to empower themselves.
In development such support usually comes in the form of an intervention (e.g. strategy, programme, project) explicitly focused on, or with elements of “empowerment”.
In my experience, truly empowered individuals – people with agency, as so well explained by Amartya Sen – are absolutely critical for development success. (Of course, this must be understood with nuance, as empowerment and power take on different forms in different contexts, something that I will discuss in a future post.)
Many people do not like the term “empowerment” when it is defined as something done to someone else and which then creates dependency that actually drains power. My definition and approach reflect the fact that as individuals we all have the ability and responsibility to empower ourselves, but our immediate environment – and hence development interventions – can either support or constrain us in doing so. Only sometimes someone else will play a more active, direct role in our empowerment. This is usually linked to the person who is above us in some formal hierarchy – for example, a boss who gives us permission or a chance to take on a new challenge, to represent the organisation, to have more authority through a promotion or larger team, and so on.
It is essential for those of us who work in evaluation to understand the multiple faces and nuances of “power” and “empowerment”. Good bedtime reading is the thought-provoking work of Jo Rowlands, Naila Kabeer and the resources posted on the website of OPHI, themselves immersed in this domain.
AWARD – inspiring empowerment
Since late 2009 I have been the M&E Advisor to the Steering Committee of AWARD, the African Women in Agricultural Research and Development programme. It is an inspiring, multifaceted fellowship programme that supports the careers of (often senior) African women agricultural researchers so that they may contribute to poverty alleviation and food security at the highest possible levels.
Between 2008 and 2014, more than 3,500 African women scientists from 11 eligible countries – Ethiopia, Ghana, Kenya, Liberia, Malawi, Mozambique, Nigeria, Rwanda, Tanzania, Uganda, and Zambia – applied for only 390 available fellowships over that whole period.
The AWARD management team decided from the start to have an M&E system that reflects the values underlying AWARD. In response, we developed a set of guiding principles (see box) that directed the design of the system.
A number of intertwined tactics made up the AWARD strategy during its first five year phase. It was based on three types of interventions – mentoring, the strengthening of scientific skills and the development of leadership skills. M&E that would be useful in such in such a programme clearly had to make use of quantitative and qualitative, perceptual as well as factual data and information.
The AWARD team used at times up to 15 different data collection instruments implemented at different times during and after the fellowship period – by all measures too many, but this was part of the special focus on knowledge generation in AWARD. It had only two objectives during its first phase, and one of these was focused on the generation of knowledge not only in and for the fellowship activities themselves, but through and for monitoring and evaluation.
In its new phase AWARD is shifting from developing individuals’ agency to helping to ensure that they have the opportunity structure to exert their agency – with nice new challenges for their M&E system.
Notes about AWARD’s M&E system
Many good things were done during the development of AWARD’s M&E system in 2009, which started right at the inception of the first phase of the programme. I mention only some of the most useful here:
First, a theory-based approach (using comprehensive albeit not “perfectly designed” theories of change and action) informed AWARD’s M&E system and facilitated adaptive management long before this became fashionable in international development evaluation.
Second, fellows and the management team participated in the establishment of the theory of change and in the set of theory-of-change led qualitative and quantitative indicators.
Third, monitoring was done of implementation progress and influencing factors, as well as of emerging impacts and their influencing factors. This was then used for critical reflection in support of adaptive management (although this did approach not always work as well as we would have liked – see some of the mistakes that had been made captured here!).
Fourth, in the application of our mixed methods approach we assessed the quality of the qualitative and perceptual information, and rigorously captured and coded the very large amounts of quantitative and rich qualitative data (i.a. fellows’ impact stories) on Dedoose research software, which made their systematic triangulation, integration and analysis possible. This was done through the tenacity of evaluation colleague Melody Mentz, who mastered Dedoose for this purpose and also had to go through the trauma of a month-long loss of all AWARD’s data because the cloud-based Dedoose software itself had crashed … (It is probably now the best protected research software in the world!).
Fifth, there was a stronger focus on sustainability, including of AWARD’s intended impacts, than is found in many of international development efforts.
Many mistakes were also made and valuable lessons learnt. But these are not the focus of this particular post. Instead, you can read more about these experiences here and here. In this post I only want to highlight one aspect: AWARD’s AWSEM framing of its effort to measure individual empowerment.
Evaluating “empowerment” using AWSEM
We developed the African (AWARD) Women in Science Empowerment Model (AWSEM) (pronounced “awesome”) a few months after we had set in place an M&E system for AWARD based on an agreed-upon, consultatively developed theory of change as well as indicators of achievement and progress.
I am always very keen to link proven ideas, models or archetypes found in the literature to my evaluation work. Early in 2010 I was studying empowerment models in the literature and was struck by how well the four powers articulated by Jo Rowlands in 1997 fitted with the indicators that were developed based on AWARD’s theory of change. I also found a related 2007 paper by Ibrahim and Alkire. I superimposed these four expressions of power on the individual empowerment part of AWARD’s theory of change, and found an almost exact match.
(More recently, the work of Sabine Alkire and colleagues at OPHI led to the Multidimensional Poverty Index based on a similar interpretation of power and agency.)
Coincidentally then, the indicators that we developed with the fellows and management team fitted to the tee the four expressions of power. Later on, with deeper analysis, we added a fifth power, one that was first raised by Arwen Bailey in her Masters project on communities of practice, and later coincidentally also by Robert Chambers. In my view AWARD has conceptualised this fifth power too narrowly around its specific focus on gender-responsiveness. Nevertheless, in our modern world the importance of this “power to empower” for leadership and success is set to grow and become more fully recognised.
All of this led to AWSEM, which has since 2010 framed AWARD’s monitoring and self-evaluation of their success in supporting their fellows’ empowerment. Indicators were developed for each of the AWSEM domains, and collected to the extent possible (some with greater, others with lesser success). Their first phase results can be found here.
I give here only AWSEM itself as articulated for AWARD. It can be tailored to a variety of contexts. I will provide further thoughts on its components, its use and contextualisation in posts to follow.
The AWARD (African) Women in Science Empowerment Model (AWSEM)