In my previous post I highlighted the notion of Grand Challenges and where they have been developed. In this post I raise issues to consider when we think about whether the evaluation profession should have its own set of Grand Challenges.
Should we have Grand Challenges for Evaluation?
The practice of evaluation tends to advance incrementally. This is not unusual; slow, meaningful evolution with occasional ‘phase changes’ characterise most disciplines and fields of professional practice. But the world is entering a new era – one in which the Newtonian/Cartesian mechanistic models of the last 400 years is being replaced by a systems perspective on life, in which the Fourth Industrial Revolution promises to accelerate technological advancement in our lifetime far beyond our imagination, and in which we recognise for the first time our role in the destruction of the planetary boundaries within which humankind has to develop and thrive.
The 2030 Agenda and its set of Sustainable Development Goals (SDGs) present the best example to date of how this new era will be reflected in development policies and strategies through collective action in both the Global North and Global South. They show that interventions have to be designed with due cognizance of the complex systems nature of development. They challenge the perception that evaluation for development can be done without asking serious questions about the underlying development models, mechanisms of causation, the nature of ‘impact’ and ‘success’, the influence of culture and context, and so forth. Fundamental questions need to be raised around the unit of analysis, methodological and organisational issues, and the commissioning of evaluations.
I believe the time is right for the evaluation profession to identify and grapple with its own set of Grand Challenges. With the exponential growth in evaluation associations (VOPEs) and the forging of a dynamic global evaluation community through the IOCE/EvalPartners initiative, the Year of Evaluation and the Global Evaluation Agenda, we now have a global architecture and vision in place that can benefit from initiatives that can energise and focus participants not only at national, but also at regional and global levels.
Furthermore, due to the current state of evaluation- including the fact that anyone can become and “evaluator” without a deep understanding of the roots and evolution of the field – there is significant fragmentation and misunderstanding among many about what has been achieved in evaluation as trans-discipline and practice, the scope of the available body of literature and work, and the debates and trends that have shaped the profession to date.
Benefits of engaging with Grand Challenges
If well done, an effort to identify and tackle Grand Challenges in evaluation has the potential to
- mobilise and focus (latent) intellectual energy in the global evaluation community on key challenges and their solutions beyond capacity strengthening
- draw resources into research and innovation in priority areas
- highlight differences and common interests per region – an argument for identifying Grand Challenges per region
- focus attention, and create shared understanding among those involved in evaluation (including micro-economists, auditing and management consulting firms often outside the mainstream of the profession)
- draw from insights beyond the profession
- support IOCE/EvalPartners and the EvalAgenda2020, and
- educate other professional fields (and the broader public) about the potential of evaluation to contribute to major challenges confronting society and the planet.
In the end, Grand Challenges should lead to research and concerted action to advance the field of evaluation and support the role it can play in development – and importantly, direct more resources towards innovations that can solve key challenges that confront the profession.
Do we need Grand Challenges specifically for ‘Evaluation for Development’?
The 2030 Agenda for Sustainable Development emphasises that development is now a global charge, and an issue for every country – not only for those classified as low or medium income countries. Focusing Global Challenges on evaluation for development somewhat narrows down the focus while emphasizing the need to link evaluation explicitly and directly to development policies, strategies and interventions, from local to global levels.
It is imperative that the Global South participates strongly in such an initiative. In the past it has been less well positioned to mobilise its intellectual and financial resources. The profession is much younger there, and resources generally scarcer and under greater strain. And there is little doubt that ‘development’ is much more demanding in lower income nations; the scope of their challenges is so much greater than in more prosperous parts of the world.
Focusing on ‘evaluation for development’ also has greater potential to grab attention and inspire an interest in the profession, as it has the potential to demonstrate in practical and visible terms how much the profession and practice of evaluation can contribute to the world.
If we develop ‘Grand Challenges for Evaluation for Development’ we have to connect every Challenge to a well-defined notion of ‘development’. This will be very beneficial, in my view, as we tend to pay too little attention to what ‘development’ actually means. But it is also true that especially technical challenges are quite similar across the world. Perhaps we need ‘Grand Challenges for Evaluation’. Period.
And perhaps we need Grand Challenges for evaluation commissioners, evaluators and (potential) evaluation users!
What about the process?
A discussion has been initiated with key persons and organisations to test the idea of launching a process for identifying Grand Challenges for the evaluation profession. The response to date has been very positive, but a final decision to move ahead still has to be taken.
Such an effort should be driven by a solid partnership between key organisations, including EvalPartners and the VOPEs, especially the regional associations, as well as other networks of important organisational actors – even from outside the evaluation field.
The identification process has to be credible, so the engagement of experts from the demand and supply side, from theory and practice in the field, and from both the Global South and the Global North – likely as one or more advisory groups -will be essential.
Of course, all the information collected and analysed, as well as the eventual list of ten or more Grand Challenges (globally and/or per region) will be a public good, while the process itself will have to be transparent. Any person or organisation can follow up with any action. It will add an extra stimulus to the effort if funding organisations pledge to provide grants or awards for opportunities to tackle each of the identified challenges. It is impressive to see the incentives involved in for example the UK Global Challenges Research Fund (GCRF) – £1.5 billion.
The process does not need to be resource-intensive if a simple website platform encourages an open and transparent process; people are prepared to provide some volunteer time for the selection process; e-forums are used to solicit contributions, e-meetings held when appropriate, and events such as conferences are used to engage around ideas.
It will be great if, as a consequence of this process, organisations will put more funding into research and innovation in evaluation. Michael Scriven’s Faster Forward Fund is one of the few such initiatives. I still remember the response when I asked a number of organisations involved in SDG measurement and evaluation whether they will be prepared to support innovation in evaluation. All but one (kudos to IFAD) kept silent.
So incentives are important, but hopefully not imperative.
In addition to individual consultations, we are using evaluation conferences to initiate and stimulate discussions around this topic and the process to follow.
For example, there will be a relevant session at the upcoming EES Conference in Maastricht – The Fourth Industrial Revolution, the 2030 Agenda and Grand Challenges for Evaluation (Wednesday 28th September at 10:45).
If all goes well, there will be a similar session at the AfrEA conference to be held early in 2017. In the Asia-Pacific region, the evaluation association (APEA) is planning to have a ‘Great Evaluation Debate’ in collaboration with the Evaluation Capacity Development Group (ECDG) at the upcoming APEA conference in Hanoi.
While we hope these events will ignite discussions and create awareness, a more formal process will have to be launched to solicit contributions from individuals, VOPEs and other networks and organisations around the world. Should appropriate partnerships be established and relevant expertise be brought on board, the process of identifying Grand Challenges for Evaluation (for Development) – at regional and/or global levels – can be run during the first half of 2017, leaving space for analysis, selection and a round of discussions at the conferences that tend to permeate schedules during the last few months of each year.
Let us consider this as global evaluation community, and move forward if we think it holds promise as a desirable and viable idea that can accelerate the impact of our profession and practice.