In 15 years I have not conducted a single evaluation where I did not ask many questions about the reasons why things were working well, or not so well – and triangulating from different perspectives and sources of information. I cannot imagine NOT doing this. It has given me the most valuable insights for judgments about progress or success.
I still fail to understand how we can ever think of doing evaluation without understanding the causes of lack of progress, or of success or failure, and the significance of such causes. Or, for that matter, ensure good interrogation of whether our ideas of “success” were appropriate in the first place.
The judgments of evaluators HAVE to be informed by such insights.
This means that I have great empathy with the second of the seven lessons for evaluation that call us to action in the SDG era. This lesson draws our attention - once again - to the need to stop treating development as if it is a simple matter of linear planning and linear, predictable trajectories towards predictable results. It is time that we get serious about getting rid of this folly. And in the process, we have to get serious about understanding the macro level influences on development.
Call to Action 2
Always consider the macro influences on development as part of treating the latter as complex system - even when dealing with short-term, time-bound interventions.
As planners and evaluators working in development we should turn the well-worn term, the ”Internet of Things” or “IoT” into the “Interconnectedness of Things”. It is clear that the world is moving away from the old mechanistic Cartesian paradigm to a systems understanding of life - and of development. Yet evaluative analyses often fail to make the necessary connections. This is particularly true when dealing with global phenomena and trends, and with the often complex dynamics among countries and regions.
This is also part of the niche of Michael Quinn Patton’s new “Blue Marble” initiative. Of course, this arena is pretty familiar terrain for evaluators and evaluation commissioners who work with complex national and transnational development and evaluation initiatives. We are all essentially “Blue Marble” evaluators, even if not always systematic enough in our approaches (there is certainly room for improvement in this respect). Yet it is clear that there is way too little attention by evaluation commissioners and evaluators on the global systems and dynamics that influence development, often right up to local level.
One loud voice focusing on these issues over the years in the development evaluation arena has been Bob Picciotto, as he has done again in 2015 in “The Fifth Wave” (for a short version see here). Those of us monitoring the evolution of trends can see for example that policy coherence and coordination have only slowly caught on as critical issues for inclusion in evaluation terms of reference - and they often, like several of the other issues highlighted below, remain mostly in the realm of rhetoric.
For the sake of development, let’s change this in the new era before us irrespective of how difficult this might be technically and politically. We need to contribute to the understanding major influences on development and their interconnectedness from local to global levels, and tailor our judgments accordingly.
In the next decades it will also be imperative to enable development in ways that are realistic yet not dependent on economic growth. Just look at the numbers. We all know about the overwhelming disparities in many countries in income between rich and poor parts of a population, and between rich and poor countries. But look at other issues that will affect transnational development. For example, the world needs an estimated US$6 trillion in infrastructure investment annually over the next 15 years just to address global warming only! The G30 has estimated that in the face of insufficient financial liquidity the annual investment that the nine top economies in the world (which account for 60% of world output) have to make to sustain moderate levels of economic growth (which some see as the engine that drives the global economy) is US$18.8 trillion (in real terms by 2020).
And by one popular estimate, 65% of children entering primary school today will ultimately end up working in completely new job types that don’t yet exist!
We certainly have to ask ourselves: With such forces at work, what role can evaluation possibly play to help make significant advances in how development is conceptualized and done? Can our work make a difference on a scale large enough to matter? What is the extent of the power that evaluation offers?
Macro influences insufficiently considered during the MDG era
The MDG era lessons point to the following critical influences that were not sufficiently considered in evaluations and statements about progress towards achieving MDG targets.
Whether we are operating at local or global level, at the very least we need to search actively during monitoring and evaluation activities for such macro level influences and the connections between them, assess their positive and negative impacts on development progress trajectories, and synthesise insights to improve recommendations and solutions.
First, changing or interrupted resource flows. Societal disparities and crises in donor countries are increasing. Several major donor countries now explicitly align their aid with national - often commercial - goals. This is resulting in shifts in aid budget allocations. Withheld, inadequate or fragmented flows of finance between countries have also in the past thwarted effective transition from global to national goals, and resulted in disillusionment and mistrust between partners. Furthermore, excessive emphasis on sector-based interventions caused sometimes detrimental displacement or skewing of national budgets.
Second, new types of investment. Public-private partnerships, impact investing, private sector investments in health and other services, and new financing modalities by the BRICS and other emerging economies through flourishing South-South cooperation have opened opportunities for a wider variety of development models and agendas. These types of investments demand expertise different from the prevailing “expert planner” approach, and beyond the fields of expertise and experience of most evaluators.
Third, obstructions and shocks, including geopolitical tensions and man-made and natural disasters: Tensions between countries or regions can paralyze, or encourage manipulation of, international fora. It can also affect resource flows, challenge conventional development models and internationally agreed norms and rights, and contribute to man-made disasters and national disasters. International conventions and dynamics around trade, financial systems, intellectual property rights and migration, conflict and the instigation of conflict, financial and economic crises, climate change and other man-made and natural hazards and disasters - all have knock-on effects on development progress and success that need to be better understood and incorporated in development.
Fourth, diversity and context-responsiveness. Thankfully, the era of “best practices”, “replication” and blueprint ideas of development forced upon countries and regions is slowly coming to an end. Tolerance and respect for diversity, and context-responsiveness are growing. The Global South is interested in creating or adapting development models for different values, cultures and circumstances, while the 2030 Agenda is explicit about the need to respect the context-dependent nature of development. It also reinforces the principle of common but differentiated responsibilities among countries.
Fifth, policy and strategy alignment and coherence. Integrated development was fashionable already several decades ago. Policy coherence and alignment are by now almost truisms. Yet evaluation seldom considers these issues with sufficient nuance. Coherence can have positive or negative impacts on development trajectories; what is good for one country or region is not necessarily good for others. Alignment between the SDGs and national development priorities and approaches is not a given, nor alignment in values and strategies between countries, or regions - in spite of negotiated “universal norms”. (For example, policies and interventions that advance the youth while marginalizing elders and their traditions might be acceptable in North America, yet have significant social consequences in Africa which are often not visible to an outsider, and not taken into account during evaluation.)
Sixth, technological advancements. Major trends in technological advancements, such as the increasing number of emerging disruptive technologies associated with the Fourth Industrial Revolution, have yet to be interrogated for implications for development models and their evaluation. There are also many other trends and shocks that are shaping our world and that will impact development. I will discuss this issue specifically in a next post on this second call to action.
Zenda Ofir is an independent South African evaluator at present based near Geneva. She works primarily in Africa and Asia, and advises organisations around the world. She is a former AfrEA President, IOCE and IDEAS Vice-President, AEA Board member, Honorary Professor at Stellenbosch University, Richard von Weizsäcker Fellow, and at present Interim Council Chair of the new International Evaluation Academy.