**4 min read**
Fred Carden is an outstanding evaluation professional. I have known him for many years as former director of evaluation at IDRC in Canada, where he has led, supported and encouraged high quality, useful evaluations as well as innovations in evaluation. He shares 30 years of experience in this one very pertinent message for YEEs – and for the rest of us.
Evaluation uses data and evidence from a range of sources. Data gets collected through surveys, interviews, organizational documents, spreadsheets, literature reviews, focus groups and so on. We all use many if not all these tools in our work. Most of us will also use many different methods and frameworks to conduct evaluations because no one method does everything or works in all cases. The ‘best’ method to use is the one fit for purpose. As evaluators we are concerned about evidence informing what we find in our studies and in the policy recommendations we make. We must also remember that what evidence gets promoted is a political choice, not an absolute truth.
Whatever methods and frameworks we use, we tend to rely on the scientific, the ‘objective’ data and evidence; we have been socialized to view it as the most reliable and honest evidence. We privilege this knowledge over other forms of knowledge, and so privilege the producers of scientific knowledge over other producers of knowledge. But scientific knowledge comes with its own cultural, gendered, political and class assumptions, so we need to be cautious in how we understand its findings.
In a recent volume with colleagues in Indonesia, we made the case that there are three main forms of knowledge. Scientific knowledge is certainly one of them. But it is joined by professional knowledge and local knowledge. Our book, Local Knowledge Matters, makes the case that good policy relies on the effective use of all forms of knowledge.
Professional knowledge is the knowledge held by bureaucrats, intermediaries (such as think tanks), and advocates. Professional knowledge synthesizes and consolidates ideas from a range of sources and, most importantly, connects them to context. It is the process knowledge that brings evidence and ideas to a community in ways it can be effectively used.
Local knowledge emerges from a society’s experiences and practice. We learn local knowledge through experience and observation of others in our communities. It is the social capital that allows individuals to become citizens and form communities. Local knowledge is most often a co-production between communities and their environments. It is often tacitly held and passed on orally from generation to generation. Importantly, like other forms of knowledge, it evolves and changes over time.
We make the case in the book that policy influence is strengthened when local, professional and scientific knowledge work together. We can apply the same argument to evaluation. It is stronger when we recognize and respect all forms of knowledge as we examine a project, a policy, a program or a systemic problem (such as climate change or the SDGs). Bringing local and professional knowledge to the table gives us much stronger possibilities that the findings will resonate with communities and be implemented. Both professional and local knowledge rely heavily on context and operating environment, and we know from forty years of research on implementation, that context matters a great deal.
In their seminal work on implementation in 1980, Pressman and Wildavsky examine the failures of the Economic Development Administration in the US. In looking at why economic policies failed, a major conclusion was around context – policies defined in Washington could not take effective account of different contexts, in rural Wyoming, New York City, in Los Angeles, or in Mississippi. Without a mechanism to bring local context to bear, policies would continue to miss the mark.
Putting its best foot forward, evaluation is about implementation, about improving the human condition through a better understanding of what works, for whom and in what contexts. If we are going to achieve that, we need to respect and use multiple forms of knowledge to make the best possible contribution to change.
Fred Carden has over 30 years’ experience in evaluating development research and programming, and in the development of evaluation methodologies – including 10 years as Director of Evaluation at IDRC in Canada. In 2015, he established Using Evidence Inc., a consultancy that focuses on efforts to improve the use of research and evaluative evidence in decision making. He has written numerous articles and eight books translated into a dozen languages. Fred holds a PhD from the Université de Montréal and a Fellowship in Sustainability Science at the Harvard Kennedy School.
See more at https://usingevidence.com. Fred can be contacted at fred@usingevidence.com.
I agree on the importance of the three kinds of knowledge but would add a caveat or two:
Think tanks are intermediaries in then US, Canada or the UK (and even there this can be contested). But in most developing countries they are the main producers of high quality academic/scientific knowledge.
Policy influence is not the same as informed policy. I think this is the ultimate outcome. So, you should instead argue that “policy is better informed when…”
Agreed on your last point. But in terms of high quality research, the Global Young Academies and the many national young academies demonstrate there are many researchers willing and able to produce high quality research in universities. Think tanks are privileged with donor finding to produce research on what the donors think is of interest. This is okay, and often very useful. But it should not lead to the marginalization of university-based researchers. A much stronger basic research environment is needed in any country that hopes for strong innovation and national economic growth.