**4 min read**
I met Leslie more than a decade ago when we served together on the Board of the American Evaluation Association (AEA). We had so much fun together with a great group of Board members. I loved Leslie’s sharp mind, excellent evaluation knowledge, great sense of humour and total commitment to our practice. She also sincerely cares about societies less privileged than those in the US. No wonder she was elected AEA President for 2018. She shares great insights – very important things that we all need to remember as we go about our practice as evaluation specialists.
I’m Leslie Goodyear, and I have recently finished my year as president of the American Evaluation Association. In my day job, I’m an evaluation practitioner at Education Development Center (EDC) in Waltham, MA. I’ve been studying and conducting evaluations for over 20 years. But before that, I worked in human service organizations in the US; I was a preschool teacher, an HIV prevention outreach educator, and a counselor and director of door-to-door fundraising at a rape crisis center. While I learned important evaluation theories and methodological skills in my graduate program in evaluation, my experiences in these three jobs inform my evaluation practice every day.
What did I learn?
Top Tip 1. Work in a program!
If you’re going to evaluate programs, know what it’s like to work in one. Take a job as a program staff person if possible. When I worked in these direct service non-profits, I helped people every day to access services; organized and distributed resources; wiped noses and tied shoes; answered crisis hotlines; educated families asked and answered questions; and raised funds.
My colleagues and I encountered all kinds of daily challenges related to staffing, funding, and responding to clients’ needs. And sometimes the challenge was that evaluation got in the way of doing our work. I experienced evaluators asking us to collect information that was time-consuming to gather, ask questions that were intrusive and demanded trust, and fill out forms that required time beyond our regular work day.
Evaluators bring amazing skills and expertise to help programs describe and demonstrate their influence. But those are not enough. Understanding what it’s like to manage multiple demands in real time – including those of an evaluator who’s breathing down your neck – helps to ensure that your evaluation work will at base not interfere with the good work being done each day, and at best acknowledge the ways in which the program staff can use and learn from the insights you generate.
Top Tip 2. We don’t change the world.
In the last few years the evaluation field has taken a turn toward a focus on social justice and activism. I think this is an important development. But it is critical to remember that evaluators don’t change the world; programs and the people who work in them do.
Building on my first tip, remember that it is our job as evaluators to ask good questions, collect data that address those questions, and provide information to stakeholders (at all levels) that helps them know whether what they are doing is making the change they seek. But that change is theirs to effect, not ours. We facilitate learning. We engage stakeholders and listen to the voices of those who are affected by the program. We respond to and manage evaluation funders, and sometimes delicately balance meeting their needs and the needs of the program and its beneficiaries. And we share our expertise, intuitions and insights with those who manage and implement the programs.
But we don’t answer the phone when someone calls in crisis; we don’t kiss the boo-boos and wipe the noses; and we don’t spend our days talking with addicts about the importance of using clean needles when they shoot up. No matter how much we care and how committed we are to changing the world – or even, how close our own experiences are to those who implement or benefit from the program – our job is to ask and listen, gather and share, measure and assess, and respect and support those who are on the front lines of change.
Top Tip 3. Outcomes are humans, too.
The job of an evaluator is to represent people’s lives – their experiences, hopes, dreams, challenges, failures, biases, accomplishments and relationships. This is an enormous responsibility - and one that can be forgotten when we turn people’s lives into data. No matter what data we collect, they are always about humans and their experiences, and we make a living interpreting and representing those experiences.
The format of the representation can bring those interpretations front and center – as in representations that use performance or stories to highlight stakeholder voices – or obscure them behind passive-voice reports, population level inferences, data visualizations, and statistics.
Regardless of whether we do a performance of interview transcripts or create beautiful data visualizations from quantitative data, whether we are different from the people who are served by the program or have similar backgrounds and experiences, we are still using our privileged position to interpret and represent (and even judge) others’ lives. And we need to be cognizant of this every day, in all our work, so that we never lose sight that we hold immense power to shape perceptions, influence decisions and affect lives.
Leslie Goodyear is a Principal Evaluation Director at Education Development Center in the US. She leads complex evaluations of national and international programs and systems, particularly government-funded programs focused on youth and education. She was president of the American Evaluation Association (AEA) in 2018, and has been associate editor of the American Journal of Evaluation, board member of the American Evaluation Association (AEA) and chair of the AEA Ethics Committee.