Evaluation vs. Learning

Guest Blogger, Cindy Willard:  Challening the ways we measure program effectiveness

"Not everything that can be counted counts and not everything that counts can be counted." - Albert Einstein

There is a lot to talk about evaluation in the nonprofit sector. If your mission is to end hunger, how do you know you are meeting that mission? Current trends in evaluation ask for logic models, theories of change and other diagrams to design how inputs (programs and services) affect outcomes (changes in clients).

I recently attended a conference where I had the privilege to hear Peter York from the TCC Group speak on evaluation. He was speaking in front of a group of foundations and the title of his session was “Grant the Money and Run.” He challenged us all to re-think traditional evaluation. Often we are asking organizations to be accountable for variables out of their immediate control. He contends that typical evaluation measures only effect, not necessarily the cause of change. Instead, what if we aimed to understand both the cause and effect; what if we focused on learning instead of just evaluation.

He defines it this way. Evaluation – Did it work? Learning – What worked? And even more specifically, What worked, For whom and Under what conditions? Overall outcome data tells just one thing typically, but drilling down into program elements and organizational processes can tell a lot more about the work.

One great example he provided was of a Meals on Wheels program. These programs provide home-delivered meals to seniors and those with disabilities in order to help them maintain independence and increase their nutrition. Well, it’s pretty hard to scientifically measure nutrition levels without being really invasive. One way the organization could talk about their nutrition was to measure the nutrition in each meal and share that data. But were the seniors actually eating the meals? When the group did a survey, they found that several of the seniors were splitting the meals they received with their pets, because of course on a limited income there wasn’t a lot extra to secure pet food. So, the group started delivering either cat or dog food to these clients along with the meals for the humans. Each client in the household got what they needed, and most likely increased their health.

As a funder, this type of learning is impressive and important. It shows me that the organization is in touch with its clients and is willing to make adjustments as needed. It also shows that now seniors are mostly likely getting more nutrition than they were before. All good things.

Peter Senge has developed work over many years to help companies and nonprofits become learning organizations. It is hard work and requires time and resources. But so does traditional evaluation. Where will your organization see a bigger return on investment?

 - Cindy Willard, Private Foundation Sr. Program Officer