On Measuring Success

Miriam’s comment on Deb’s post, “Ninety-five,” motivated me to address something I have years of experience with: measuring success at nonprofits. (There is obviously also the question of success for movements, and I’ll address that another day.) In short, Miriam asks: “Is there any quantitative evidence that open door policies at sanctuaries leads to veganism?”

I’ll start with some jargon that I myself often find annoying, so I apologize in advance for when you begin to cringe.

• Every nonprofit—and animal rights organizations and vegan education organizations should be included—has a mission.
• In the service of that mission are goals and objectives (and the verbiage there isn’t clear, as they are often used interchangeably), which are often more tangible and specific than the mission.
• To demonstrate progress there are performance measures.
• There might also be benchmarks to be reached.
• Finally, outcomes are what you look at at the end of a given period to determine whether or to what extent you are achieving your goals/fulfilling your mission.

The programs or actions or interventions you conduct are supposed to be designed to have the best chance of getting you toward your goals. In the event that such “best practices” have already been established, it behooves you to use (“utilize“) them so as not to reinvent the wheel. This also makes the most effective use of your time and resources, as someone has already figured out the best way to do what you want to do. Therein lies the rub. The fact is that we don’t know exactly what will cause the average individual to question their relationship with sentient nonhumans and become motivated to desire justice for them. We have not isolated one specific thing, although we do know that appealing to emotions is likely to be successful, as the average human doesn’t live “in theory.” (Check out “Emotion vs. Logic” from earlier in the year for more on that.)

We have significant anecdotal evidence of many different interventions that were the inciting incident for various people: a trip to a slaughterhouse, watching Earthlings, reading Animal Liberation, Skinny Bitch or Eating Animals (I use them because people in my own life began their transitions after reading those books), attending a vegan potluck, visiting a sanctuary, listening to a podcast, picking up a leaflet. Some people even came to veganism and animal rights from different movements, such as human rights or environmentalism.

The fact remains, however, that we haven’t actually measured the success of our individual interventions. But the other fact is that I’m not sure we can, which is not to say we shouldn’t do some follow-up, survey style. That might not be as easy as it sounds, though.

There are two main points I have about this:

1) Success depends on how you’ve defined your mission and your goals, and what your performance measures are. So if your mission is to educate people about veganism, and the way you measure how well you are educating people is by the number of pamphlets you distribute, then you can quantify your success by that number. Your outcome is that number. Likewise, if you are a sanctuary and you want to conduct outreach to the general public and educate them about the lives of animals they ordinarily consider food (in addition, of course, to your primary goal of providing a healing place for them to live their lives), you could measure your performance by the number of people who came to the sanctuary for tours and various special events, and if you have literature you distribute, by that, as well.

However, the most important element that has yet to be cracked/decoded/identified is what makes a person take the step or leap from education and understanding to action–to changing their behavior. One thing we can definitely say is that . . .

2) Every intervention, every conversation, every trip to a slaughterhouse, every viewing of Peaceable Kingdom: The Journey Home is a seed. Some people need a lot of watering for that seed, and some people need none. And since we don’t yet have one “best practice” in the list of vegan education options that works for the majority of people, we have an obligation to make as many available as possible to the masses.

The average person is bombarded with messages all day long that tell them that animals are theirs to use. Everything about mainstream culture, from language to food to clothing to entertainment supports the notion that it’s perfectly normal and acceptable–if not our obligation as humans–to use sentient nonhumans (and some say we should do so “responsibly,” or “humanely”). We are better, smarter, more evolved, they were put here for us, and, of course, we can use them. I would think, then, that in order to have even the most remote of chances at succeeding at changing the behavior of people (as that is not the traditional definition of education), considering the minuscule percentage of the population identifying as vegans, we should use every angle available to us.

I think measuring progress is vital, but it’s also tricky. If you’re a sanctuary and you follow up with visitors, you’d have to have a well-designed instrument to survey them to determine your contribution to someone’s transition to veganism. Let’s say upon follow-up after six months, Visitor X is a vegan and was an omnivore at the time of his visit. What else happened in that six months? Any books, websites, films, vegan potlucks? Does Visitor X live with a vegan or know any vegans? What does Visitor X himself think led to his conversion? Qualitative surveys can easily get long and complicated, leaving data interpretation and analysis equally time-consuming. And with the primary goal of the sanctuary being the care of the animals and with everyone always in a less-than-optimal financial state, where are the resources for this evaluation coming from?

Quantitative data would be far more easily collected. Are you vegan now? Yes or No. However, that’s also deceptive because if Visitor X isn’t vegan, maybe the sanctuary was the only vegan-related seed and he just needs some more watering. Meanwhile, if the answer is Yes you can’t technically say that Visitor X is vegan because of the sanctuary visit if other things occurred (though you probably can if nothing else occurred). In fact, he might not even say it was a factor if there were others. He might say watching Peaceable Kingdom: The Journey Home did it for him. But we really can’t say because maybe if he hadn’t been to the sanctuary and met the animals, seeing the film wouldn’t have affected him as much!

I always think assessment and evaluation are great ideas and I definitely enjoy thinking about them more than the average person, but I don’t know how practical they are given the priorities of time and money involved in this particular cause.

—-

Photo from Flickr user Darren Hester