We Studied a Zoo: How Two Studies Assessed Outreach Impact

11 February 2013 by Matt Shipman, posted in Uncategorized

Credit: Scott Liddell Capuchin monkeys. Image courtesy of Scott Liddell, http://www.scottliddell.net/

At the ScienceOnline conference earlier this month, I was bemoaning the dearth of literature evaluating the impact of science outreach activities. Luckily, Mun Keat Looi was part of this conversation, and he steered me to a 2012 paper he thought I’d find interesting. He was right.

The paper, “Assessing Public Engagement with Science in a University Primate Research Centre in a National Zoo,” was published in April 2012 by PLOS ONE. The authors note in the opening lines of their abstract that funding bodies are increasingly urging scientists to engage in public outreach and that it is important to be able to quantify the impact of these outreach efforts, “whose effects are more often assumed than demonstrated.” This is an argument I’ve made many times (and will be making again in a forthcoming post).

The authors also write that “a surprisingly small literature of such studies exists,” further endearing them to me. (I acknowledge that this is confirmation bias at work. They seem to agree with me; therefore, I think they are smart.)

The study itself is pretty straightforward. In 2009, and again in 2010, the researchers evaluated how much time visitors spent in the Living Links to Human Evolution Research Centre at the Edinburgh Zoo and the extent to which those visitors interacted with informational materials in the Living Links facility. Between the 2009 and 2010 assessments, the Living Links facility was modified to incorporate outreach materials, including additional information boards, interactive activities and computer displays.

Guess what? The addition of outreach materials had an impact.

In 2009, visitors spent an average (mean) of 9 minutes and 46 seconds in Living Links. In 2010, after the new outreach elements were in place, visitors spent an average (mean) of 12 minutes and 13 seconds in Living Links. But the real impact may be larger than that.

In both 2009 and 2010, approximately one-third of visitors did not interact with any of the outreach materials. The paper notes: “If we exclude visitors who did not engage with materials from the analysis, the time spent (in seconds) in engagement did increase significantly, tripling between 2009 and 2010. . . . These results suggest that what changed between 2009 and 2010 was that the two thirds of visitors prone to some level of engagement showed on average longer dwell times [interacting with the materials], indicating higher levels of interest in the materials added between years.”

In short, the paper states, “people engaged for longer when they were better informed.”

But I still had a lot of questions about the impact of the outreach materials. For example, did they improve visitor understanding of the scientific concepts behind the research? But, honestly, that’s just nitpicking. And somebody would do that study soon, right? In fact, someone already did.

There Is a Second Paper!

Shortly after reading the first paper, and expressing my excitement on Twitter, @CogSciLibrarian sent me a link to a second PLOS ONE paper published in September 2012. The second paper assesses the extent to which visitors engaged with, and learned from, outreach efforts in 2011 at the Macaque Study Centre in Marwell Wildlife Zoological Park in the UK.

The paper, “Evidence of Public Engagement with Science: Visitor Learning at a Zoo-Housed Primate Research Centre,” is fairly brief (and definitely worth a read), but I’ll mention a couple of the key findings.

First, zoo visitors were more likely to visit the centre and interact with outreach materials when a scientist was present. Simply showing a scientist at work makes people more willing to learn about science. Researcher involvement matters.

Second, zoo visitors reported that they learned more when scientists were present. And, because they were more likely to interact with outreach materials (like informative signs), the visitors are probably right. That’s because the study found that “zoo visitors also demonstrated an increase in knowledge and understanding if they interacted with information signage relating to specific topics relevant to the scientific research.”

In short, the study shows that outreach efforts resulted in non-scientists being more likely to engage with scientific material, and to actually learn something.

Good Start

No two studies are going to turn science outreach skeptics into science outreach enthusiasts. I co-moderated two sessions on facilitating science outreach at ScienceOnline, and neither of these studies was even mentioned. But these studies are excellent first steps in assessing the impact of outreach efforts associated with living labs and zoos in general. I hope these researchers continue their efforts to quantify and analyze their outreach, and I’ll certainly want to read about it if they do.

Hopefully, other researchers — at zoos and elsewhere — will follow their lead.

 


9 Responses to “We Studied a Zoo: How Two Studies Assessed Outreach Impact”

  1. Jon Tennant Reply | Permalink

    This is awesome Matt - thanks for the intel and writing about this! I really wish more studies did this kind of thing - this is the beginning of understanding the true impact of outreach imo (and could easily be extended to the scientific literature).

    I had a brief flash of an idea the other day. What if there were some sort of website where the 'end users' could upload brief (or not) testimonials about how scientific research has directly had an influence on them, their friends, or a project or something they work with etc, that feeds directly back to the initial researchers. This could probably be comfortably extended to outreach activities too, and actually be a way of documenting the 'true' impact of research/outreach. Any thoughts..?

    • Khalil A. Cassimally Reply | Permalink

      Jon, I think that's a great idea. Taking the incentive aspect that Matt pointed out into consideration, sharing very short snippets (couple of lines) which have the potential to go viral would be incentive enough IMHO. My line of thinking is that you start a Tumblr blog and angle it with pop culture. While the end result will be flashy and more numerous submissions, the essence of each submission will still be information/data which you can use.

      Am I making sense?

  2. Matt Shipman Reply | Permalink

    I think that's a great idea, but you'd need some sort of driver or incentive for commenters, if you wanted to get a significant amount of data. Also, you'd need to have a way to filter out spam (or to prevent one person from submitting dozens of comments).

    An alternative would be to find ways to connect social science researchers (in education/psych, etc.) with researchers in the life/physical sciences who engage in outreach. It would be great if methodologies for data collection on impact could be incorporated into the outreach design from the get-go. This would (hopefully) facilitate analysis of impacts, since you'd have clearer before/after data (as they did in the second study I wrote about in this post).

  3. Jacquelyn Gill Reply | Permalink

    This is fantastic, thanks for sharing! There are lots of great outreach ideas out there but, as you say, very little sense of effective impact. Do you think it just needs to be scientists, though? I've had an idea for a whole, after visiting Madison's free zoo many times, that one way to make the exhibits more interactive would be to station student "ambassadors" from classrooms, who could help answer questions. Classrooms could adopt an animal and redesign interpretive signage with what they think are the cool facts about the animal.

    Sorry to digress; I just think the human connection increasing the incidence of visitor participation is a fascinating idea!

  4. Matt Shipman Reply | Permalink

    That's an interesting question, and I'm not sure what the answer is. Would visitors be more likely to approach a specific facility/exhibit if there are "ambassadors" there? Or was it the appeal of seeing scientists actively engaged in research that provided the "draw"? Sounds like the outline for another study! :)

  5. Jen Walton Reply | Permalink

    As an environmental and science outreach geek who has always focused on the tools and impact of outreach, data on what works and what doesn't would be extraordinarily useful to me. I read new papers on environmental communication with gusto as they explain their methods for testing communication frames, messages, and content, but then their theory of communication relativity never seems to translate into application, which has been extremely frustrating.

    We think about this all the time at NEON, where I manage communication. I've been following the ScienceOnline discussion, but sometimes it is hard for me to identify with how I fit into this community because my focus is on actively engaging many audiences, including disengaged non-scientists. I would really appreciate a more closely aligned discussion between the writers and online specialists and the folks such as myself who can use that information to create awareness and inspiration (and maybe, ultimately, behavior change). I'd love to participate in a session about that link and how we can use lessons learned to apply new tools moving forward.

  6. Jen Walton Reply | Permalink

    I haven't; thanks for sharing!

  7. Anna Reply | Permalink

    Much-delayed comment, but there is in fact a field of folks (myself included) who study learning in informal education environments such as zoos, museums, and parks. Check out the Visitor Studies Association (http://visitorstudies.org/) and their journal, Visitor Studies (http://www.tandfonline.com/toc/uvst20/current) as an introduction to the field. Searching through the report archives of http://informalscience.org/ may also provide some good leads to studies. For a general look at the impact of zoos and aquariums, check out "Why Zoos Matter", an NSF-funded study (http://www.aza.org/uploadedFiles/Education/why_zoos_matter.pdf)

Leave a Reply


7 × = seven