Unconventional Metrics: How Can I Tell If My Blog Is Working?

28 October 2012 by Matt Shipman, posted in Uncategorized

Photo: ljleavell/stock.xchng

So you started a blog to give yourself a place to write about science. You’re using social media to push your posts out, and you know that some people are reading them. But how can you tell if you’re reaching the people you want to reach? And, more importantly, are you any closer to achieving your communication goals?

On Oct. 27, I was part of a panel at the National Association of Science Writers meeting in Raleigh, N.C., that addressed this issue, and I thought I’d lay out my thoughts here. (Recap of the entire session is available on NASW's site.)

In any communications effort it is important to set goals – and then use metrics to determine whether you’re on track to meet those goals (more on this in a future SciComm 101 post). But conventional metrics are not always the best indicator of success. [Note: the Communication 101 post is now up.]

For example, conventional wisdom tells us that the success of a blog is determined by the number of unique visitors your blog gets, or by the number of page views. Part of my job as a science writer at NC State University is maintaining our research blog, The Abstract, so I’ll use that as an example.

From Oct. 1, 2011, through Oct. 1, 2012, we published 114 posts on The Abstract, which received a total of 89,395 unique visitors and 132,911 pageviews. Based on those numbers, we’re doing okay – we’re not doing great. But our goal in launching The Abstract wasn’t to get a lot of page views, so those metrics don’t mean much.

So what were our goals when we launched The Abstract, and how are we measuring progress toward them?

Goal One: Promote NC State research to external audiences, such as potential students and faculty.

Example Metric: Did the post help us amplify the story via external news media? If we were able to use the post as an effective pitching tool to reach reporters, then we would ultimately be able to count the stories that we got out of it. And, if a story ran in, say, the L.A. Times, we know that we reached a fairly broad audience. Because Abstract posts have led to hundreds of stories in outlets ranging from Scientific American to National Geographic News, we know we’re doing well on achieving this goal.

Goal Two: Help researchers disseminate findings (which makes funding agencies happy, may boost citation rates, etc.).

Example Metric: Look at whether your posts are being featured on the sites of funding agencies. For example, between Oct. 1, 2011, and Oct. 1, 2012, 39 Abstract posts were featured on the National Science Foundation’s news site, Science360. That means 34 percent of our posts were being highlighted by an important federal funding agency, which indicates progress toward our goal.

Goal Three: Achieve specific communication goals as they come up (I’ll explain below).

Example Metric: Sometimes a researcher will come to us with a specific problem/challenge, and ask for our help. Because the challenge is different each time, you have to come up with a different metric for each set of circumstances.

For example, a computer science researcher was launching a data-sharing initiative called the Android Malware Genome Project earlier this year. Data-sharing initiatives present a circular problem, from a story-telling perspective: the researcher can’t do anything until people know about it and start sharing data, but there’s nothing to tell people about until the researcher has started doing something. How do you spread the word when there’s not much to say? Answer: short blog post.

We posted a four-paragraph post on May 22. I did a Google search for the term "Android Malware Genome Project" that morning. It turned up three responses -- all of which were on NC State pages. I pushed it out via social media (Twitter, Facebook, G+, Reddit, Slashdot, Ars Technica forums, etc.) and did some targeted pitching to mobile security beat writers. Three weeks later, a Google search turned up more than 300,000 hits. The word was out, and the researcher was fielding requests for additional information on the project from potential collaborators, etc. It was a very specific communications goal, and the metric for success was the number of collaborators and potential collaborators that contacted the researcher.

Coming up with good metrics requires some critical thinking. Don’t rely solely on the easy analytics, like pageviews. Spend some time and mental energy to figure out what you really want out of your blog, and then spend some more time and mental energy to come up with meaningful ways to determine whether you’re getting it.


3 Responses to “Unconventional Metrics: How Can I Tell If My Blog Is Working?”

  1. David Wescott Reply | Permalink

    really interesting. Of all the metrics you mention, the one associated with goal two strikes me as the strongest - you can show a reasonably high probability that funders saw what you're doing.

    in the for-profit world I'm constantly trying to find meaningful metrics as well, and it remains an enormous challenge. Like your world, social media is just a part of a larger, integrated communications strategy, hopefully linked to outcomes.

    To me social media is used to open a channel of communication to a specific person or well-defined group with the goal of influencing them. That way the metric question is "did they respond?" The unit of measurement I like to use most is the relationship. Can we now reach a person we've determined is influential (a funder, a consumer with a well-read column or blog, a regulator, a legislator, a client, etc) to the point where we know they'll take our content seriously and respond accordingly? To that end, on occasion I've written blog posts designed to reach a single person.

    When we start thinking about social media as a form of broadcast media, that's when I think metrics break down. Generally speaking the number of "likes" a brand has on Facebook has virtually no connection with changes in rate of sales. The YouTube videos that have the most views are decidedly NOT influential when it comes to changing attitudes - you could probably count the number of videos that have truly moved opinion on a broad scale on one hand.

    the hard part for me is justifying the investment to a client. without more traditional metrics and arguments for ROI, it's hard for the bottom-line driven private sector to commit scarce resources over channels that have demonstrated effectiveness such as lobbying or advertising.

  2. Matt Shipman Reply | Permalink

    Agreed. I feel one of the most common social media mistakes is over-valuing "likes" or "+1s." Whether you are promoting a field of study, the findings of a specific research paper or a line of cleaning products, you are probably not moving significantly closer to your end goals by getting someone to "like" a post on Facebook (for example). The message I'm hoping to get across here is that it is important to bring some critical thinking skills to bear on what your goals actually are, and finding some (perhaps creative/unconventional) means of measuring progress toward those goals.

Leave a Reply


1 × = seven