Sigman’s cherry-picking problem

9 October 2012 by Pete Etchells, posted in Uncategorized

Aric Sigman is back in the news today, with a number of media outlets picking up that he has written a review for the Archives of Disease in Childhood on the need to set limits on the amount of time children spend watching TV and playing video games. This is a clearly a hugely important issue - as I've noted before, there is a good deal of evidence to suggest that hours spent watching television are correlated with negative health outcomes, such as excess weight (Hancox & Poulton, 2006), decreased hip bone mineral density in girls (Janz et al., 2001), as well as being a predictor of weight gain in adulthood (Ekelund et al., 2006). However, I don't think this latest piece by Sigman is all that it's cracked up to be.

In the review, Sigman discusses physical health and obesity issues, brain and cognition, and psychosocial health in the context of screen time, backing up his arguments with 83 references. And therein lies a problem. A quick search on PubMed for terms including "television", "screen time", "obesity" or "psychosocial health" pulls up over 1,200 articles. Expanding it to include “social media”, “cognition”, “psychology” and “psychiatry” pulls up over 3,000 hits. In short, the research literature on this area is huge. Sigman's paper discusses less than 1% of it (plus, it's worth noting that some of his references are to newspaper and media articles, the TV licensing website, and the Nintendo website). So the worry, then, is that this review is not entirely representative of what we currently know about how technology use affects us and affects children. Now, obviously, it's a little unreasonable to ask someone to cover thousands of research articles in a single, 6-page review. One thing you can do, though, is look for reviews and meta-analyses - and to be fair, there are some in this review. One is an oft-cited meta-analysis of 72 studies on empathy, which found that empathy levels among some 14000 American college students dropped from 1979 to 2009. The authors of this study speculate as to the potential causes of this drop, and one of them is the rise in technology use. Unfortunately, speculation is just that; no effects of technology were directly assessed in this study, so we don't really know the actual cause of the empathy drop. Other reviews include one showing an association between television viewing and unhealthy dietary behaviours, a systematic review of how to reduce screen time in young children, and ,er, the effects of smokefree legislation.

This is all well and good, but paints the slightly erroneous picture that there's a consensus in the scientific community about what state we're in, and what technology use is doing to us. For example, a systematic review by Marshall and colleagues in 2006 suggested that the amount of TV that children watch hasn't changed in some 50 years, and conclude by suggesting that the role of TV and video game use in the aetology of childhood obesity might be overstated. A recent systematic review in the Archives of Pediatric and Adolescent Medicine looked at active video game usage (i.e. using things like the Nintendo Wii and Microsoft Kinect) actually promoting light-to-moderate physical activity in children. Others, such as this review of the effects of television viewing on preschoolers, acknowledge that the experimental evidence we have on such behavioural effects is still in its infancy (pardon the pun), and by no means do we have a clear idea of what's going on.

As has been noted by others before, the problem with cherry-picking data to suit your argument is that it sends out all sort of incorrect messages, not just about the state of the nation, but about what actually might work to correct the problem you're interested in. Sigman devotes an entire section of his review to what other countries are doing in terms of issuing guidelines about technology use in children, and chastises the British and European medical establishments for not having any. The trouble is, a paper in the International Journal of Behavioural Nutrition and Physical Activity in 2010 points out that there isn't a consistent association across geographical locations between the number of hours of sedentary screen-based activity and the levels of physical activity. In other words, legislation that might work in one country won't necessarily work in another.

So, rather than issue recommendations about limiting screen time and developing (costly) awareness initiatives directed at parents, day care centres, health visitors and schools, as Sigman's review suggests, what we should do first is to develop research programmes based here in the UK and Europe to try and understand precisely what the effects are of screen time on our children. Only then can we really figure out what the best sort of initiatives might be to promote healthy lifestyles in the young. A blanket ban on screen time that ignores the positive effects of newer, more active technologies, might actually hinder that process.

At the very least, fuelling scaremongering headlines in the media off the back of a single review article doesn't seem like a particularly objective or impartial way to progress science or promote public health, and we should be more sceptical of those who engage in such behaviour.


11 Responses to “Sigman’s cherry-picking problem”

  1. Dorothy Bishop Reply | Permalink

    I'm frankly astounded that a reputable journal from the BMJ stable published this as a 'leading article'. Surely a leading article with implications for policy should be written by someone with strong academic credentials, and if it is reviewing the literature, then it should adopt standards of a systematic review. It's hard to find a CV for Sigman on the web. He did a PhD on hypnotic self-regulation at North East London Polytechnic in 1987. His only entries on Web of Science are two abstracts based on the thesis, published in 1985 and never cited. He does not appear to have any academic or clinical position and it is unclear if he has any qualifications in child health. His main claims to fame are books and reviews in The Biologist, all of which adopt the approach of assembling evidence demonstrating the adverse effects of various aspects of modern life on children: daycare, drinking, and television, and speculating about how these may affect brain development. He does not have a good track record for impartiality or accuracy, e.g.: http://www.guardian.co.uk/commentisfree/2011/sep/14/daycare-cortisol-levels-children
    Sigman writes on important topics and he is entitled to his views. But like Pete, I am disturbed that he has been given a platform by Archives of Disease in Childhood. This can only add to the view that he is some kind of expert - as well as
    allowing him to increase his H-index from zero to one.

  2. Dr Aust Reply | Permalink

    Not on Sigman, but on the general theme of serious journals and/or of eminent organisations giving people a platform for their pet hobby-horses, try this

  3. Pete Etchells Reply | Permalink

    Thanks both for the comments. Dorothy, in regards to your comments, one thing that I find particularly troubling about Sigman's past form is a statement that he released in response to criticisms about one of his Biologist pieces. You can find the original statement on Sigman's website here, where he states "I selected studies that were generally negative precisely because most biologists and policy makers are unaware of the existence of these studies." I've written to the ADC to express concerns that he has similarly selected studies for this current piece. Will update as and when I get a response.

  4. Cary Bazalgette Reply | Permalink

    I was delighted to find your blog post on Sigman, and Dr Bishop's response. My blog (toddlersandtv.blogspot.co.uk) has also looked at him: I'm very depressed about the quarter-million-plus Google results on Sigman's article, but cheered by the much more sensible parental responses in social media.

  5. Saadi Haeri Reply | Permalink

    This article assumes Sigman is alone in saying these things. France has already banned TV aimed at under 3s and the US government advises against screen viewing for under 3s. Australians have a government guidline saying under 2s should watch no TV and Canadians have similar health department guideline. The American Academy of Pediatrics have not based their recommendation that no child under 3 watches any screens on Sigman's work. They have probably never heard of him as he is based in the UK. The UK seems to be gunning for this individual but the issue will not go away if he is eliminated from the equation.

  6. Saadi Haer Reply | Permalink

    When it comes to the health of infants it is better to err on the side of caution. Is your approach the same as the man who did not know whether he needed a nail or a screw and decided to drive a nail in to see if the planks splits?
    China banned gaming consoles in 2000 solely on health fears for the young and would you say they and these Koreans have been cherry picking with Sigman?
    http://www.bbc.co.uk/news/world-asia-pacific-14361420

    • Pete Etchells Reply | Permalink

      There's a difference between erring on the side of caution, and scaremongering. If I told you that stepladders cause cancer, then by your logic we should ban them all, because it's the 'safe' option - regardless of whether there is any substance to my claim. My point is that it's easy to make something look obviously bad (or good) if you selectively pick evidence to suit your needs. That's not helpful to anyone involved.

  7. Natalie Doherty Reply | Permalink

    Thanks for this refreshing blog. I'm trying to write a literature review and come up with a research question on whether television viewing influences language development in children under 2 for a postgrad course and I'm so sick of Sigman's media profile but lack of good research behind what he says. Any tips appreciated. Cant even find viewing stats for this age group in UK.

  8. SPX Reply | Permalink

    To contact "On the Road," or to send us a story idea,Shawn "Jay-Z" Carter will release his highly anticipated new album "Magna Carta Holy Grail" in early July of 2013.

  9. Anonymous Reply | Permalink

    She complained that key positions within the security forces were politicized, rekindling old rivalries among factions from the civil war era.

Leave a Reply