Perspective on cannabis dependence and IQ

29 August 2012 by Suzi Gage, posted in Uncategorized

Reports from various news outlets have been discussing a paper published a couple of days ago in PNAS. The paper investigates cannabis dependence, and it’s relation to change in IQ. Media articles have interpreted these findings as ‘cannabis use is harmful to adolescent brains, but not afterwards’. Now, before I go on, I should say that I’m not disputing that this may be the case, but I don’t think this paper provides as strong evidence as is being reported, for a number of reasons, which I’ll go through here.

1. Sample size. OK, 1000 people sounds like a lot, but in terms of observational epidemiology, it’s not massive. This study had 5 levels of cannabis use, 3 of which had at least some form of dependence. Although cannabis use is quite common, cannabis dependence is not, and so the number of people in the highest cannabis use categories are very small, 35 and 38 in the top two. So although 1000 sounds like a lot, it can mean very small numbers in each category. Indeed, later in the paper, when the authors compare those dependent by age 18 with those dependent after age 18, they are looking at groups of less than 15 people in some cases.

Also, when looking at the sample they report, it looks like their analyses are only done on 874 of those 1037 people they mention in the abstract. But I can’t see any mention of why this would be.

2. Cannabis dependence. As I mention above, although cannabis use is quite common, cannabis dependence isn’t. The smaller a sample size, the less representative they are of the population at large. This can result in a larger standard deviation, or wider confidence intervals (measures which represent the spread of likely ‘true’ underlying values assessed by a sample).  55% of the whole sample is in their second category, ‘used cannabis, never diagnosed dependent’. A further 28% of the sample have never used cannabis at all, leaving only 17% of the sample who have ever been diagnosed as cannabis dependent. These 153 people are then further divided in to 3 dependence categories. Since most people are in the ‘used, not dependent’ category, to me it would make more sense to divide this category up a little, as there’s a lot of variation in recreational use before a person would reach dependence which is lost.

Of course, this may not have been possible. One of the joys of observational data is that by the time you come to use it, the data has been collected years previously, so you don’t have any say over what was asked. However, looking at previous work from the Dunedin cohort using cannabis use data, other papers have used measures of frequency of cannabis use (none/less than monthly/less than weekly/weekly), whether they had ever used cannabis by age 15 or 18, or early versus late onset of cannabis use at ages 21 and 26 (defined as when they first said they’d used cannabis at least monthly)! It is strange, when using the same exposure measure (in this case cannabis use) to vary the cutoff points. Looking at different ages of use can be explained depending on the outcome perhaps, but these studies define ‘use’ differently, one as ‘ever’ and another as ‘at least monthly’. I don’t know why they did this, but I would be really interested to see whether their results from each study would change with different cannabis cutoff points.

3. Adolescence. While I think this paper’s method of looking at IQ well before cannabis use starts (age 13) is great, that their first cannabis measure is at age 18 puzzles me, as they refer frequently to ‘adolescent’ cannabis use. Adolescence is a somewhat nebulous term, and there is some evidence that our brains continue to develop up to our 20s, but reports in the media about this paper have been somewhat misleading. Cannabis dependence before age 17 is not assessed, and past year cannabis dependence assessed at age 18 is compared with cannabis dependence afterwards, which includes their interview at age 21. So if the brain develops up to the early 20s, perhaps they should have compared dependence before 21 to after, rather than picking 18 as the cutoff.

4. IQ. Even though IQ is not a perfect tool for measuring intelligence, using it may have been the only option. But in these data, IQ differs quite a lot at baseline, depending on the type of cannabis user the person will become in the future! The difference between the highest and lowest IQ group pre cannabis use is 6 points, the same as the largest change seen over time (in the persistently cannabis dependent group). Although the authors attempt to account for this by looking at individual change, perhaps the reason that people of differing IQs go on to have different cannabis use patterns, and different decreases in IQ over time is important. Which brings us to:

5. Confounding. One of observational epidemiology’s chief limitations is lack of randomisation. People who choose to use cannabis, and choose to use it to excess are different to people who don’t. So maybe one of these differences is causing the difference in IQ, not the cannabis use per se. The authors chose to conduct a number of analyses where they remove groups of people, such as those dependent on tobacco or what they call ‘hard drugs’, or those with schizophrenia. But there are some potential confounders that they don’t consider. For example Professor Val Curran, quoted on a Nature blog, mentions depression, which could be a cause or an effect of cannabis use, and can lead to decreased motivation, which could affect IQ performance.

6. Statistics. Finally, I find the choice of statistics in this paper a little puzzling. Rather than reporting effect sizes and confidence intervals, they report t-tests (without stating the degrees of freedom) and p-values. Now, p-values should always come with a health warning, but as the authors conduct so many different sub-analyses of their data, their t-tests should ideally be corrected to account for this, as the more analyses you do, the more likely you are to find an effect by chance, rather than one that truly exists. I can't see any evidence that this has been done, although they may have just not reported it. Also, although their change scores hint at an effect size, it would be great to get more of an idea of the likelihood that if the population were tested again the same behaviour would again be observed, standard errors or confidence intervals for example would really make things clearer, and help when comparing one group to another. Although the effect sizes might be different, in small samples, the variance may be so large that confidence intervals would overlap, meaning there is no statistical difference. It doesn't look like that is happening here, but without confidence intervals, it's hard to tell.

 

I hope this doesn’t come across as too negative about this paper, which I think has important findings about cannabis’ association with IQ, but I think that these findings are not representative of the relationships between cannabis and IQ in most people, only in those few who use to excess, and also not really adolescents, but those aged 17 and over. I’d love to hear other people’s thoughts on the paper, and the media reporting of it.


48 Responses to “Perspective on cannabis dependence and IQ”

  1. Alex Stevens Reply | Permalink

    Interesting post. What did you think of the claimed linear dose-response relationship in table 2?

  2. Suzi Gage Reply | Permalink

    I think it's interesting, but this only controls for sex, no other confounders whatsoever (ie not years in education, although they do consider this later). I'd also be keen to see confidence intervals for the effect. Also, I'd much rather there was a more nuanced idea of 'dose' of cannabis, rather than lumping all the 'used but not dependent' together. What do you think? I read with interest your tweets about this paper yesterday.

  3. Alex Stevens Reply | Permalink

    Agree with you on confidence intervals. I would also have liked to have seen the results reported separately for adult and 'adolescent' never-dependent users to check Moffitt's reported comment that cannabis is risky for the adolescent brain but safe for the adult brain.

  4. Neil Davies Reply | Permalink

    Hi Suzi, excellent post.

    I agree with all of your points, although I'm a little more skeptical about whether this paper provides any evidence of the causal effects of cannabis on IQ.

    I think the paper overstates the strength of the evidence they present.

    "We ruled out six alternative explanations for the observed effects of persistent cannabis use on neuropsychological functioning, namely that these effects could be explained by (i) past 24-h cannabis use, (ii) past-week cannabis use, (iii) persistent tobacco dependence, (iv) persistent hard-drug dependence, (v) persistent alcohol dependence, and (vi) schizophrenia."

    To rule out these alternative explanations they reran the results excluding each of the six groups above. This did not change their basic associations.

    However, this doesn’t prove that these factors don’t explain their results. For example, if any of the six factors suffered from measurement error, then these factors could still explain their results. Similarly the fact that the association with cannabis still remains "significant" after adjustment for these factors does not provide much reassurance that residual confounding is not an issue.

    Similarly their education results show that cannabis use is very strongly associated with education. From Table 4, the odds-ratio on 3+ diagnoses and having a high-school education or less is 5.0, and 4.2 for 2 or more diagnoses. (It would have been really good to present the associations of the exposures and potential confounders).

    Education increases IQ, cannabis use may cause or be associated with the amount of education the participants achieved. But education is a complex experience, and it seems unlikely that the categorical measures used in this paper fully can account for all the differences between their participants' educational experiences.

    Formally they assume:

    E[C | IQ_baseline, education, cannabis use] - E[C | IQ_baseline, education, no cannabis use]=0.

    Where C is all other potential confounders.

    So for their results to reflect the causal effects of cannabis use, they must assume that there are no other meaningful differences associated with IQ, between their participants who used cannabis and those who did not, aside from their measures of education.

    Which is equivalent to assuming conditional on IQ at baseline and education, their participants were effectively randomized to their cannabis use.

    This is clearly not a tenable assumption.

    This applies across all their outcomes, so the fact the association is evident across a number of measures does not provide much reassurance either. It would have been interesting to see if cannabis was associated with an outcome that we don’t expect, e.g. BMI, blood pressure or height at 38?

    The paper discusses the "effects" of cannabis. Yet this paper can only provide evidence of the association of cannabis and IQ.

    Pearl (1998) "Why there is no statistical test for confounding, why many think there is, and why they are almost right" is a really good reference for this.

    But the press reported it pretty faithfully, the paper made strong claims on the basis of weak data. I don't think we can blame the press for accurately reporting their paper.

    Finding out how the world works is hard, really hard, but the first step is to recognize the limitations of the data you work with.

    It's important to be realistic about what we can and cannot prove for any given data.

    Although saying that this is basically the argument RA Fisher made about smoking and lung cancer in 1957.....

  5. Dhogan Reply | Permalink

    You realize they looked retrospectively and thus could not time the points where they measured IQ right? Perhaps that is why it baffles you. Didn't bother to read the rest of your article with such a basic misunderstanding of the literature.

    In addition, they did not do an analysis on all the participants because many had missing data - pretty clearly stated in the article. Back to stats class.

    • casualty Reply | Permalink

      Then you missed the fact that she is pretty much spot on in her critique of the worth of such a limited and narrow study lacking in proper controls to even begin to understand the causality of the decline in IQ testing performance. The broad claims being reported are not scientifically sound.

    • Suzi Gage Reply | Permalink

      Hi Dhogan. Thanks for your comment, although I'm afraid I disagree. Dunedin is a prospective cohort, meaning measures are taken as the children grow up, rather than retrospectively collecting a group of 38 year olds and asking them what their IQ was back when they were 13 (of course they won't know). If by 'retrospective' you mean the authors of this study couldn't choose when to ask IQ, then of course I agree. I'm pretty sure I make that point in my article!

      As for observations being lost due to missing data, I agree this is most likely what happened, but it is good practise in an article to report the sample size for your analyses, as well as the whole dataset. For example, I work with ALSPAC (a birth cohort based in Bristol UK where I live). The full dataset is over 14,000, but due to missing data the sample sizes I work with are always smaller. If I was writing a paper I would include both the full sample and the sample I actually used.

  6. casualty Reply | Permalink

    Very nice break down. The report is likely being spun at request of the ONDCP as part of their strategy to curb adolescent drug abuse. They are the same government arm that brought us the famous "This is your brain on drugs egg in the frying pan commercials.) The study was funded by NIDA and they have a congressional mandate to support research that specifically is searching for detrimental effects associated with the substances on the scheduling list. It's unfortunate that the system is designed this way.

    • Suzi Gage Reply | Permalink

      Thanks very much for your comment. I don't think that the NZ Government have anything to do with the reporting though. I personally am funded by a UK research council, but they in no way interfere with my work, and I would be very surprised (and disappointed) if that happened over in NZ either. Indeed, the lead author is based at Kings in London, so she's unlikely to be under pressure from NZ Government. Maybe you think I'm naive, but I really hope science is independent of the Government funding it receives. I would certainly step away from my work if I felt I was being pushed in any direction by a funder. Maybe it's getting more press attention because it agrees with policy, but I think a paper like this would always get a lot of attention...

      • casualty Reply | Permalink

        I was referring to the U.S. government, the agencies I mentioned are by law mandated to oppose any efforts to legalize schedule 1 substances in the United States and often work closely with popular media outlets here. We have 3 legalization measures coming up in the November elections. The National Institute on Drug Abuse was one of the entities that funded this study as they must only fund studies that aim to legitimize the illegality of these controlled substances. Scientific evidence that is generally lacking for most of the drugs was apparently not a concern when the Controlled Substances Act was formulated as well as the UN treaties that demanded all countries that sign must follow suit. For instance NIDA commissioned a study comparing the dangers of tobacco smoking with cannabis smoking in the hopes that it would provide the conclusive evidence that cannabis also causes lung cancer and emphysema (2006 Tashkin.) Much to their dismay not only was no evidence found to corroborate that hypothesis instead the research concluded that cannabis provided a protective effect against the development of tumors. NIDA cut off all funding to Dr. Tashkins work after he published the findings of the 30 year study. This is just one example of many.
        Australia and NZ are pretty notorious for publishing borderline yellow journalism with regards to cannabis research.. they recently put out an article claiming conclusive evidence of permanent brain damage directly caused by cannabis based of a small study..
        http://www.medicaldaily.com/articles/11417/20120809/marijuana-brain-damage-memory-learning-drug-habit-addiction.htm
        This report was completely ignored by U.S. media yet the new study we are discussing that was backed by U.S. interests was plastered everywhere. It is odd to say the least. Perhaps it's all just a random thing..who knows hehe.

        • casualty Reply | Permalink

          I should also mention that I don't blame the researchers for conducting this work as it is important and I know how difficult it is to obtain grants nowadays, my father is the Chairman of Immunology at a prominent U.S. research institute. Drug war analysis is a bit of a hobby of mine so please excuse me for inundating your blog with all this information that may not interest you in the slightest, my apologies.

  7. Mondragon Reply | Permalink

    Speaking as someone who abstained until I was 20, but then found that Cannabinoids were absolutely my substance of choice, to the point that I'm still hard at it, still enjoying it, yet am as intellectually capable as ever I was (2:1 from ancient university etc), I was never as dumb as when I was in my heavy drinking phase frm 15 until 20/21, even my curiosity was blunted by booze, and I remain eternally thankful that I turned to the Green Side. I also think that if Legalisation never happens, which I think it will, that musicians should get special dope smoking dispensation, as should all artists. How the hell are we going to be shamanic channelers of psychic energies (man) if we can't tune into them properly.

    Rambling Lunatic out

      • Mondragon Reply | Permalink

        As an Art Historian/Library worker hard science data makes my brain bleed, nevertheless I thoroughly approve of it. I think that Cannabinoids clearly need as much empirical, non-judgemental study as possible, something prohibition impedes to say the least. Keep up the analysin' and stuff...;)

  8. casualty Reply | Permalink

    The NHS published this conclusion about the study.

    Conclusion

    Overall, this study provided some evidence to support the growing literature on the potential harm of cannabis, particularly among adolescents.

    The most important limitation is that, despite the author’s efforts to adjust for confounders, it is always possible that other factors (for example, socioeconomic factors or other unmeasured mental health issues) influenced the results and were underlying the apparent association. It is important to note that this research does not prove that there is a direct causal link (that is, teenage cannabis use leads to an IQ decline) only that there is an association.

    It’s also worth noting that this research undertook extensive statistical calculations to look at the relationship between different measures of IQ and different duration of cannabis use, some of which involved only small sample sizes. For example, despite the large initial sample size (1,037), only 41 people (3.95% of the people surveyed) used cannabis regularly at all three time points. Calculations based on such small sample size decreases the reliability of these risk associations.

    Another issue is whether cannabis use was accurately recorded. Interestingly only seven participants reported trying cannabis by age 13, and cannabis use during adolescence was only accounted for in the assessment at 18 years where participants were asked about use in the previous year, that is, while they were 17. It is also possible that participants did not accurately report their cannabis use patterns in the year before each of the assessments, which can make the results less reliable.

    The fact that informants were also asked to assess participants on things such as memory and attention span may also make the results somewhat unreliable as people’s personal opinions are by definition highly subjective.

    Validation of cannabis use using laboratory measures would have made the results more reliable. However, persuading ‘dope-smoking 20-somethings’ to attend regular blood tests may be somewhat challenging.

    Another factor is that the potency of the cannabis smoked was not reported. There has been a growing trend of people smoking stronger strains of cannabis (such as skunk) over the course of the last few decades.

    So if there is a dose-dependent effect between cannabis use and IQ impairment then the effect in today’s teenagers could be even more pronounced.

    The researchers note that further research is needed on the effects of the quantity, frequency and the age-of-onset of cannabis use on neuropsychological impairment.

    The full longer effects of cannabis use are not completely known, but in the shorter term cannabis can cause variable psychological effects, which may differ extensively between individuals.

    Despite these limitations this is a useful piece of research that adds to a growing body of work that suggests persistent cannabis use at a young age can have a negative impact on mental health and mental functioning.

  9. Stephan Schleim Reply | Permalink

    When I was a teenager, it was not uncommon among my friends to smoke some cannabis occasionally. There were some, I think, who rather used it as a self-treatment to regulate emotion, to sleep better, etc. I would consider this a "benefit" and, compared to alcohol, for example, none of us was ever aggressive when "stoned" – the most notable bothersome side-effect might have been telling the same jokes over and over again.

    Most of us stopped around the age 16-18, including myself. The most serious concern I have looking back at this time was that we had to buy the stuff from rather untrustworthy foreigners standing in the dark corners of certain streets. They could have sold us everything. I wonder whether the authors of this study took into consideration that the putative loss of intelligence might have been due to other toxic compounds contained in impure samples of street drugs.

    It's no secret that cannabis can increase one's risk of certain psychological problems – but so can any psychopharmacological substance, too (just read the packet insert; for example, a couple of US psychiatrists voiced their concerns that several drugs increase the risk of obesity which, in turn, shortens life-spans of people). Some of my (former) friends who kept consuming it for a very long time themselves say that they have the impression that it impaired their memory.

    I have seen a couple of papers based on this NZ dataset that are interpreted in the context of current political debates, e.g. juvenile aggression, and found the conclusions and particularly the media reports of them annoying. So I am grateful that Suzi took the time to read this paper thoroughly (which I didn't) and share her analysis with us.

    Suzi, you wrote in the comments that if you were paid by some government to do research in support of a certain political agenda, you would stop with it. But what do you think about the current publishing system in general? Aren't there explicit and implicit pressures to interpret your findings in such a way that they appear immediately relevant to certain social ends, such as developing therapies?

  10. Suzi Gage Reply | Permalink

    Hi casualty!
    Thanks for the comments - that NHS article is really nice, thanks for sharing. And I am very interested in your comments about funding in the US, I hadn't heard this claim before, I will certainly look in to it. I particularly stressed that Governments don't (or shouldn't) have sway over what research is conducted as I have had run-ins with 'tobacco apologists' (for want of a better term) who accuse researchers of an 'anti tobacco' agenda. I work with people doing research in this area, and so I know that they would publish whatever they found, good or bad. They don't set out to 'prove' tobacco is bad, they set out to look for evidence. Of course there's a hypothesis, but hypotheses aren't always correct, and as scientists we should publish either way.

    Oops, perhaps I'm rambling myself a bit now!

  11. Suzi Gage Reply | Permalink

    Stephan, thanks for your comment. As for your question...well...
    Perhaps I am naive, but I think there is a marked difference between (for example) making comments in a discussion about potential implications of the work, and something like writing a paper using 'causal' language (ie the effects of x upon y) throughout, when the data is observational and causality cannot be assumed. I would hope that journals would not encourage exaggeration of findings for a media splash, and I personally have not experienced this being the case, but I'm still a relative newcomer to first author publication. Certainly superior colleagues I have worked with have taught me that such behaviour would be negative, rather than positive, for my research career. Perhaps I am lucky to have had this experience. Are you a researcher, and have you found this to be the case? Or has anyone else perusing the comments section found this, I'd be really interested to know...!

  12. Alex Brown Reply | Permalink

    Hi Suzi,

    Without having read the paper, nor in fact seen any media coverage of it (am based outside the UK these days - the influence of national agendas on media content would be a whole different conversation), here are some thoughts:

    - it turns out that epidemiology is hard.
    - your dissection is very clear and well written.
    - as is often the case (and as you mention), the paper no doubt contains some useful information, but the extrapolated claims seem unjustified
    - my interest would be in finding out what control the authors had over any PR assoicated with the paper, although ones dares to hope they were alrgely in control of their own writing and interpretations within the paper itself, and not too heavily directed by funders, institutions or journals/reviewers. have they published other, related work? how does that stack up against this?

    Alex

    • Suzi Gage Reply | Permalink

      That's really interesting that there wasn't much media coverage outside the UK...

      • Alex Brown Reply | Permalink

        Well, there might have been, but I haven't seen it here (France/Switzerland). It's a common phenomenon that PR, including science PR, sits within International borders. The MMR vaccine is only a "problem" in the UK, USA and Australia as far as I know. In other countries, such as France, no-one has heard of it being "contentious", although there are other fears instead, also largely influenced by the media.

      • Stephan Schleim Reply | Permalink

        There was a lot of coverage in German (internet) media, including articles on the title pages of major news portals with huge pictures showing people smoking cannabis or close-ups of joints etc. (see, e.g. Langzeitstudie: Jahrelanges Kiffen lässt IQ sinken, Long-Term-Study: Long lasting blowing lets your IQ decline). The general message always was that consuming a lot of cannabis in your adolescence makes you stupid.

        I have also watched a video presented on one of these portals which covered some original footage of the NZ researchers and culminated in the interview of a long-bearded stoned (English-speaking) hippie expressing his concerns that the study might be misinterpreted as "cannabis blows your brain" (Cannabis-Langzeitstudie: Dümmer durch Dope, Cannabis long-term-study: More stupid through dope).

        (Some meta information: Spiegel Online is one of the most visited websites in German language and had almost 200 million visitors and 1 billion hits in January 2012.)

        • Suzi Gage Reply | Permalink

          Thanks for this information. I think a project about the differential reporting of media health scares/stories across countries would be really interesting. Maybe that's something for post-PhD fun...!

  13. Suzi Gage Reply | Permalink

    Of course it's not surprising that this is the case, but it's something I have never considered before. Would make really interesting research, not to mention a good way to dissuade people from 'believing the hype' to point out it's entirely media driven...

  14. jasmine Reply | Permalink

    Thank you to the author of this article. Ever since the Heath Tulan Study of 1974 we know there is a lot of interest invested in proving that Cannabis kills brain cells. In light of that fact, I find it only healthy to approach any such claims with skepticism. This study in question seems to have no real control and is rather vague, but it did come just in time to put a dent into the legalize movement around the world…coincidence ?

    • Suzi Gage Reply | Permalink

      Thanks for commenting Jasmine! The study looked at people who had never used cannabis versus people who used but weren't dependent versus those who were dependent, so I think it was fairly well controlled (as well as observational studies can be). As for the timing, I believe although I'm not 100% certain that the cohort only recently had the data on IQ collected, so that is why it's appeared at this time.

  15. James Little Reply | Permalink

    Nice analysis. The NHS conclusion suggests that todays cannabis is more "potent" than that of decades ago, and that this might cause further harm in todays youth. There is an assumption that higher levels of cannabinoids (almost always measured as THC level) per weight leads to people ingesting more cannabinoids, which I'm not sure is valid. Smokers usually self administer until they achieve the desired effect, rather than smoking a specific quantity. Perhaps the most significant change in cannabis over the last few decades has been in the ratio of cannabioids, form around 1:1 THC:CBD in 80s hash to 20:1 THC:CBD in todays skunk type strains, which are less sedating and more mind bending at any potency. Cannabis is not a specific drug, its a non standardized natural product containing hundreds of different cannabinoids, each of which affect the human body in different ways. To further complicate things it looks like CBD has anti addictive properties e.g. http://www.jneurosci.org/content/29/47/14764.full.pdf and that people smoke less if their cannabis has more CBD in it, so a lack of CBD could lead to higher consumption of THC rather than the presence of more THC in a sample.

    • Suzi Gage Reply | Permalink

      Thanks James. I think you're absolutely right. As far as I'm aware, there's hardly any research in to whether cannabis users 'titrate the dose' in the same way that smokers do for nicotine levels. A friend of mine is just about to start a 4 year fellowship investigating just that! So watch this space... ! Disentangling the effects of CBD versus THC is very tricky.

Leave a Reply