Perspective on cannabis dependence and IQ
Reports from various news outlets have been discussing a paper published a couple of days ago in PNAS. The paper investigates cannabis dependence, and it’s relation to change in IQ. Media articles have interpreted these findings as ‘cannabis use is harmful to adolescent brains, but not afterwards’. Now, before I go on, I should say that I’m not disputing that this may be the case, but I don’t think this paper provides as strong evidence as is being reported, for a number of reasons, which I’ll go through here.
1. Sample size. OK, 1000 people sounds like a lot, but in terms of observational epidemiology, it’s not massive. This study had 5 levels of cannabis use, 3 of which had at least some form of dependence. Although cannabis use is quite common, cannabis dependence is not, and so the number of people in the highest cannabis use categories are very small, 35 and 38 in the top two. So although 1000 sounds like a lot, it can mean very small numbers in each category. Indeed, later in the paper, when the authors compare those dependent by age 18 with those dependent after age 18, they are looking at groups of less than 15 people in some cases.
Also, when looking at the sample they report, it looks like their analyses are only done on 874 of those 1037 people they mention in the abstract. But I can’t see any mention of why this would be.
2. Cannabis dependence. As I mention above, although cannabis use is quite common, cannabis dependence isn’t. The smaller a sample size, the less representative they are of the population at large. This can result in a larger standard deviation, or wider confidence intervals (measures which represent the spread of likely ‘true’ underlying values assessed by a sample). 55% of the whole sample is in their second category, ‘used cannabis, never diagnosed dependent’. A further 28% of the sample have never used cannabis at all, leaving only 17% of the sample who have ever been diagnosed as cannabis dependent. These 153 people are then further divided in to 3 dependence categories. Since most people are in the ‘used, not dependent’ category, to me it would make more sense to divide this category up a little, as there’s a lot of variation in recreational use before a person would reach dependence which is lost.
Of course, this may not have been possible. One of the joys of observational data is that by the time you come to use it, the data has been collected years previously, so you don’t have any say over what was asked. However, looking at previous work from the Dunedin cohort using cannabis use data, other papers have used measures of frequency of cannabis use (none/less than monthly/less than weekly/weekly), whether they had ever used cannabis by age 15 or 18, or early versus late onset of cannabis use at ages 21 and 26 (defined as when they first said they’d used cannabis at least monthly)! It is strange, when using the same exposure measure (in this case cannabis use) to vary the cutoff points. Looking at different ages of use can be explained depending on the outcome perhaps, but these studies define ‘use’ differently, one as ‘ever’ and another as ‘at least monthly’. I don’t know why they did this, but I would be really interested to see whether their results from each study would change with different cannabis cutoff points.
3. Adolescence. While I think this paper’s method of looking at IQ well before cannabis use starts (age 13) is great, that their first cannabis measure is at age 18 puzzles me, as they refer frequently to ‘adolescent’ cannabis use. Adolescence is a somewhat nebulous term, and there is some evidence that our brains continue to develop up to our 20s, but reports in the media about this paper have been somewhat misleading. Cannabis dependence before age 17 is not assessed, and past year cannabis dependence assessed at age 18 is compared with cannabis dependence afterwards, which includes their interview at age 21. So if the brain develops up to the early 20s, perhaps they should have compared dependence before 21 to after, rather than picking 18 as the cutoff.
4. IQ. Even though IQ is not a perfect tool for measuring intelligence, using it may have been the only option. But in these data, IQ differs quite a lot at baseline, depending on the type of cannabis user the person will become in the future! The difference between the highest and lowest IQ group pre cannabis use is 6 points, the same as the largest change seen over time (in the persistently cannabis dependent group). Although the authors attempt to account for this by looking at individual change, perhaps the reason that people of differing IQs go on to have different cannabis use patterns, and different decreases in IQ over time is important. Which brings us to:
5. Confounding. One of observational epidemiology’s chief limitations is lack of randomisation. People who choose to use cannabis, and choose to use it to excess are different to people who don’t. So maybe one of these differences is causing the difference in IQ, not the cannabis use per se. The authors chose to conduct a number of analyses where they remove groups of people, such as those dependent on tobacco or what they call ‘hard drugs’, or those with schizophrenia. But there are some potential confounders that they don’t consider. For example Professor Val Curran, quoted on a Nature blog, mentions depression, which could be a cause or an effect of cannabis use, and can lead to decreased motivation, which could affect IQ performance.
6. Statistics. Finally, I find the choice of statistics in this paper a little puzzling. Rather than reporting effect sizes and confidence intervals, they report t-tests (without stating the degrees of freedom) and p-values. Now, p-values should always come with a health warning, but as the authors conduct so many different sub-analyses of their data, their t-tests should ideally be corrected to account for this, as the more analyses you do, the more likely you are to find an effect by chance, rather than one that truly exists. I can't see any evidence that this has been done, although they may have just not reported it. Also, although their change scores hint at an effect size, it would be great to get more of an idea of the likelihood that if the population were tested again the same behaviour would again be observed, standard errors or confidence intervals for example would really make things clearer, and help when comparing one group to another. Although the effect sizes might be different, in small samples, the variance may be so large that confidence intervals would overlap, meaning there is no statistical difference. It doesn't look like that is happening here, but without confidence intervals, it's hard to tell.
I hope this doesn’t come across as too negative about this paper, which I think has important findings about cannabis’ association with IQ, but I think that these findings are not representative of the relationships between cannabis and IQ in most people, only in those few who use to excess, and also not really adolescents, but those aged 17 and over. I’d love to hear other people’s thoughts on the paper, and the media reporting of it.