The inevitable return to TV this week of Masterchef, after a close season shorter even than the English Premier League, has (for strange reasons that I hope nonetheless will become clear) triggered this response of sorts to Brian McGill’s post on Statistical machismo over at Dynamic Ecology last year. Brian lamented the use by ecologists of the latest ‘must use’ statistical method, which is typically complicated both to perform and (perhaps especially) to interpret, without necessarily having much of an effect on the conclusions drawn. He felt this macho posturing – as he puts it, “my paper is better because I used tougher statistics”; in Masterchef terms, “analysis doesn’t get any tougher than this” – ends up overcomplicating papers and wasting everyone’s time. I enjoyed the post at the time, and felt it raised some interesting points; and though I disagreed with the thrust of it, this was not to the extent that I felt compelled to comment, still less to respond. Now that I’ve come up with a convoluted, almost certainly over-played culinary analogy, though, I’m going to have a bash at expressing my thoughts on the matter properly.
If you watch Masterchef (especially the early rounds) you’ll probably see a great deal of culinary machismo. Even if you don’t, you probably know what I mean: food prepared by someone who is a decent chef, but a pretty awful cook. Smears of jus and droplets of fluid gel on big white plates, but the chicken’s raw; burnt chips in a flowerpot; spun sugar on a duff dessert. Contrast this with what a good non-cheffy cook might produce: a really excellent, well seasoned, ugly stew; a pudding that tastes sublime but looks like a car crash. When I lived in Thornton le Clay near York, our pub specialised in the latter: fantastic, simple, pub food, cooked to perfection with no pretension (it's unfair on them to suggest it was ugly, but the emphasis was on flavour not prettiness). Next village we lived, the pub was very gastro, and the food – though twice the price, and served on wooden boards as likely as not – was nowhere near as good.
This, I think (bear with me!), is similar to the issue that Brian raises. In particular, the use of advanced techniques – statistical or culinary – without having mastered the basics, indeed without even considering the basics, reeks of posturing. In these cases, I agree, we should beware.
Consider for example something like Generalised Linear Mixed Effects Models (GLMMs) as a statistical equivalent of nitro-poached aperitifs or popping candy cheesecake. I am very wary of GLMMs. Ben Boelker’s TREE paper on them basically says as much: do not go here unless you really know what you’re doing. As a minimum, you ought to have mastered the basic component techniques of GLMs and LMMs (and naturally, you need to know your LMs for either of them). Yet I see students who describe themselves as ‘not very confident’ at statistics merrily fitting GLMMs with no clear idea of what model they’ve fitted, or why. Not machismo in this case, but rather blindly following a statistical recipe which demands a great deal more skill than their current aptitude allows.
So yes, in these kinds of cases – and similarly in some of the others Brian mentions – doing a simple analysis well is probably preferable to making a dogs dinner out of a complicated one.
And yet…
Let’s stretch this analogy further. If you really want perfect chips, you’ll triple cook them. Liquid nitrogen really does make excellent ice cream. The way to ensure your meat is exactly à point every time is to cook it sous vide in a water bath. Simply put: some methods of cooking are better than others, and if you can master a Blumenthal-esque skillset, the resulting food will be objectively, qualitatively superior to the lovely, hearty stuff I used to eat in my local, or that I aim to cook at home.
In the same way, some methods of statistical analysis are simply better than others. Brian’s post mentions phylogenetic correction, for example, complaining that it hardly ever affects the result of an analysis, yet entails a great deal of work and additional assumptions. Well perhaps (and his point about errors in phylogenies is a good one), and of course you can fluke the ‘correct’ result with simple statistics, just as you can fluke excellent food with a less scientific approach than that employed by the molecular gastronomists. But if you want consistent excellence – if you want to do something right – you use the best available methods.
Specifically regarding the inclusion of phylogenies in comparative analyses, it’s largely immaterial in my view whether or not this has a large effect on your results; rather it’s simply sensible to consider evolutionary processes when you're modelling a pattern which is the result of evolution. This point is nicely made in a new paper in Methods in Ecology and Evolution by Hernández et al., in which they make a plea for moving beyond phylogenies as ‘statistical fix’ (i.e., ‘phylogenetic correction’) and embracing instead a fully evolutionary view of macroecology in which we test mechanistic hypotheses rather than just describing patterns. (One could of course make a similar case for including spatial processes.)
The cooking/statistics analogy breaks down in one important aspect, however: there are very good reasons why you might not even attempt to master those fancy cooking skills. I read the Fat Duck cookbook much as I might read an account of the building of a great cathedral: full of admiration for the skill, craftmanship and effort involved, but with no intention of even attempting to replicate the endeavour. Blumethal’s Pot roast loin of pork, braised belly, gratin of truffled macaroni, for instance, includes 74 incredients, including two separate stocks (a further 24 ingredients and several hours of prep time), and requires nine separate procedures to produce a single course. You (or at least, I) would never do that to feed two at home; it is only feasible at a restaurant scale. Even those recipes that look technically manageable need expensive equipment, putting them well out the reach of the home cook, who might be better advised to concentrate on mastering more simple skills.
Developing beyond being a good ‘home statistician’ – mastering the essentials of analysis – on the other hand, requires none of this expense. Unlike haute cuisine, mastering statistics – especially in the age of R – is free. We have no excuse not to master the best available methods. So you maybe should roll up your sleeves and chase that Michelin (Fisher? Pearson? Gaussian?) Star after all. Not because you feel you have to in order to show off – I’m with Brian there – but because doing things right is important.