What are your weaknesses as a scientist? What should we do when we become aware of those weaknesses, or when someone makes us aware?
Backstory: Several months ago I started doing Crossfit here in France1. It’s been a fun way to meet new people and get some exercise. It also lets me work out one muscle that I really need to improve: my French skills. Of course all of the coaching and instruction happens in French, with the result that I am sometimes only 80% sure I understand a complicated movement. Fortunately there is an English reference text handy – with a bonus amazing title: Becoming a Supple Leopard2 by Dr. Kelly Starrett:
So the other day I picked it up and was learning about the jerk press when this paragraph jumped out at me:
What hit me immediately is how different this is from the training that we usually get in science. In this short paragraph, Dr Starrett is telling his readers that hiding their weaknesses is not doing them any favours in the long run. If you hold yourself back from fixing what you know is imperfect, you’re going to end up “compensating into bad positions” and soon you’ll be “failing at the margins of your experience”.
Sort of the opposite of a scientist, in many ways. Many of us hide our weaknesses, and insist that we are strong “through a range of motion” (i.e. from the start to the end of our scientific process). I don’t think anyone just sits down grad students and tells them this, but I think we do absorb it over our training. For example, there is the delicate art of deflecting criticism and comments after a paper presentation: “No, we haven’t tried that analysis, but I don’t think it would change the answer because..”. Or if somebody says that an experiment is poorly controlled or badly replicated, the first response is never “how could I fix it?” but “Yeah but, I don’t think that control would actually matter because …”. I don’t think that anyone is ever taught this behaviour explicitly, but I’d guess we’ve all seen it in seminars, in thesis defenses, and at conferences. I also wonder if we do this in writing, too, when we frame our reasoning in a vague way – in part to deflect attention from arguments that are not very sound.
Maybe weakness-hiding is most reflexive, and most challenging to correct, when we’re doing statistics. Biologists are implicitly taught to figure out all the ways a model is right , rather than learning how it is wrong. Residual diagnostic plots, for example, are a very handy tool – but they’ll only tell you part of the truth. Lots of models might produce flattering quantile-quantile plots, but that doesn’t mean they’re all going to be wrong in the same way! And all models are wrong somewhere, just as every athlete has something they need to improve upon somewhere.
Do scientists face some of the same consequences as athletes? Do we also “compensate into bad positions” or “fail at the margins of our experience”? I feel like not to the same degree. Science is a competition too, but it’s an extremely multivariate one: scientists can “compete” for funding and jobs while doing activities which have nothing in common – at least not on the surface. In other words, it is perfectly possible for an ecologist to keep comfortably to the middle of their comfort zone, seldom or never exploring the edge of their intellectual range of motion. I think it is also fair to say that the incentive system in science encourages some doubtful practices. We might sometimes have bad form, but that’s because we’re playing a broken game.
But I think it sometimes does come back to punish us. How many grad students have fruitlessly built research projects on past work from Dr. Supervisor’s lab, only to find the fundamental result didn’t replicate? How many bugs in R code have flipped the results in published figures? And how many times do we repeat or retread work that was done decades ago, accidentally ignoring whole sections of the literature?
I’m not saying I’m perfect! Far from it, I believe all these errors are present in me and in my work. My hope is that, in my work as a scientist (and maybe one day as a mentor to other younger scientists) I can adopt an attitude that is closer to weightlifting than to a scientist. You’re guaranteed to have weaknesses and imperfections. Don’t fear them, and certainly don’t hide them! Don’t involve your ego and your pride, they just get in the way. When you notice (or when someone reminds you of) your scientific weaknesses: park your ego, get hungry, and go after them.