I am continuing my blogging series on why grad school sucks.  This series has only had one entry so far, in which I talk about how bad physics talks are, and how this worsens impostor syndrome.  Today I will talk about how scientific success is based on luck.

If you have ever read any popularizations of science, you’ve likely heard that many scientific discoveries are made by serendipity.  This makes sense, because if a discovery isn’t a big surprise, then it’s not much of a discovery, is it?

We have one of these stories in the field of superconductivity too.  Kamerlingh Onnes is credited with the discovery of superconductivity in 1911.  But that’s not what his work was really about.  His real accomplishment was being the first person to liquefy helium.  He just tried cooling a bunch of things, and that’s how superconductivity was discovered.  That’s serendipity!  Kinda?

The thing is, serendipitous discoveries might make for a fun story, but it’s garbage to actually live through.  If you go to grad school, will you hit upon something truly interesting?  Or will you just produce a bunch of unremarkable studies that nobody cares about?  Nobody knows!  But your career success depends on it!

Thankfully, serendipitous discoveries don’t play quite so large a role as popular accounts would have you believe.  First of all, scientific progress doesn’t really revolve around discoveries.  A lot of time, we’re just measuring some quantity we’d like to know, and it doesn’t particularly matter what the result is as long as we know it.  We also spend a lot of time developing new ideas and techniques, or applying old techniques in new contexts.  This is the sort of thing that doesn’t depend on luck, but rather skill, talent, and hard work.  (Skill and talent also depend on luck, but I suppose they don’t feel bad in the same way that luck does.)

Even when it comes to big discoveries, there are plenty of things we can do to improve our chances.  With scientific wisdom, we can recognize search spaces that are likely to turn up something new and interesting.  And with hard work, we can explore as large a search space as possible.  Kamerlingh Onnes may have been lucky to have found something as big as superconductivity, but he was the one who put himself in the right place and the right time.  He had the wisdom to try cooling a bunch of materials to low temperatures and measure their properties.

Still, even if one’s scientific work does not revolve around big discoveries, one often hopes that the results of an experiment end up one way, rather than another.  For example, suppose I observe some unusual phenomenon, and I perform some followup experiments to diagnose it.  One possible result is that the unusual phenomenon has deep implications for scientists who study superconductors.  Another possible result is that the unusual phenomenon is the result of an instrumental error, which has deep implications only for scientists who use similar instruments.  Another possible result is that the results are ambiguous, and I have no clue.  Depending on the results, I may waste more or less time, publish in a more or less prestigious journal, or not publish at all.

You might ask, doesn’t that bias the interpretation of scientific data?  Hell yes.

As for “scientific wisdom”, the big problem is that new grad students don’t really have it.  For the most part, you have to rely on the scientific wisdom of your advisor, the one who selects your scientific project.  So for grad students, there’s a huge luck factor arising from how good your advisor is at choosing projects.  Also, how good your advisor is at maintaining funding.  And I suppose it must be a burden for advisors too, to carry that responsibility.

As I said, thankfully it’s not all a matter of luck–skill and hard work also play important roles.  But in a way, that can be worse.  Because now, if I attain a lesser degree of success, there’s the question: Am I unlucky, or am I just bad at science?  If I’m already suffering from impostor syndrome, I might take a guess.  And my future employers could take a guess as well.

By the way, the amount of luck involved… I’m not saying this is inherently worse than any other occupation, or any other aspect of life.  The thesis was “grad school sucks,” not “grad school sucks worse than everything else”.  Regardless of whether grad school is any worse than any other occupation, it seems that the least we can do is, stop celebrating all these damn stories about serendipitous discoveries!

(Disclaimer: None of this series should be taken as a reflection of my personal experience with grad school, even when I use myself in rhetorical examples.  Some of it is based on personal experience but some of it is based on other people’s experiences, or on speculation.)