Epidemiological Studies are not Experiments

A simple study cannot prove anything

March 27, 2021

So you may be familiar with my general view of the modern scientific community and that I believe that the you should approach the stuff they put out there with skepticism until you've spent some time looking into it and thinking about it yourself, we should never blindly follow anyone. But this does not mean that I don't have faith in the scientific method, I believe that if used correctly the scientific method can find the answers to many (not all) important questions. This is not something that can be said of epidemiological studies.

Much of the content of this post is inspired by this article that I read a month or two ago, I won't be going into nearly as much detail as this article does so I would recommend reading it if you want to learn more.

Epidemiology's origin

Epidemiology is the study of epidemics, many say that its father was Dr. John Snow who in the mid 1800s tracked down the source of a London cholera outbreak. He studied what areas and people were getting cholera and after his investigation he determined that the source of the outbreak was likely the Broad Street water pump, so he had the city close the water pump and the cholera outbreak soon ended. Since then epidemiology has become a major field of study and is being used to find the source of more than just small epidemics.

The problem with Epidemiology

When people talk about this story, or really any other story involving a success in the field of epidemiology they forget an important detail. An Epidemiological study can't prove anything, it is only a tool to form better hypothesises to be used in the scientific method. The scientific method has two basic steps, form a hypothesis, then perform an experiment to prove the validity of that hypothesis, No hypothesis is to be taken seriously until it is proven in an experiment. Dr. Snow's hypothesis about the Broad Street pump was proven right through the experiment of closing it down, not by whatever methods he used to form that hypothesis. Snow could have been wrong about the pump (luckily he wasn't) and if the cholera had continued to spread the doctors of London would have had to continue searching for the source of the epidemic.

People give too much credit to the field of epidemiology, now this is not to say that when it comes to outbreaks of infectious diseases they aren't to be trusted, epidemiologists are usually quite accurate when they are working with those sorts of things since that is what epidemiological studies were designed for. But what happens when an epidemiological study is used for something else? Consider these headlines:

News articles with these sorts of headlines are almost always ones which report on an epidemiological study, the problem with these is that things like your life expectancy, your brain health, and your likeliness to contract heart disease have way more variables to deal with than a simple viral or bacterial outbreak does. Epidemiology cannot save you from diabetes, at best it can give you some good advice regarding diabetes. And when you look into the way these studies are carried out in most cases no reasonable person would put too much confidence in them, how can an annual survey of 40 year old women asking them how much fruit they ate six months prior logically have anything to do with their memory when they are 65?

Another important thing to notice with these studies is the fact that they can serve as indirect advertisements. Two of the three headlines I made up above say that a certain product can help you. How would your opinion of the headline claiming wine can help you live longer change if you learned that the study was funded by one of the world's largest wine producers? That whole article becomes a lot less credible doesn't it?

These articles are also often things that are designed to make people feel good about themselves. Notice how again two out of the three things that I write there hare things that are easy for people to do and generally things that people like doing anyways. Self improvement is almost always hard these headlines exist to encourage you not to do the hard things like waking up earlier and quitting alcohol that can have an incredibly positive impact on your life.

Learn to recognize the difference between an epidemiological study and an actual experiment. Don't be fooled into thinking a hypothesis is a conclusion.

* See the end not on this article for advice on how to approach the saying "reduce your risk" and other similar statements.