Gun Control and the Post Hoc Ergo Propter Hoc Fallacy

It always irks me when I read an article about the supposed effect of a policy change, say a gun ban in the UK or Australia, by looking at the effect on some variable (gun deaths, mass shootings, etc.) after the legislation was passed. The problem is that you can mistakenly conclude a causal relationship between the policy and its impact when none exists. It’s the post hoc ergo propter hoc fallacy, after this therefore because of this.

As an Economics major, I learned that you still need a control and treatment group when dealing with time series, just like you would with cross-sectional data. Other majors, I’m sure, were taught the same thing. We’re familiar with the example of a drug trial, for example, where you find two groups of people that are identical in almost every way and give one group the drug (say, to reduce blood pressure). Ideally, if you see the blood pressure levels drop in the treatment group then you’re confident it’s the result of the drug.

But what about when we’re looking at something like gun deaths or mass shootings?

Well, that’s time series data, a measurement over time rather than a cross-section at a particular point in time. Taking the example of a gun ban, we want to know the effect that change in policy on a variable measured over time (e.g., gun deaths).

Here, the treatment is the policy change itself, just like the blood pressure drug in the previous example. But your “control group” is the time period before the policy changed. In fact, one of the first things a researcher wants to do is look at the data and see if there is an apparent change at the time the policy is enacted.

In the chart above, we can see that workplace deaths steadily decreased over time. If you ignored that and looked at what happened after OSHA was passed you’d mistakenly conclude that the law was successful at increasing workplace safety because, well, deaths declined afterwards. In reality, the law was passed after a trend was well underway.

So, whenever you read an article that reads, “after law X was enacted, variable Y changed” in a way that suggests it was because of the law, the first question you should ask is: what was happening before law X was passed? In the case of Australia, which is often cited as an example of where a gun ban worked, gun-related homicides began a downward trend in 1969, well before the ban was enacted in 1996. You can see how pervasive the fallacy is because most articles you’ll see focus on what happened to the level of gun deaths after a ban was enacted.