Bayesian Reasoning: What Counts as Evidence?
[Continuing my look at Bayesian reasoning. My previous explanation of Bayes’ theorem.]
Perhaps the most fundamental question underlying Bayes’ theorem is what counts as evidence for a theory. Strangely there is a correct answer to this question which applies not just to Bayes’ theorem but all evidence: something is evidence for a theory if and only if it was more likely to observe that evidence if that theory is true as opposed to the likelihood of observing that evidence if it was false.
Admittedly that is a lot of verbiage but let’s take an example: Suppose I am inside my house one morning and I have the hypothesis that it snowed overnight. Considering I’d watched a weather report yesterday that said there was a 50% chance of getting the first snow of the season overnight (and that such reports had been more or less accurate in the past) I think there is a 50% chance that it snowed. How can you settle this, what would count as evidence that it snowed? I can look outside and see if there is any snow on my porch. I can even go outside and take samples from the tops of different surfaces, checking them for snow. Suppose I do these things and it turns out that I can see snow outside and that when I go outside I am able to capture some snow. Seeing the snow should raise my probability that it snowed last night and likewise physically taking samples should do the same.
However note that this shouldn’t bring your probability up to 100% for the hypothesis that it snowed as unbeknownst to you someone could have dumped that snow on your font porch while you were sleeping. Also there is the minute, but real, chance that you were hallucinating both when you saw the snow and when you believing you were physically picking it up. Or even more unlikely, unbeknownst to you someone could have physically moved your house to somewhere with snow already on the ground. There is always a some chance that the data you observe could be explained by some other hypothesis but so long as the data you observe is more likely if your hypothesis is true rather than if your hypothesis was false, you are justified in considering that data as evidence for your hypothesis.
All of this is intuitively obvious and it’s equally obvious that observing, for example, that there were cars outside has no bearing on whether or not it snowed last night. That’s simply irrelevant because it is unaffected by whether or not it snowed. Bayes’ theorem just allows us to quantify the effect of these observations, whether they be for, against or irrelevant to competing hypotheses. However Bayesianism does something which is not so intuitive, explicitly consider there to be evidence which counts against every theory. If I don’t observe snow outside this counts against the hypothesis it snowed last night.
One fundamental problem with conspiracy theories is that conspiracists refuse to accept that there is evidence that makes their theory less likely—the absence of evidence of my theory proves my theory!—but if nothing counts against a theory nothing counts for it either. Despite the popularity of the opposite phrase, absence of evidence really is evidence of absence. The belief that it isn’t is a symptom of black and white thinking. To see why just imagine someone who believes it snowed last night coming outside to find no trace of snow. While this doesn’t absolutely disprove the idea, there’s always the chance it snowed but all traces of snow are now gone, it should (greatly) reduce the confidence in the belief. There’s a much greater chance to observe no snow on the ground because it didn’t snow than to observe no snow on the ground if it did snow. Or imagine a prosecutor trying this reasoning out with a jury: Just because I don’t have the evidence that would be expected if the defendant committed the crime that doesn’t count against my claim that he did commit the crime.
The fundamental mistake being made is asking yourself “Is this possibly consistent with my theory?” which only rules out the impossible, when we should be asking “Does this observation make my theory more likely?” which leads to hypotheses which are more likely than others. Using this type of Bayesian reasoning helps us escape from yes/no dichotomies to the more accurate range of probabilities that should make up our beliefs and perhaps most importantly it helps us realize when our hypotheses don’t fit the data.
