How many flips of a coin coming up heads in a row would it take for you to believe the coin is a trick coin? Two? Ten? 1,000? Our prior belief that any given coin is fair is quite strong but it is not invulnerable to evidence. After say, 15 consecutive heads, the probability we would give to the theory the coin is a trick coin would have raised significantly, but after 1,000 consecutive heads we would have virtually no doubt the coin was a trick coin simply because evidence accumulates. Indeed, after seeing 1,000 consecutive heads seeing the 1,001 toss come up tails shouldn’t suddenly convince you that the coin is fair.
This seemingly tedious example highlights our intuitive grasp of the common dictum extraordinary claims require extraordinary evidence, which is not merely a common phrase but normatively correct. This, for example, is why you believe a friend who says they missed the bus but don’t believe the same friend when they say they slayed a dragon. The kind of evidence is the same, testimony of a friend, and in both cases the friend could be mistaken or lying, but the prior probability you assign to someone missing a bus and someone slaying a dragon are vastly different (and rightly so).
Of course I say seemingly tedious example because though we surely apply this rule regularly there are places where this rule is apparently unwelcome.
Bring up this coin tossing example when discussing telepathy or miracles and you may be regaled with tales of how someone’s personal experience with their favorite belief eliminates the need to consider what we think we know about physics or history. Nevermind the fact they are simply assuming they could not have been mistaken, personal experience simply becomes less important in areas where there is a huge data set. It is true everyone has different experiences but as the data set gets larger our prior beliefs should have less and less bearing on our conclusion. The fact that our prior probabilities of any given coin being fair may have been substantially different matters far more after 15 tosses than it does after 1000.
If you think that 1000 is large, even ignoring the theoretical problems, how large do you imagine the set is of humans failing to demonstrate telepathy or walk on water? To call those sets massive would be an understatement. Likewise in order to overturn our conviction these feats aren’t possible we would need a colossal amount of evidence and surely, as in the case of your friend the dragon slayer, personal testimony doesn’t suffice.
You can’t escape Bayesian reasoning simply by claiming your priors are different, unless of course where you are going you don’t need rules.
Update 2/6/13: Removed anomalous indication of a footnote that didn’t exist