Don’t “Wait for More Data”
There’s a conventional wisdom which has infected our collective mind, specifically in the political punditocracy, which warns us to “wait for more data” before making a decision, or even worse, before changing our mind. There are multiple ways to take such advice but I don’t see any of them as useful.
The most charitable of the possible interpretations is merely a reminder to not let a small amount of evidence for (or against) a claim outweigh a large amount of evidence to the contrary. According to this the prudent thinker is just waiting for more data in the sense that they don’t let a surprising, but limited, new finding completely undermine their confidence in well established claims. However even taken this way the claim seems to be either a terrible way to make a point or poor advice. It is undoubtedly true that we shouldn’t give undue consideration to small amounts of evidence but if that is what is meant, this latter statement would be a much less ambiguous way to make this point. Contrarily, it also true that we should update our beliefs as the data comes in but that includes all of the evidence to date, making the desire to wait for more data an unambiguously wrong way to update your beliefs. It’s true that there are some claims so strong, like your the confidence in the truth of gravity, that small amounts of conflicting evidence will make no practical difference but that still misses the point that you should update your beliefs as the evidence comes in, even if it doesn’t make a substantial difference at the moment.
This is more apparent in a less charitable interpretation of “waiting for more data” or in situations in which your initial position is held with less confidence. In a less charitable take on the old advice the admonishment to wait for more data amounts to a refusal to update your beliefs until there is overwhelming evidence you are wrong. This seems to be little more than an excuse for expressing undue confidence in beliefs right up until they are almost certainly false. Similarly if we aren’t talking about claims like those of basic physics, those which haven’t been proven to be true in countless observations, but instead contested issues or much less clear fields then the advice to wait for more data is a prescription for almost never changing your mind. There just isn’t likely to be the kind of overwhelming evidence in questions of public policy (and to a lesser extent economics), where you routinely here this type of rhetoric, that there is the so-called “hard sciences.” So the demand for more data can easily amount to a self-justification to stick with unwarranted levels of confidence in beliefs.
If your goal is to align your beliefs with reality then it makes no sense to ignore all the relevant data until the evidence is of such force that you have to move from very confident in the truth of a proposition to the opposite. It could be said this desire to wait for more data stems from our need to make statements of certainty in a world of ever-shifting probabilities. However, being too slow to detect a trend isn’t a sign of good judgment but is equally as bad as moving too quickly. “Over-adjust to the evidence as it comes in” would be seen as obviously bad advice and, I think, its counterpart in “wait for more data” should be equally discarded.
