Hempel's Paradox

Hempel's Paradox is yet another problem with induction. Hume taught us that much of what we consider to be induction is fallible and probabilistic, rather than certain. But this raises a whole host of difficulties, difficulties that Hume - who lacked training in mathematics and statistics - dimly perceived. Statisticians from CS Peirce to Bruno de Finetti have tried to firmly grasp what Hume offered, and found much to disagree with each other even within this. Bruno de Finetti was a Bayesian - perhaps the Bayesian. In the frequentist/error statistics tradition from CS Peirce to Deborah Mayo, have attempted to use probability in a way that has been called ampliative, they amplify our knowledge. Deborah Mayo is explicit on this point, a severe test teaches us something new. There is another tradition running from Frank Ramsey to de Finetti and through to modern Bayesian theory, that probability is a form of logic - and therefore not ampliative. I don't mean that probability is founded on logic - nobody doubts that probability is a branch of mathematics and statistics is applied mathematics. What is at stake is what statistics could teach us even in theory. The question "Is probability a form of logic?".

C Hempel

Carl Hempel added a new wrinkle to this debate with his raven paradox. How would you go about testing the proposition "All ravens are black."? Look for an albino raven? Save your time, there's a much easier way. Obviously, this is equivalent to testing the proposition "If something is a raven, then it is black.". This is equivalent to saying "If something is not black then it isn't a raven.". So, look at your shoes. They're not black and not ravens. So that's some evidence. And hey, your fingernails aren't black either, are they? Thinking about it, how many van Gogh paintings aren't black? How many dots in a Seurat painting? The world is mostly evidence that there are no white ravens!

There are, of course, solutions for those who hold, with de Finetti, that probability is a form of logic. The technique is to use the size of the sets involved. There are so many non-black non-raven things in this world that the weight of observing one is low. That is, we do learn that there are no black ravens by drinking white milk, but it is weak evidential milk. This is a bit counter-intuitive, since the eye color of Chuck Berry's eyes seems to be completely unrelated to ornithology. In addition, this response gives rise to odd ducks (perhaps odd ravens...), like Laplace's argument about the probability of the sun rise.

This argument is popular with those who accept a Bayesian view of probability as something more than a sometimes useful tool. What is the frequentist/error statistic point of view? It might seem extreme, but it actually makes good sense. Observations of ravens gives us an estimate of the ratio of black ravens to the count of ravens. Obviously, this ratio will be near one. But what the frequentist test gives is error bars, confidence intervals. These confidence intervals can overlap, be useless or shrink around one. In the last case, the frequentist/error statistician will converge on popular opinion. But what of the non-white non-raven? Since a frequentist does not accept that tests are closed under logical operations, they don't accept them as tests. They are only interested in tests of the object in question.

In my view, this makes error statistics less expressive, but less paradoxical than Bayesian statistics. Which you need depends on the practical situation, and so I say let ten thousand flowers bloom. Incidentally, the philsopher Willard Quine also wrote on this subject. Quine's solution was to claim that non-black things didn't form a natural kind. This is worse than accepting the paradox, since it will soon mean natural kinds aren't closed under any logical operation, yet we're supposed to use logical analysis and probability theory to learn... Good luck writing a classifier with that in mind!