Wednesday, May 25, 2016

Kantian Origins Of Peircean Frequentism

Immanuel Kant

Charles Sanders Peirce was an early "evolutionary" philosopher. He believed that while our knowledge was now imperfect, correct science would - as a whole - learn to reduce those imperfections. He was a serious student of German Idealism (famously, he studied philosophy by reading one page of Critique of Pure Reason a day). He also helped found statistics, experimental psychology, modern logic and much else. Today, I want to look into how his interest in philosophy and statistics cross-bred.

CS Peirce

How much of an evolutionary philosopher was Peirce? He went so far as to define "truth" as the outcome of an ideal scientific process. For instance, imagine we didn't know Peirce's first name. We could look it up in a book, you say. That's the best scientific practice, therefore that's the truth. Let's be more extreme. Say that, for some reason, direct records of his first name had been lost. At first we would only know that his name is in a certain set. By our knowledge of human language we know that his name isn't "Hmxfrzt". By historical considerations we can eliminate "Cao Pei" and "Christina". Through long search and careful philological textual criticism, eventually we figure out it was probably "Charles". Therefore, it is true that Peirce's first name was "Charles".

This is eccentric because we normally think of "Charles" as being Peirce's first name because of actions done in the past (namely, his being named by his father), not because it is an outcome of actions of philologists of the future. This definition will even have an important effect in his statistical prescriptions.

Peirce's definition didn't come out of thin air. To recapitulate: On the one hand, he was an experimental scientist inspired by his work in physics, psychology, etc. On the other hand, he was a serious Kant-inspired philosopher. In particular, Kant's image of the sensible world of experience and the unknowable world of things-in-themselves was an inspiration to Peirce as a statistician. The world we can see, hear, smell, taste & feel is called the "phenomenal world" (as in, it's where phenomena occur), the deeper underlying world is called the "noumenal world" (we'll get to why in the next paragraph).

How do we gain knowledge of the noumenal world? Remember that this is the old days, before some young Germans questioned Newton & Euclid. So most people believed we did have knowledge of the underlying world-in-itself. Kant did not. Kant believed that the phenomenal world was basically psychological and sociological. Human beings evolved to perceive the abstract world-in-itself in Newtonian/Euclidean ways, he thought. We - our society - adopted conventions constrained by those evolved capacities. This mode of thought was further developed by Schopenhauer and I've covered it on this blog before.

Peirce (and, earlier, Hegel) disagreed with Kant. They hoped that perception of the things-in-themselves would turn out to be solid and objective rather than subjective and biological. Hegel defined truth as the outcome of a long social process - one which, unfortunately, only existed in his mind. Peirce defined, as we saw above, as the outcome of a convergent scientific process - processes that he then went out and tried to do.

In Peirce's theory, the real world-in-itself is a set of interacting (possibly/often non-measurable) facts and relations between these facts. These facts can be constants, such as the 19 parameters of the Standard Model, or they can be variables, such as the total population of a country or temperature. These facts can be basic, like energy, or "emergent", like temperature. That underlying world could only be approximated sadly phenomenal studies. Therefore, even crafty experiments surrounded the true values (of, say, the fine structure constant) with error bars. Pierce called these error bars the "probable error", today we call the equivalent notion "confidence interval". Peirce's work is, in many ways, the beginning of statistics.

Peirce first developed his statistical ideas when studying the experimental errors of using pendulums to study the acceleration due to gravity, but it is equally valid to consider coin-flips. The facts of a given sequence of coin-flips are statistically related to the underlying reality of governing the coin. In the case of coin-flips, we can appeal to Bernoulli's theorem to prove that the scientific best practice leads to The Truth, the coin-in-itself.

This is a mathematical version of the general example I gave above, when we learned Peirce's first name. You then might again notice that Peirce's definition of truth is eccentric. Mathematically, one must posit a true value and prove convergence toward it. I think Peirce would reply that this is a mathematical convenience and the truth was the reverse, a coin is known as fair from the throwing. Peirce developed this definition in scientifically relevant ways. For instance, he would say that Bayesian methods are not scientifically relevant unless paired with a robust convergence proof. One can construct instances in which a Bayesian procedure does not converge. From Peirce's point of view, this would mean that for such agents, the truth is meaningless.

So we see how philosophy affected statistics. Peirce's forward looking definition of truth ruled out Bayesianism, his love of Kant made Frequentism attractive. Notice that these are logically quite separate!

All this would have been by itself interesting, but Peirce actually went further. He gave a specific quantitative guide to such reason in his "Note on the Theory of Economy of Research". The essence of Peirce's reasoning is here. Peirce's discovery is even more remarkable because not only did he notice the parallel with the ratio of marginal utility - he also did so in 1879, making him the among the first important American Marginal theorists of any kind!

Given the importance of Marginalism in his thought, one should not be surprised when he says: "The truth is a kind of efficiency.". Surely someone who could say that can be called a pragmatist.

Though Peirce had a chance to become one of the great economists of his time, he didn't take it up. In addition to the above, he was also the first to state the axiom of transitivity of preferences (he had to be - he also invented relational algebra). Interestingly for the proto-frequentist, he was also the first to measure systemically subjective probabilities and among the first to rigorously define probability in terms of economic decisions. Unfortunately, he rarely took the time to find deeper implications of his economic thoughts (the above being the only exception to this rule). Certainly, his rival Simon Newcomb (interestingly, the rivalry, while well-attested, was unknown to Peirce...) would not have appreciated it.

Karl Marx

All that brings me to the next 19th century philosopher/economist to explain: Karl Marx.

No comments:

Post a Comment