Foundations and Other Unneccessary Things

The economist John Maynard Keynes once said "Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back.". Practical men, to paraphrase, are usually under the spell of popular philosophy. A few weeks back I did a post on Wittgenstein's criticism of the logicist program. I concentrated on a technical aspect, he pointed out that the interpretation of quantification over infinite sets is left open (that is, there are multiple models for given sets of axioms), therefore the alleged foundations of mathematics don't specify a specific mathematical language. Modern mathematicians admit this, but don't care. I didn't go into as much detail about a stronger, but more philosophical, criticism. Principa Mathematica , Die Grundlagen der Arithmetik, etc claimed to be the foundations of mathematics, but if we found an error in them (and an error was found in Grundlagen), then we would dispose of the book and not mathematics. In other words, in practice, there is nothing special about axioms that make them "below" theorems. Mathematics, and Wittgenstein argues science and even more life in general, is more like a hyperbolic tower where everything leans on everything else than an inverted pyramid where everything leans on the bottom stone. I bring up Keynes because I realize now that there is no way to read this and not be affected. I may or may not be a Wittgensteinian, but he has affected how I see things in a fundamental way. I must keep this in mind when I enter into "foundational" controversies.

Shukhov Tower

After my Jaynes post, I did a bit of re-reading of his big book. What is the value of Cox's Theorem? What makes it superior to the usual Kolmogorov Axioms? To the extent that Cox and Kolmogorov disagree, so much the worse for Cox (as far as I can tell). Kolmogorov's axioms are deliberately vague as to interpretation. They are models for statements about normalized mass or subjective valuations of probability. Cox's theorem is no shorter or more intuitive. I don't think that the interpretation that the functional equations are about subjective degrees of belief is any more suggested than in the Kolmogorov axioms (that is, it isn't at all). Why? We can interpret f to be "the sand in this unit bucket outside this set", then recognize that being outside the outside is being inside, etc. Therefore, Cox isn't any better a foundations for subjective probability than Kolmogorov.

Azazoth

Cox's theorem isn't strong enough to constrain countable unions, which means that if it was The Real foundation of probability then it would run into strange problems. As I said in the Wittgenstein post, mathematicians like to deal with the infinite by making it as much like the finite as we can without risking contradiction. Countable additivity is a way of doing this. If you have half a bucket of sand and half a bucket of sand, then you have (half plus half equals) one bucket of sand. That's additivity in a nutshell. But if you add up infinitesimal (that is, limits of smaller and smaller scoops of) grains of sand (in a limiting procedure), what happens? In countable additivity, you get a bucket of sand - lucky you. In finite additivity, the answer isn't defined. There's no reason to think that you wont add up bits of sand and get Azazoth. In other words, you give up the ability to compute probabilities.

The problems are even worse for the Bayesian, because finite additivity isn't consistent with conditionalization (hat tip: A Fine Theorem). Since finite additivity is all Cox's Theorem gives you, clearly it needs to be made more robust! (Unlike, say Kolmogorov's Axioms) Obviously, I strongly disagree with that paper's thesis that de Finetti gave "compelling reasons" to abandon countable additivity, and regard de Finetti's examples of "intuitive priors" as bizarre. (Also, I find A Fine Theorem's Kevin Bryan's arguments even weaker. It isn't obvious to me that his hostile description of frequentist consistency is induction in any sense, much less a bad one...). The famous Bayesian Jaynes must have at least sensed this, because he was always combatitively pro-countable unions. But is his foundation for himself a castle built on sand? The answer is obvious to me, Jaynes just never cared about such things, thought it was a merely technical problem without deep import to general theory (he says in the appendix that the only difference between his approach and Kolmogorov's was that Kolmogorov took an infinity first method and him an infinity last).

Dr Fine, Dr Howard and Dr Howard in deep philosophical debate

This issue might be worth maintaining low level controversy about it, and Kolmogorov put it in the right place - as a questionable but reasonable assumption. An "axiom" as we mathematicians say. Sure, countable additivity is so useful and clearly correct in so many contexts that giving it up seems like giving up your legs. But science is multithreaded being, and intellectual controversy often ends in clarification. But in the Cox framework, finite additivity isn't a theorem, it's just a quirk of not constraining our function enough. It just doesn't feel like enough to me, it seems to me that if Kolmogorov, Doob et al were wrong they must be wrong in a much deeper way. Anyway, that's enough about countable probability.

As I said from the outset, it seems obvious to me that axioms are philosophical matters and arguing about them gets you into nothing but a Wittgensteinian language game. But there are differences between Kolmogorov and Cox about finite additivity (and whether functional equations are more intuitive than measure theory). So maybe there is some, small content there. Therefore, I will now e-beg for answers. Tell me about the wonders of Cox's Theorem, internet! I'm all ears!

Quick Review: Phoenix Wright - Ace Attorney

Well, you can't spend all your time working hard. Even the sick, dying and people in dire poverty have to entertain themselves. Even though I have a plethora of projects, a few ideas that need exploration to become projects and of course there's all the time I ought to be improving myself in some way. Let me share with you some of what I spend that time doing:

The Ace Attorney series is a set of video games, members of a genre called visual novels. A visual novel is sort of like the old choose-your-own-adventure books or, more directly, adventure games where the main form of puzzle is choosing dialog. I don't have much experience with this genre of games, in fact this series is basically the only ones I've played. Even then, I've only played the core part of the series the Phoenix Wright trilogy. From the little bit of research I did, it seems that Ace Attorney is a bit more like an adventure game in that there are many puzzles other than choosing dialog. The idea is that a visual novel will make up for its reduced emphasis on traditional video game design to tell a more compelling story. Again, I haven't played much, but I expect that the quality of the stories varies wildly.

P Mason

In essence, these are murder mystery stories where you are cast in the role of the great detective yourself. If you've seen or read Perry Mason, you'll find the basic goals familiar. You have the clues, you have to solve the case and you have to force the killer into a dramatic courtroom confession. This participation, a fact unique to its video game format, is what sets the series apart. Well, that and its off the wall humor.

The legal system in the game is vaguely based on the Japanese legal system (very judge oriented, confessions by the defendant aren't compelling, prosecutors take even the concept of losing a case to be a personal insult, etc.) and beyond that fairly absurd. I'd like to read a lawyer's comments, but what I'd really like to read is a philosopher of law talk about the series. The reason is that a good lawyer has a lot of knowledge of one kind of law, just like a good surgeon has a knowledge of one species, but a philosopher of law would be better to comment on an arbitrary legal system - and the legal system of the Ace Attorney series is often very arbitrary!

Being a series of stories about murder, betrayal, decade long schemes, endless lies, corruption at high levels of government and innumerable small ills that accompany the big ones, the series is mostly a comedy. The characters are as over-the-top and broadly drawn as anything Dickens wrote. David Simon once said "Murderers lie because they have to; witnesses and other participants lie because they think they have to; everyone else lies for the sheer joy of it...". And so it is, everybody is filled with secrets, sometimes things embarrassing only to them and sometimes issues that are more comprehensible. One murder involved a low rent tokusatsu studio. One young fan refused to tell what he saw because he was traumatized. Not by seeing a murder, but because he thinks he saw his favorite hero be defeated! Truly we live in trying times.

Another recurring element in the series is mysticism. Not dark, esoteric lore like Eric Voeglin or more intellectual fare like the Shin Megami Tensei series. Possession and spirit channeling are major elements of the series. This seems to be a trademark of producer and man in charge Shu Takumi, since he also made the excellent puzzle game Ghost Trick which is about ... well, ghosts. If this bugs you, then so it goes.

Complaints, I have a few. The series is sometimes too linear. Many of the times I've failed to present evidence it was because there are two problems with testimony and I was going to present them in the wrong order. The series doesn't have a lot of replay value, especially since you'll spend time asking everybody everything the first run through. Well, anyway, back to work - this time for real!

Personal Note: Posts full of dull pictures of scientists streak broken! Now I will be less self-conscious.

Private Life of a Cat

And many others at the Prelinger Archive!

Pseudo-Review: Physical Cosmology And Philosophy

The first place I ever read about Physical Cosmology was Jim Peebles's Principals of Physical Cosmology, a good case study in the genre "textbook". If I were to publish any real philosophy of science, I would certainly place the chapter gently, but firmly explaining what is wrong with fractal cosmology ideas high on the list of things to analyze.

Imagine my surprise when a philosophy section in a local used book store, a section usually barren of anything interesting, contained a collection essays Physical Cosmology and Philosophy. I haven't read every essay yet, so this is a tentative pseudo-review. Most of the writers are pursuing their own hobby horses, and the editor can't be accused of homogenizing them. I don't know how he made decisions with respect to ordering. Many writers talk about deep issues such as the interpretation of quantum mechanics and singularities in general relativity, but there's no essay introducing those concepts for non-experts.

G Gamow

I skipped the first essay, so that the book starts for me on page 51, with the second essay - Modern Cosmology, written by George Gamow. Gamow is a very good writer, and the essay is sufficiently hedged and popular that it hasn't aged terribly bad. Good enough for philosophers, but Cosmology today is a very lively and data-driven field, so it might not be best for people who want hard equations. This is continued by an essay by W B Bonner on the cosmological principle. Big Crunch and steady state theories were once popular, and one shows up in this and several other essays. These are no longer of great interest, and won't be without a significant theoretical and empirical revolution.

 Trurl

There was one essay that annoyed me. Richard Swinburne makes an extremely crude argument from design for the existence of God, quite inferior to what I expect are his abilities. I think he is out of his element. His argument is, of course, subject to all the objections of Hume and his successors. He certainly fails to give any reason to suspect that The Ancient of Days rather than Trurl is behind things.

There are many essays by people with non-standard views on cosmology, frequently airing their methodological views on why the evidence isn't as strongly pro-standard view as it might seem. This is to be expected, but I think that the standard view could be expounded and defended. As it is, nobody is given the chance to answer these arguments.

In summary, this is a highly variable book that shouldn't be read by those who are looking for a quick introduction to a field and it's possible philosophical implications, but to those who are learned enough not to fall for everything they read and willing to put up with some old fashioned views and ... well, I guess that's not too many people anyway.

Bayesian Theory and the Supernatural: Edwin Jaynes's Perspective

E Jaynes (top left)

Edwin Jaynes was a physicist, statistician and probability theorist. He believed these fields to be inexorably linked, that they were all extended forms of logic applied to experimental experience. He was, in essence, a hardcore logical positivist who fully embraced Positivism's subjective implications. Obviously, Jaynes was a Bayesian. For him probability was just the formal thinking through of beliefs. He was inspired by Gibbs's approach to statistical mechanics, which involves (most importantly) an ensemble defined by what experimental parameters you can measure or control - that is, subjectively. What you can't control is considered to be maximally random given what you can, that is, the entropy is maximized. This is all done before the data was looked at, which is to say a priori. Therefore, the distribution over the ensemble is called the "prior" distribution. Gibbs could be confident in this approach, he had physical and mathematical arguments that gave him seemingly reasonable priors. Beyond these priors, we can use Bayes rule to change our beliefs when we get data. Therefore, this method is called "Bayesianism".

T Bayes

Jaynes felt that all of science could be done with just those mathematical arguments to do all of science. He was inspired by Claude Shannon's information theory, which made Jaynes realize that there was a certain universality about Gibbs's arguments beyond their physical basis. There have been objections. I want to emphasize that these are objections to the basics of the theory. Obviously, with enough unmotivated tricks and lowering of standards, any philosophy can be saved (it isn't as clear to me that any theory can be, but philosophers of science seem to think so). Anti-Bayesian Cosma Shalizi notes that the core method always gives exponential distributions, which is very worrying. One can do different things to get heavy tails, but it's not obvious to me that these are well motivated in Jaynesian theory. We were promised that the Gibbs Approach To Statistical Mechanics was the path to science because of the universality of Shannon Entropy theorems. How much of that can we get rid of before we are sold a bill of goods?

 

Psychic Phenomena

I would like to open a line of inquisition that I think goes deeper than technical matters. I will do so in a specific context: parapsychology. In many situations, Bayesian solutions do not appear to give us what we want, evaluations of whether a phenomenon is real or not. The reason is not just subjectivism, but personal subjectivism (if that makes sense).

A young person who reads history may be surprised how many Victorian thinkers were deeply interested in the supernatural, and not just the usual crowd. Henry Sidgwick, an economist, philosopher and moralist, was a founder of the Society for Psychical Research and his successors included the psychologist and philosopher William James and the physicist William Crookes. Ian Hacking has shown that the philosopher, mathematician, statistician, physicist and co-founder of experimental psychology Charles Sanders Peirce invented the modern concept of randomization partly to deal with practical theories in psychology (namely, "Is there a threshold below which weights cannot be distinguished?" is answered "No."), and it was applied widely to para-psychological research, where it has been a powerful force for skepticism. The classic randomization set up is described by R A Fisher to test whether a lady could tell whether tea or milk was added to a cup first (the answer, incidentally, was "Yes."). This simple test was perceived by Fisher to have deep lessons for experiments in general. Randomization is a part of (note: not the foundation, as Hacking above makes clear) Deborah Mayo's error statistics approach to frequentism, because randomization is a kind of error probe. And in this case, it gives us what we say we want, an answer to the question" Are the alleged phenomena distinguishable from blind chance?".

With Thanks

My objection to Jaynes concerns Chapter 5 of Probability Theory: The Logic Of Science, "Queer Uses For Probability Theory". In this chapter, Jaynes takes up the subject of psychic phenomena, as an example of how probability theory can be used to evaluate subjective degrees of belief. This is an important subject. In truth, Professor X cannot tell whether tea or or milk was put first in the cup, but Dr Bristol can. Of course, Professor X claims otherwise. His claim might be honest, as many magicians fool themselves eventually. RA Fisher would say, let's see how they do in the same test, and he'd compare the likelihood ratio, etc. Jaynes's examination is different. Jaynes does and must say that we should start by evaluating our internal degree of belief in the manner of IJ Good:

"Our brains work pretty much the way this [Bayesian] robot works, but we have an intuitive feeling for plausibility only when it's not too far from 0 db. We get fairly definite feelings that something is more than likely to be so or less than likely to be so. So the trick is to imagine an experiment. How much evidence would it take to bring your state of belief up to the place where you felt very perplexed and unsure about it? Not to the place where you believed it - that would overshoot the mark, and again we'd lose our resolving power. How much evidence would it take to bring you just up to the point where you were beginning to consider the possibility seriously?

We take this man who says he has extrasensory perception, and we will write down some numbers from 1 to 10 on a piece of paper and ask him to guess which numbers we've written down. We'll take the usual precautions to make sure against other ways of finding out. If he guesses the first number correctly, of course we will all say 'You're a very lucky person, but I don't believe it.' And if he guesses two numbers correctly, we'll still say 'You're a very lucky person, but I don't believe it.' By the time he's guessed four numbers correctly - well, I still wouldn't believe it. So my state of belief is certainly lower than 40 db."

Jaynes estimates that he has a belief degree of -100 db in psychic phenomena. These so-called psychics often ask us to keep our minds open, to not do what Jaynes is doing. Why should we not oblige them, at least in a formal manner? It seems that it would be better dialectically if they still couldn't convince us even if we gave them the benefit of the doubt. In the long run, experiments will eventually overcome any initial degree of belief, but strong disbelief can make it take a very long time. Jaynes goes on to give himself another out: "In fact, if he guessed 1000 numbers correctly, I still would not believe that he has ESP...". He describes an experiment that supposedly gives very strong evidence of psychic phenomena. But it would not convince Jaynes. "[The psychic] will then react with anger and dismay when, in spite of what he considers this overwhelming evidence, we persist in not believing in ESP. Why are we [Jaynesians], as [the psychic] sees it, so perversely illogical and unscientific?". Jaynes argues that the data can never prove psychic phenomena to Jaynes, because all a high likelihood (that is, data improbable to be produced by chance) does is increase the probability of error and deception.

But this is not what we say we want. How do we know that if this idea were not pursued that it wouldn't end up giving you error statistics, perhaps error statistics constrained by Bayesian constraints? I think that it would, though I don't have much argument for it. Further, how does this rule in cases of mistaken disbelief. If I had a mathematically intimidating (but physically naive) argument from statistical mechanics that it shouldn't matter whether tea or milk is put first, I could and should use this out to disbelieve Dr Bristol as much as Professor X. The fact of her complete success is meaningless, since I can simply say that she managed to deceive me. Write mathematically intimidating but ill founded models, and bend the evidence to their will? Perhaps this is what economists do all day (joke)!

What is "wrong" here is that the Bayesian solution is for yourself. If you are a firm that is evaluating its goods and you want to convince yourself they are good, then Bayesianism works well. Good (and Savage and Friedman, etc.) broke their teeth on the statistical problems of WWII, many of which were of this sort. But one advantage of the error statistics approach is dialectical - it is for convincing others. At the end of the experiment, Dr Bristol and Sir Fisher agree. When particle physicists discovered the Higgs Boson, they used (among other things) p-value analysis and other frequentist techniques. This is because they are in dialog with themselves about what the ultimate laws of physics are, not loners trying to make their priors a little sharper. Minimax estimators are the ultimate expression of the Hegelian philosophy (even worse joke)!

Don't make the mistake that dogmatic adherence to only the strictest of frequentist ideas (for instance, banishing stopping times) is the only path forward! In fact, the man who designed the lady drinking tea experiment described had some very strange ideas about probability. Problems formally equivalent to interior monologues happen all the time. Bayesian theorists have made many contributions to science. I mean to say this as a corrective, not as a declaration of purpose. It's an easy mistake to make that just because one side seems wrong, the other must be right. Instead, what I think is right is to let one hundred flowers bloom, but only enjoy them at the appropriate time. For instance, if optional stopping is desirable, there is no reason to report statistics unstable in their presence. If broad agreement is necessary then there is no reason to report "results" that depend on unmotivated priors. This is obviously correct, but sadly totally informal. What is to be done?

The Mandelbrot Approach To Statistical Mechanics

B Mandelbrot (gasp, color!)

 
The mathematician Benoit Mandelbrot is most famous for coining the term "fractal" and being one of the first to treat them as anything more than theoretical curiosities. He was widely known to be in the first rank from a young age, working some of the best scientific institutions in France and the United States. Fans of mathematical genealogy will be pleased to learn that he was sponsored at the Institute of Advanced Study by John von Neumann himself! He spent most of his mathematical life working at IBM, where he used and developed both the tools (computers, visualization, statistics) he would use for the rest of his life and found the problems that interested him the most. Mandelbrot was a "visual thinker", more so than most, and his most famous book The Fractal Geometry of Nature was a best seller at least partly because he was able to expound his mathematical view of the world without dense equations. I'd recommend it, but it has been quite some time since I've even so much as leafed through it.

Strangely enough, this post isn't a place for going into Mandelbrot's most famous views and discoveries, the things that will make him immortal. Instead, I want to highlight some really interesting applied mathematics work he did at IBM. Last post, I sort of mentioned a few approaches to statistical mechanics, two of which got big time names behind them the Boltzmann (single set-up, large number of particles) and Gibbs (ensemble, ergodic system) approaches. I mentioned that there are two other approaches, both avoiding interpretation as much as possible in different ways. The last approach mentioned was to do straight up dynamical systems. This approach is very rigorous, very slow, and open to complaints that we no longer believe in dynamics smooth in the very small (in other words, it's still classical but nature is quantum). Good work continues to be done in this field, such as this description of a deterministic random number generator by Fields Medalist Artur Avila. The other method was to avoid dynamics as much as possible, postulate the existence of free energy functions and derive what you need from there. This method is highly visual and very fruitful, but has certain scientific limitations because, as mentioned before, free energy functions are not always analytic. What I completely failed to mention is that Mandelbrot made some huge contributions to this field

Wrong kind of field

The first contribution is this paper, which is very excellent. It concerns a statistical interpretation of the free energy function approach, making it more clear that this really is a kind of statistical mechanics. This is in many ways a precursor to Jaynes's method, but doesn't have the same subjective implications because the probabilities aren't conceived as coming from a subjectively defined ensemble. In fact, it is very important to Mandelbrot that the estimators will converge on the truth, which he calls "self-consistency". The paper shows that many thermodynamic variables of interest (such as temperature) are sufficient statistics of whatever it is that underlies them. That is, if you know the volume, temperature, mass, et cetera of a system, you can't learn more by tracking the particles or otherwise zooming in. One remarkable about this paper is its inversion of our usual values. We usually think of the randomness and chaos of statistical mechanics as being the interesting part, the entropy, the second law. In this approach, it is the measurability (the zeroth and first laws) of these systems that has the deep implications and powerful theorems!

This work is extended in this paper and its sequels, in which several more classical concepts are introduced. This paper is, in many ways, better than the first, being more clearly written, but the first is more focused and impressive. It contains an interesting "uncertainty relationship": the better energy is estimated, the worst (inverse) temperature is estimated. This is a classical uncertainty relationship, a consequence of the linearity of Laplace transforms. This paper also pushes a more philosophical line, which is interesting in itself. Mandelbrot was obviously impressed by the positivists, and isn't afraid to let a little subjectivity get into his arguments as a result. I will suspend judgment about this.

I think this part of Mandelbrot's work is of the very highest quality, but I will remain agnostic about judging it against the other methods, especially due to problems about phase transitions, etc. I think his contributions as being good enough that I will call this approach The Mandelbrot Approach, placing him along side Gibbs and Boltzmann.

A Few Approches To Statistical Mechanics

J C Maxwell (this is gonna be another of those black and white men posts...)

I've been collecting some thoughts recently about statistical mechanics, after repurchasing some old books. There are several approaches to statistical mechanics, which can have different seeming implications. I am not going to attempt to carefully delineate them, theorems from one may be used in another. I am going to just list a few thoughts, no real substance yet, but maybe in the future there will be more. There's tons to think about.

R Clausius

The first and oldest is no interpretation at all. More to the point, in the original purpose of statistical mechanics was to get classical thermodynamics out of Newton's Laws. One could avoid this, try to derive thermodynamic relations by presuming that they can be derived by working with analytic free energy functions. These functions are tractable and have beautiful and suggestive visualizations. This technique was used by James Clerk Maxwell and the earlier Gibbs - Gibbs's massive monograph "On The Equilibrium Of Heterogeneous Substances" is basically a formalization of this method. This method is also covered in many an engineering textbook. One could try to stick to these guns, but there are severe limitations on this method. As outlined here, this theory gives poor result when it comes to phase transitions and other physically important phenomena. In addition to it's limitations, it gave a somewhat false view of it's theorems. For instance, Maxwell thought that the early, static approach to the second law of thermodynamics was necessary to avoid perpetual motion. But I've discussed this before.

Inspired by his work on Saturn's rings and with an eye on extending preliminary results due to people like Bernoulli and especially Clausius pictured above. Instead of trying to use global results (more on this later), Maxwell concentrated on the whole distribution. Conceptualizing a gas as being made of hard spheres that impart velocity on each other. Maxwell showed that there was only one stable distribution with certain symmetry properties (there have been other derivations since). By considering the average case, Maxwell was able to develop statistical laws. Thus was the birth of statistical mechanics.

L Boltzmann

The first of the three big approaches I'll call the Boltzmann approach. Boltzmann was a physicist who worked in what he felt to be the tradition of Maxwell (in several of his polemical papers, he uses the Maxwell name as a weapon). I do think his thinking was along these lines, but I wouldn't mind being informed of differences. Boltzmann's approach was based on very large assemblages. This forces the dimension of the subject very high, allowing one to use concentration of measure results to argue that the system will spend almost all of its time in a specific state. Concentration of measure results are direct rather than asymptotic, making them useful for machine learning (hint hint!). A quick introduction to concentration of measure can be found here. The dynamics are important (frequently, Markov Models are used or found), but not as central as the concentration of measure results. One can use these methods to find concentration of measure results, which makes this approach really interesting to me.

Because the measure of the actual system really is concentrated and only the actual system enters into the result, this approach is very objective. For instance, entropy is a semi-objective property. Entropy is the count of microstates compatible with the observable macrostate (some subjectivity comes in how we define microstates). There's no obvious connection to information theory in this approach, and being objective has nothing to do with Bayesianism. In this approach, ensembles are a calculation device justified by ergodic theorems and equilibrium arguments

J W Gibbs

The second of the approaches I will call the Gibbs approach. This approach is ensemble based. Instead of just considering the actual system, every possible system - subject to certain constraints. Entropy is the spread of the ensembles. These constraints are often more about our measuring systems than the system itself, which leads to subjectivity. That entropy is subjective makes this sit well with Bayesian approaches. Bootstrapping and other resampling methods have their origin in this kind of thinking.

Again, though dynamics can be important, they are secondary to information theoretic arguments. Arch-Bayesian Edwin Jaynes is famous for his dogmatic, chapter and verse defense of this approach. How does the use of ensembles sit with the supposedly Bayesian likliehood principle? Aren't we supposed to only consider the actual results? I don't know.

I don't know how the Gibbs approach relates to the Boltzmann approach, especially out of equilibrium. Sorry.

H Poincare

The third approach is the dynamical systems approach. This approach is very, very powerful and underlies the others. However, it basically never has answers. As an example, it was 103 years before even one dynamical system was directly found that obeys the above mentioned Maxwell distribution. At the time of Maxwell and Boltzmann this was the uncontroversial foundation of all things, but these days quantum mechanics and quantum field theory are known to be more basic. The first big result in this field is Liouville's Theorem. Other important results are proofs of chaotic behavior, ergodic theorems, etc. I've talked about my favorite learning device for learning this theory in this post. This is the best approach if you like your science extremely rigorous and moving at the speed of pitch.

Personal Note: had a conversation about application of Kelvin Inversion to economics and statistics (!!!) can't remember a single thing. Look for this later.