Who Cares About Ergodic Systems?

This is a quick teaching post. This stuff is high school level, but to make it formal can push it beyond the research level into high level philosophy.

 

The creator of Dr Pepper, Dr Alderton

 
First, some physical intuition. I pour some Dr Pepper into a cup. What shape does the fluid take on? There are really three fluids - Dr Pepper (largely water, basically incompressible), carbon dioxide (which takes the shape of tiny, interacting bubbles) and ambient air. There are many forces - solid resistance, buoyancy force, skin and interaction forces on the bubbles, gravity and (since the Dr Pepper and carbon dioxide are colder than the ambient temperature) thermal forces. The short run question of what happens to the fluids is complicated and depends on many tiny factors.

Despite this, the long run solution is easy - basic fluid statics tells us that the Dr Pepper will take the form of the cup and basic thermostatics tells us it will be the same temperature as the ambient atmosphere.

 

John von Neumann

How do we capture this intuition that - roughly - in the short run history matters but in the long run only structure matters? For many years, physicists and mathematicians have turned to Ergodic Theory to answer this question. Ergodic theory doesn't exactly have a great reputation.

Many people - including high powered top level experts - think that not only does ergodic theory require the formal manipulation skills of a von Neumann, the geometric insight of a Clerk Maxwell and the engineering experience of a Shannon - it doesn't even solve the problem.

But really ergodic theory is very simple - except for all the parts that are hard. Shannon's paper can be polished off in a couple days, and (with all due respect to Joe Doob) it's not clear that there is more to the theory than that.

You don't want to take a few days. Well, here's the few minutes version.

 

A connected and a disconnected network

We start with the intuitive idea of a network. We call the nodes the state. There are finitely many states and the each have a name. From a given state, there is a rule to transfer to one of the other states to which that node is connected. The rule and the network together are called the system. In theory the rule can be anything, for instance it might be "always go as far down as possible" where down is defined geometrically or topologically. The rules can be probabilistic.

A system is called "ergodic" if the long run amount of time spent at each node is independent of which node you start at. The idea of state gives us the short run detail dependence and the ergodicity gives us long run structure dependence.

For my deliberately dumb "go as far down as you can" rule on the above connected network, I have six possible runs

Pink, Purple, Purple, Purple...
Brown, Purple, Purple, Purple...
Blue, Black,  Purple, Purple, Purple...
Orange, Purple, Purple, Purple...
Black, Purple, Purple, Purple...
Purple, Purple, Purple...

No matter where I start, the long run relative frequency \(f_{purple} = 1\) and all others are \( 0 \). Therefore, this dumb system is ergodic. If we try the same thing on the disconnected network:

Black, Brown, Pink, Pink, Pink...
Brown, Pink, Pink, Pink...
Purple, Orange, Orange, Orange...
Blue, Orange, Orange, Orange...
Orange, Orange, Orange...

For the first two starting places the relative frequency of pink goes to one, for the second three, the relative frequency of orange goes to one. This dumb system is non-ergodic. But notice it is two ergodic pieces. In general, a non-ergodic system can be severed into ergodic components (in this case, the two connected subnetworks).

The underlying being connected isn't in general sufficient for being ergodic. On the above left graph, sever the Orange-Purple connection and follow the "go down" rule (question to check if you understand: what are the two ergodic subsystems?). It turns out* kinds of rules that are of physical interest are usually of the form "Given that I am on state N, I go down each connection NM with a certain probability \( p_{NM}\neq 0 \)". For such a rule, being connected is sufficient for ergodicity**. So in this informal blog post I'll choose rules and networks such that connectedness and ergodicity are equivalent.

The ergodic distribution tells us the long run behavior of the system, but it also teaches us about the medium run behavior. We know that if the frequency at a state is "too low" (compared to the ergodic frequency), then we will see a flow into that state. This is more or less a definition of what a flow is!


This is all well and good, but what does it have to do with physics? A continuous system is ergodic if the one can cut up the possible states of the system into a discrete ergodic system. Let's make a pair of networks out of a physical model - a billiards model. I mentally divide a square billiard table into four regions A, B, C and D

Being in a region isn't sufficient to fix the dynamics - I need to know the velocities. I think that it's obvious that velocity digitizes into four chunks based on the number of regions away from the starting region you end up in after a time step. So the states are really:

A0, A1, A2, A3
B0, B1, B2, B3
C0, C1, C2, C3
D0, D1, D2, D3

Each 0 connects only with itself (remember, the billiard isn't necessarily staying still, it could be through all four blocks in one tick). There's a cycle A1 connects to B1 connects to C1 connects to D3 connects to C3 connects to B3 connects to A1. There are three cycles of length two, A2 connects to C2 connects to A2, B2 connects to D2 connects to B2 and A3 connects to D1 connects to A3. This is illustrated below

This particular digitization of the underlying continuous system isn't ergodic. If you start off with A0, then \( f_{A0} =1 \), if you start off with a non-zero velocity state, then \( f_{A0} =0 \). That's enough to show that this isn't an ergodic system.

It turns out that there is no nontrivial digitization of this system that is ergodic. That's because this system is exactly solvable... and I won't tell you why that's connected to ergodicity***.

Let's put a circular block in the middle of the square. Now the graph isn't disconnected. A particle that started four blocks per tick can have it's angle of attack by hitting the circular block to now be 3 blocks per tick (that is, it may be turned around because 3=-1). I don't know if this particular graph is really ergodic and I'm not going to check. In a tour de force, Yakov Sinai proved that this system has an ergodic digitization. This shows that the system is itself ergodic.

That means if the particle isn't in, say, C enough (compared to the ergodic distribution) we will see a flow towards C, just as in the discrete case. This is how ergodicity connects to physical quantities.

Finally: wasn't that Black Thought freestyle great?

*By the magic of symbolic dynamics
** By the magic of Markov Chains
*** I'll let wikipedia do it

What We Talk About When We Talk About Food

The vast majority of Discourse about health is lies. Go to a supermarket and look at the "health shakes". Beyond the outright lies that are the overwhelmingly greater part there are the tentative part-truths - usually represented as absolute certainties. Supposedly there could logically could be definite truths. I have never seen them.

How do we talk about food? In episode 2 of Frasier, Frasier Crane discusses his preferred breakfast with his father. A "low fat, high fiber" breakfast with (terribly expensive) plain black coffee. There's a lot to be said about the implicit social views of eating in this scene. But instead imagine this: out of the woodwork I wander onto the show and say "Actually Dr Crane, you would be better off substituting that bran muffin for bacon.".

What does that mean? A calorie neutral substitution? A mass neutral substitution? A subjective substitution? The second is not a joke - it's clear (is it?) that a person who is fed via IV would "feel hungry" and attempt to eat whether the IV was sugar or oil. In the language of economics such a person would hit their first order conditions for diet optimality (I.e. calories would be right) but not their second order conditions. They're at a minimum utility. This cannot last. Believing in diet advice unstable to perturbations is unscientific - completely methodologically unsound. The third option is not necessarily unscientific - subjective feelings of fullness are related to the health relavent properties of food. Attempting to lose weight via a simplistic, objective calorie/mass accounting system may again put you at an unstable equilbrium. These kind of yo-yo unstable diets aren't obviously healthy.

The market for food is broken, possibly irrevocably broken. There has never been, in all the history of mankind, a society where farm labor is economically valued. As a result, all industrial societies prop up production - often in highly distortionary ways. It is obvious that, for instance, the US overproduces corn carbohydrates. This is a Bad Thing.

On the consumer side, it must be admitted by any person who desires to be taken seriously that branding and monopolistic competition more generally are real. Government intervention has been ineffective at policing this, even when it has been pointed in the right direction. This is unsurprising - Coase on the right and Stiglitz on the left are always fond of pointing out that the conditions in which governments/markets can fail are the exact conditions markets/governments can fail.

This brings us back to the first point. How do we talk about food? There is an enormous signaling problem here. Just as every prospective worker has an incentive to appear to be valuable to a prospective employer, so every prospective meal has an incentive to appear to be healthy to a prospective eater. (Healthiness & productivity of course defined variationally) Eaters therefore statistically discriminate, choosing foods with a few easily observed outward signs of "healthiness". Sugary cereals put photos of nutritious meals on the box. Having blueberries and a green container turns a malted into a diet food. More informative statistics - carbohydrate and protein and fat measures, total calories, ingredients, etc. - are buried in confusing, neutrally colored, small type statistical abstracts.

Adding exercise to a given diet is generally good, since exercise to a large extent determines the distribution of variable masses for a person of a given mass. We may not be indifferent between being Akebono and Bob Sapp. As Kimball notes above & every bodybuilder knows - total weight balances the mass of food that comes in and goes out. (Also, cardio health is good, even though the mass of the cardio system isn't particularly variable in mass) But there are huge difficulties here. First, it isn't methodologically sound to assume a person can vary exercise and not vary their diet (it assumes that their old equilibrium was unstable, which is exactly what it is not for an obese person who can't lose weight). The market for exercises is not obviously healthy. Like with food, every prospective exercise plan has an incentive to seem healthy & sustainable even if it is not. Survivor bias is endemic here - everyone who keeps up My Super Special Program long enough achieves their weight & weight distribution targets and everyone who doesn't leaves.

You can't sit around thinking about your diet all day. Simplistic accounting techniques can beat advanced techniques just because they're easier to understand. It may not be easy to account for carbohydrate, protein or fat intake simply because their are so many kinds of carbohydrates, proteins and fats that eating the "wrong" kind may not give the eater noticeable feedback. Fasting can then empirically outperform theoretically superior modes because it's easy to notice when you cheat.

These are only the simplest economic metaphors. Beyond this there are cultural and even political factors. But that'll have to wait for another post.

Nozick On ... Inequality?

Robert Nozick was a Harvard philosopher, a political philospher among other things. He was an odd duck with an interesting sense of humor - speculating that autofellatio plaid a role in classical Hindu yoga was typical of his crass jokes. But he was very serious about one thing - Nozick sincerely believed in a philosophical theory of social desert - that one should be allowed to own all and only the goods entitled to you. Nozick traced this theory to the Lockean theory of production & distribution. Locke believed that one owned those goods which one mixed with one's labour (if you were European anyway).

Nozick brought his beliefs to so-called "libertarian" ends. If you are entitled to those goods to the extent which you mixed your labor into them, then it is not clear that you are entitled to any public goods at all. Nozick had an argument - the Utility Monster argument - that utilitarianism (the philosophical position that public policy should aim at some measure of aggregate happiness) could not be a priori true. Consider a society consisting of two consumer classes, one with decreasing returns to consumption (normal people) and one with increasing returns to comsumption (utility monsters). Holding a nations's output constant, the utilitarian political advisor says it is always worth it to tax the normal people and subsidize the utility monsters, which seems unjust a priori. Nozick says, shortly, that utilitarianism is false because it can excuse income inequality.

But here Nozick reaches an impasse. You see, it's not clear that a entitlement theory of desert avoids income inequality. In fact, Nozick argues that entitlement is true despite the fact that it can justify income inequality. His argument is not complex. First of all, he assumes that the set of just objects is closed - any outcome that is reached by individually just actions is just. Next he constructs a possible world where income inequality seems justified. In this imaginary Pittsburgh, everyone starts with $1. But one person is special - he's Wilt Chamberlin. If even one person pays 1¢ to see an exhibition from Wilt The Stilt then he becomes - through no fault of his own - the richest man in Pittsburgh. Why is this unjust?

There are several responses to this. One is tu quoque - why is this income inequality obviously good but utilitarian income inequality obviously bad? But there are sharper critiques. Despite the appearance of dollars and cents, Nozick's example is not economic. The people of imaginary Pittsburgh are not given alternate uses for their money. Why would they hold cash? Why does Chamberlin hold cash? This points us to the deeper problem arising from Nozick's economic ignorance - income is a flow but he treats it as a stock. It is of interest to Nozick that Chamberlin's capital recieves dividends, but it's clear he hasn't placed those dividends in a society - his "possible world" is unworthy of the name.

The choice of Wilt Chamberlin is careful rhetoric. Most capital that pays dividends as the Wilt Chamberlin case can be transferred - in the case of extreme regimes, land can be nationalized, factories can be seized, etc.. But Nozick clearly believes Chamberlin's God given talents are just that - in born natural talent (that Nozick thinks this about Chamberlin brings up the issue of race in ways I won't address) that can't be transferred. There might be a similar point about transferable utility in the utility monster case. The only way to redistribute the returns on Chamberlin-capital is to tax the income - right?

Well, maybe maybe not. Most modern theorists on inequality concentrate on wealth inequality rather than income inequality. Unlike income inequality, wealth inequality doesn't correlate with Wilt Chamberlin like capital - not with things that a priori seem justified. Not IQ, for instance. This is a positive, not a priori case but it is still a hole in Nozick's point. Even if one believes that Chamberlin deserves the income he recieves from his non-transferable capital, that doesn't mean that one believes he deserves his stock of wealth. So Nozick's argument is all a little bit old fashioned.

In sum, I think that Nozick's argument is unpersuasive for two reasons. It isn't obvious that he solves what he thinks are problems with other theories and even the internal coherence of his story is questionable. Still, Invariances is a good book.