Losing my marbles over entropy

In a previous post on Entropy, I offered a garbled notion of "statistical symmetry." I'm currently reading Carlo Rovelli's The Order of Time, and chapter two laid out the idea that I was grasping at concisely, clearly and -- because Rovelli is an actual physicist -- correctly.

What follows is a fairly long and rambling discussion of the same toy system as the previous post, of five marbles in a square box with 25 compartments. It does eventually circle back to the idea of symmetry, but it's really more of a brain dump of me trying to make sure I've got the concepts right. If that sounds interesting, feel free to dive in. Otherwise, you may want to skip this one.

In the earlier post, I described a box split into 25 little compartments with marbles in five of the compartments. If you start with, say, all the marbles on one row (originally I said on one diagonal, but that just made things a bit messier) and give the box a good shake, the odds that the marbles all end up in the same row that they started in are low, about one in 50,000 for this small example. So far, so good.

But this is really true for any starting configuration -- if there are twenty-five compartments in a five-by-five grid, numbered from left to right then top to bottom, and the marbles start out in, say, compartments 2, 7,  8, 20 and 24, the odds that they'll still be in those compartments after you shake the box are exactly the same, about one in 50,000.

On the one hand, it seems  like going from five marbles in a row to five marbles in whatever random positions they end up in is making the box more disordered. On the other hand, if you just look at the positions of the individual marbles, you've gone from a set of five numbers from 1 to 25 ... to a set of numbers from 1 to 25, possibly the one you started with. Nothing special has happened.

This is why the technical definition of entropy doesn't mention "disorder". The actual definition of entropy is in terms of microstates and macrostates. A microstate is a particular configuration of the individual components of a system, in this case, the positions of the marbles in the compartments. A macrostate is a collection of microstates that we consider to be equivalent in some sense.

Let's say there are two macrostates: Let's call any microstate with all five marbles in the same row lined-up, and any other microstate scattered.  In all there are 53,130 microstates (25 choose 5). Of those, five have all the marbles in a row (one for each row), and the other 53,125 don't. That is, there are five microstates in the lined-up microstate and 53,125 in the scattered microstate.

The entropy of a macrostate is related to the number of microstates consistent with that macrostate (for more context, see the earlier post on entropy, which I put a lot more care into). Specifically, it is the logarithm of the number of such states, multiplied by a factor called the Boltzmann constant to make the units come out right and to scale the numbers down, because in real systems the numbers are ridiculously large (though not as large as some of these numbers), and even their logarithms are quite large. Boltzman's constant is 1.380649×10⁻²³ Joules per Kelvin.

The natural logarithm of 5 is about 1.6 and the natural logarithm of 53,125 is about 10.9. Multiplying by Boltzmann's constant doesn't change their relative size: The scattered macrostate has about 6.8 times the entropy of the lined-up macrostate.

If you start with the marbles in the low-entropy lined-up macrostate and give the box a good shake, 10,625 times out of 10,626 you'll end up in the higher-entropy scattered macrostate. Five marbles in 25 compartments is a tiny system, considering that there are somewhere around 10,800,000,000,000,000,000,000,000 molecules in a milliliter of water. In any real system, except cases like very low-temperature systems with handfuls of particles, the differences in entropy are large enough that "10,625 times out of 10,626" turns into "always" for all intents and purposes.

This distinction between microstates and macrostates gives a rigorous basis for the intuition that going from lined-up marbles to scattered-wherever marbles is a significant change, while going from one particular scattered state to another isn't.

In both cases, the marbles are going from one microstate to another, possibly but very rarely the one they started in. In the first case, the marbles go from one macrostate to another. In the second, they don't. Macrostate changes are, by definition, the ones we consider significant, in this case, between lined-up and scattered. Because of how we've defined the macrostates, the first change is significant and the second isn't.

Let's slice this a bit more finely and consider a scenario where only part of a system can change at any given time. Suppose you don't shake up the box entirely. Instead, you take out one marble and put it back in a random position, including, possibly, the one it came from. In that case, the chance of going from lined-up to scattered is 20 in 21, since out of the 21 positions the marble can end up in, only one, its original position, has the marbles all lined up, and in any case it doesn't matter which marble you choose.

What about the other way around? Of the 53,120 microstates in the scattered macrostate, only 500 have four of the five marbles in one row. For any microstate, there are 105 different ways to take one marble out and replace it: Five marbles times 21 empty places to put it, including the place it came from.

For the 500 microstates with four marbles in a row, only one of those 105 possibilities will result in all five marbles in a row: Remove the lone marble that's not in a row and put it in the only empty place in the row of four. For the other 52,615 microstates in the scattered macrostate, there's no way at all to end up with five marbles lined up by moving only one marble.

So there are 500 cases where the scattered macrostate becomes lined-up, 500*104 cases where it might but doesn't, and 52,615*105 cases where it couldn't possibly. In all, that means that the odds are 11,153.15 to one against scattered becoming lined-up by removing and replacing one marble randomly.

Suppose that the marbles are lined up at some starting time, and every time the clock ticks, one marble gets removed and replaced randomly. After one clock tick, there is a 104 in 105 chance that the marbles will be in the high-entropy scattered state. How about after two ticks? How about if we let the clock run indefinitely -- what portion of the time will the system spend in the lined-up macrostate?

The there are tools to answer questions like this, particularly Markov chains and stochastic matrices (that's the same Markov Chain that can generate random text that resembles an input text). I'll spare you the details, but the answer requires defining a few more macrostates, one for each way to represent the number five as the sum of whole numbers: [5], [4, 1], [3, 2], [3, 1, 1], [2, 2, 1], [2, 1, 1, 1] and [1, 1, 1, 1, 1].

The macrostate [5] comprises all microstates with five marbles in one row, the macrostate [4, 1] comprises all microstates with four marbles in one row and one in another row, the macrostate [2, 2, 1] comprises all microstates with two marbles in one row, two marbles in another row and one marble in a third one, and so forth.

Here's a summary

Macrostate	Microstates	Entropy
[5]	5	1.6
[4,1]	500	6.2
[3,2]	2,000	7.6
[3,1,1]	7,500	8.9
[2,2,1]	15,000	9.6
[2,1,1,1]	25,000	10.1
[1,1,1,1,1]	3,125	8.0

The Entropy column is the natural logarithm of the Microstates column, without multiplying by Boltzmann's constant. Again, this is just to give a basis for comparison. For example [2,1,1,1] is the highest-entropy state, and [2,2,1] has four times the entropy of [5]. 

It's straightforward, but tedious, to count the number of ways one macrostate can transition to another. For example, of the 105 transitions for [3,2], 4 end up in [4,1], 26 end up back in [3,2] (not always by putting the removed marble back where it was), 30 end up in [3, 1, 1] and 45 end up in [2, 2, 1]. Putting all this into a matrix and taking the matrix to the 10th power (enough to see where this is converging) gives

Macrostate	% time	% microstates
[5]	.0094	.0094
[4,1]	.94	.94
[3,2]	3.8	3.8
[3,1,1]	14	14
[2,2,1]	28	28
[2,1,1,1]	47	47
[1,1,1,1,1]	5.9	5.9

The second column is the result of the tedious matrix calculations. The third column is just the size of the macrostate as the portion of the total number of microstates. For example, there are 500 microstates in [4,1], which is 0.94% of the total, which is also the portion of the time that the matrix calculation says system will spend in [4, 1]. Technically, this means the system is ergodic, which means I didn't have to bother with the matrix and counting all the different transitions.

Even in this toy example, the system will spend very little of its time in the low-entropy lined-up state [5], and if it ever does end up there, it won't stay there for long.

Given some basic assumptions, a system that evolves over time, transitioning from microstate to microstate, will spend the same amount of time in any given microstate (as usual, that's not quite right technically), which means that the time spent in each macrostate is proportional to its size. Higher-entropy states are larger than lower-entropy states, and because entropy is a logarithm, they're actually a lot larger.

For example, the odds of an entropy decrease of one millionth of a Joule per Kelvin are about one in e⁽¹⁰¹⁷⁾. That's a number with somewhere around 40 quadrillion digits. To a mathematician, the odds still aren't zero, but to anyone else they would be.

For all but the tiniest, coldest systems, the chance of entropy decreasing even by a measurable amount are not just small, but incomprehensibly small. The only systems where the number of microstates isn't incomprehensibly huge are are small collections of particles near absolute zero.

I'm pretty sure I've read about experiments where such a system can go from a higher-entropy state to a very slightly lower-entropy state and vice versa, though I haven't had any luck tracking them down. Even if no one's ever done it, such a system wouldn't violate any laws of thermodynamics, because the laws of thermodynamics are statistical (and there's also the question of definition over whether such a system is in equilibrium).

So you're saying ... there's a chance? Yes, but actually no, in any but the tiniest, coldest systems. Any decrease in entropy that could actually occur in the real world and persist long enough to be measured would be in the vicinity of 10⁻²³ Joules per Kelvin, which is much, much too small to be measured except under very special circumstances.

For example, if you have 1.43 grams of pure oxygen in a one-liter container at standard temperature and pressure, it's very unlikely that you know any of the variables involved -- the mass of the oxygen, its purity, the size of the container, the temperature or the pressure, to even one part in a billion. Detecting changes 100,000,000,000,000 times smaller than that is not going to happen.

But none of that is what got me started on this post. What got me started was that the earlier post tried to define some sort of notion of "statistical symmetry", which isn't really a thing, and what got me started on that was my coming to understand that higher-entropy states are more symmetrical. That in turn was jarring because entropy is usually taken as a synonym for disorder, and symmetry is usually taken as a synonym for order.

Part of the resolution of that paradox is that entropy is a measure of uncertainty, not disorder. The earlier post got that right, but evidently that hasn't stopped my for hammering on the point for dozens more paragraphs and a couple of tables in this one, using a slightly different marbles-in-compartments example.

The other part is that more symmetry doesn't really mean more order, at least not in the way that we usually think about it.

From a mathematical point of view, a symmetry of an object is something you can do to it that doesn't change some aspect of the object that you're interested in. For example, if something has mirror symmetry, that means that it looks the same in the mirror as it does ordinarily.

It matters where you put the mirror. The letter W looks the same if you put a mirror vertically down the middle of it -- it has one axis of symmetry. The letter X looks the same if you put the mirror vertically in the middle, but it also looks the same if you put it horizontally in the middle -- it has two axes of symmetry.

Another way to say this is that if you could draw a vertical line through the middle of the W and rotate the W out of the page around that line, and kept going for 180 degrees until the W was back in the page, but flipped over, it would still look the same. If you chose some other line, it would look different (even if you picked a different vertical line, it would end up in a different place). That is, if you do something to the W -- rotate it around the vertical line through the middle -- it ends up looking the same. The aspect you care about here is how the W looks.

To put it somewhat more rigorously: if f is the particular mapping that takes each point to its mirror image across the axis, then f takes the set of points in the W to the exact same set of points. Any point on the axis maps to itself, and any point off the axis maps to its mirror image, which is also part of the W. The map f is defined for every point on the plane and it moves all of them except for the axis. The aspect we care about, which f doesn't change, is whether a particular point is in the W.

If you look at all the things you can do to an object without changing the aspect you care about, you have a mathematical group. For a W, there are two things you can do: leave it alone and flip it over. For an X, you have four options: leave it alone, flip it around the vertical axis, flip it around the horizontal axis, or do both. Leaving an object alone is called the identity transformation, and it's always considered a symmetry, because math. An asymmetrical object has only that symmetry (it's symmetry group is trivial).

In normal speech, saying something is symmetrical usually means it has the same symmetry group as a W -- half of it is a mirror image of the other half. Technically, it has bilateral symmetry. In some sense, though, an X is more symmetrical, since its symmetry group is larger, and a hexagon, which has 12 elements in its symmetry group, is more symmetrical yet.

A figure with 19 sides, each of which is the same lopsided squiggle, would have a symmetry group of 19 (rotate by 1/19 of a full circle, 2/19 ... 18/19, and also don't rotate at all). That would make it more symmetrical than a hexagon, and quite a bit more symmetrical than a W, but if you asked people which was most symmetrical, they would probably put the 19-sided squigglegon last of the three.

Our visual system is mostly trained to recognize bilateral symmetry. Except for special situations like reflections in a pond, pretty much everything in nature with bilateral symmetry is an animal, which is pretty useful information when it comes to eating and not being eaten. We also recognize rotational symmetry, which includes flowers and some sea creatures, also useful information.

It would make sense, then, that in day to day life, "more symmetrical" generally means "closer to bilateral symmetry". If a house has an equal number of windows at the same level on either side of the front door, we think of it as symmetrical,  even though the windows may not be exactly the same, the door itself probably has a doorknob on one side or the other and so forth, so it's not quite exactly symmetrical. We'd still say it's pretty symmetrical, even though from a mathematical point of view it either has bilateral symmetry or it doesn't (and in the real world, nothing we can see is perfectly symmetrical).

That should go some way toward explaining why, along with so many other things, symmetry doesn't necessarily mean the same thing in its mathematical sense as it does ordinarily. The mathematical definition includes things that we don't necessarily think of as symmetry.

Continuing with shapes and their symmetries, you can think of each shape as a macrostate. You can  associate a microstate with each mapping (technically, in this case, any rigid transformation of the plane) that leaves the shape unchanged. The macrostate W has two microstates: one for the identity transformation, which leaves the plane unchanged, and one for the mirror transformation around the W's axis.

The X macrostate has four microstates, one for the identity, one for the flip around the vertical axis, one for the flip around the horizontal axis, and one for flipping around one axis and then the other (in this case, it doesn't matter what order you do it in). The X macrostate has a larger symmetry group, which is the same as saying it has more entropy.

In this context, a symmetry is something you can do to the microstate without changing the macrostate. A larger symmetry group -- more symmetry -- means more microstates for the same macrostate, which means more entropy, and vice-versa. They're two ways of looking at the same thing.

In the case of the marbles in a box, a symmetry is any way of switching the positions of the marbles, including not switching them around at all. Technically, this is a permutation group.

For any given microstate,  some of the possible permutations just switch the marbles around in their places (for example, switching the first two marbles in a lined-up row), and some of them will move marbles to different compartments. For a microstate of the lined-up macrostate [5], there are many fewer permutations that leave the marbles in the same macrostate (all in one row, though not necessarily the same row) than there are for [2, 1, 1, 1]. Even though five marbles in a row looks more symmetrical, since it happens to have bilateral visual symmetry, it's actually a much less symmetrical macrostate than [2, 1, 1, 1], even though most of its microstates will just look like a jumble.

In the real world, distributing marbles in boxes is really distributing energy among particles, generally a very large number of them. Real particles can be in many different states, many more than the marble/no marble states in the toy example, and different states can have the same energy, which makes the math a bit more complicated. Switching marbles around is really exchanging energy among particles, and there are all sorts of intricacies about how that happens.

Nonetheless, the same basic principles hold: Entropy is a measure of the number of microstates for a given macrostate, and a system in equilibrium will evolve toward the highest-entropy macrostate available, and stay there, simply because the probability of anything else happening is essentially zero.

And yeah, symmetry doesn't necessarily mean what you think it might.

How real are real numbers?

There is always one more counting number.

That is, no matter how high you count, you can always count one higher.  Or at least in principle.  In practice you'll eventually get tired and give up.  If you build a machine to do the counting for you, eventually the machine will break down or it will run out of capacity to say what number it's currently on.  And so forth.  Nevertheless, there is nothing inherent in the idea of "counting number" to stop you from counting higher.

In a brief sentence, which after untold work by mathematicians over the centuries we now have several ways to state completely rigorously, we've described something that can exceed the capacity of the entire observable universe as measured in the smallest units we believe to be measurable.  The counting numbers (more formally, the natural numbers) are infinite, but they can be defined not only by finite means, but fairly concisely.

There are levels of infinity beyond the natural numbers.  Infinitely many, in fact.  Again, there are several ways to define these larger infinities, but one way to define the most prominent of them, based on the real numbers, involves the concept of continuity or, more precisely, completeness in the sense that the real numbers contain any number that you can get arbitrarily close to.

For example, you can list fractions that get arbitrarily close to the square root of two: 1.4 (14/10) is fairly close, 1.41 (141/100) is even closer, 1.414 (1414/1000) is closer still, and if I asked for a fraction within one one-millionth, or trillionth, or within 1/googol, that is, one divided by ten to the hundredth power, no problem.  Any number of libraries you can download off the web can do that for you.

Nonetheless, the square root of two is not itself the ratio of two natural numbers, that is, it is not a rational number (more or less what most people would call a fraction, but with a little more math in the definition).  The earliest widely-recognized recorded proof of this goes back to the Pythagoreans.  It's not clear exactly who else also figured it out when, but the idea is certainly ancient.  No matter how closely you approach the square root of two with fractions, you'll never find a fraction whose square is exactly two.

OK, but why shouldn't the square root of two be a number?  If you draw a right triangle with legs one meter long, the hypotenuse certainly has some length, and by the Pythagorean theorem, that length squared is two.  Surely that length is a number?

Over time, there were some attempts to sweep the matter under the rug by asserting that, no, only rational numbers are really numbers and there just isn't a number that squares to two.  That triangle? Dunno, maybe its legs weren't exactly one meter long, or it's not quite a right triangle?

This is not necessarily as misguided as it might sound.  In real life, there is always uncertainty, and we only know the angles and the lengths of the sides approximately.  We can slice fractions as finely as we like, so is it really so bad to say that all numbers are rational, and therefore you can't ever actually construct a right triangle with both legs exactly the same length, even if you can get as close as you like?

Be that as it may, modern mathematics takes the view that there are more numbers than just the rationals and that if you can get arbitrarily close to some quantity, well, that's a number too.  Modern mathematics also says there's a number that squares to negative one, which has its own interesting consequences, but that's for some imaginary other post (yep, sorry, couldn't help myself).

The result of adding all these numbers-you-can-get-arbitrarily-close-to to the original rational numbers (every rational number is already arbitrarily close to itself) is called the real numbers.  It turns out that (math-speak for "I'm not going to tell you why", but see the post on counting for an outline) in defining the real numbers you bring in not only infinitely many more numbers, but so infinitely many more numbers that the original rational numbers "form a set of measure zero", meaning that the chances of any particular real number being rational are zero (as usual, the actual machinery that allows you to apply probabilities here is a bit more involved).

To recap, we started with the infinitely many rational numbers -- countably infinite since it turns out that you can match them up one-for-one with the natural numbers* -- and now we have an uncountably infinite set of numbers, infinitely too big to match up with the naturals.

But again we did this with a finite amount of machinery.  We started with the rule "There is always one more counting number", snuck in some rules about fractions and division, and then added "if you can get arbitrarily close to something with rational numbers, then that something is a number, too".  More concisely, limits always exist (with a few stipulations, since this is math).

One might ask at this point how real any of this is.  In the real world we can only measure uncertainly, and as a result we can generally get by with only a small portion of even the rational numbers, say just those with a hundred decimal digits or fewer, and for most purposes probably those with just a few digits (a while ago I discussed just how tiny a set like this is).  By definition anything we, or all of the civilizations in the observable universe, can do is literally as nothing compared to infinity, so are we really dealing with an infinity of numbers, or just a finite set of rules for talking about them?

One possible reply comes from the world of quantum mechanics, a bit ironic since the whole point of quantum mechanics is that the world, or at least important aspects of it, is quantized, meaning that a given system can only take on a specific set of discrete states (though, to be fair, there are generally a countable infinity of such states, most of them vanishingly unlikely).  An atom is made of a discrete set of particles, each with an electric charge that's either 1, 0 or -1 times the charge of the electron, the particles of an atom can only have a discrete set of energies, and so forth (not everything is necessarily quantized, but that's a discussion well beyond my depth).

All of this stems from the Schrödinger Equation.  The discrete nature of quantum systems comes from there only being a discrete set of solutions to that equation for a particular set of boundary conditions.  This is actually a fairly common phenomenon.  It's the same reason that you can only get a certain set of tones by blowing over the opening of a bottle (at least in theory).

The equation itself is a partial differential equation defined over the complex numbers, which have the same completeness property as the real numbers (in fact, a complex number can be expressed as a pair of real numbers).  This is not an incidental feature, but a fundamental part of the definition in at least two ways: Differential equations, including the Schrödinger equation, are defined in terms of limits, and this only works for numbers like the reals or the complex numbers where the limits in question are guaranteed to exist.  Also, it includes π, which is not just irrational, but transcendental, which more or less means it can only be defined as a limit of an infinite sequence.

In other words, the discrete world of quantum mechanics, our best attempt so far at describing the behavior of the world under most conditions, depends critically on the kind of continuous mathematics in which infinities, both countable and uncountable, are a fundamental part of the landscape.  If you can't describe the real world without such infinities, then they must, in some sense, be real.

Of course, it's not actually that simple.

When I said "differential equations are defined in terms of limits", I should have said "differential equations can be defined in terms of limits."  One facet of modern mathematics is the tendency to find multiple ways of expressing the same concept.  There are, for example, several different but equivalent ways of expressing the completeness of the real numbers, and several different ways of defining differential equations.

One common technique in modern mathematics (a technique is a trick you use more than once) is to start with one way of defining a concept, find some interesting properties, and then switch perspective and say that those interesting properties are the actual definition.

For example, if you start with the usual definition of the natural numbers: zero and an "add one" operation to give you the next number, you can define addition in terms of adding one repeatedly -- adding three is the same as adding one three times, because three is the result of adding one to zero three times.  You can then prove that addition gives the same result no matter what order you add numbers in (the commutative property).  You can also prove that adding two numbers and then adding a third one is the same as adding the first number to the sum of the other two (the associative property).

Then you can turn around and say "Addition is an operation that's commutative and associative, with a special number 0 such that adding 0 to a number always gives you that number back."  Suddenly you have a more powerful definition of addition that can apply not just to natural numbers, but to the reals, the complex numbers, the finite set of numbers on a clock face, rotations of a two-dimensional object, orderings of a (finite or infinite) list of items and all sorts of other things.  The original objects that were used to define addition -- the natural numbers 0, 1, 2 ... -- are no longer needed.  The new definition works for them, too, of course, but they're no longer essential to the definition.

You can do the same thing with a system like quantum mechanics.  Instead of saying that the behavior of particles is defined by the Schrödinger equation, you can say that quantum particles behave according to such-and-such rules, which are compatible with the Schrödinger equation the same way the more abstract definition of addition in terms of properties is compatible with the natural numbers.

This has been done, or at least attempted, in a few different ways (of course).  The catch is these more abstract systems depend on the notion of a Hilbert Space, which has even more and hairier infinities in it than the real numbers as described above.

How did we get from "there is always one more number" to "more and hairier infinities"?

The question that got us here was "Are we really dealing with an infinity of numbers, or just a finite set of rules for talking about them?"  In some sense, it has to be the latter -- as finite beings, we can only deal with a finite set of rules and try to figure out their consequences.  But that doesn't tell us anything one way or another about what the world is "really" like.

So then the question becomes something more like "Is the behavior of the real world best described by rules that imply things like infinities and limits?"  The best guess right now is "yes", but maybe the jury is still out.  Maybe we can define a more abstract version of quantum physics that doesn't require infinities, in the same way that defining addition doesn't require defining the natural numbers.  Then the question is whether that version is in some way "better" than the usual definition.

It's also possible that, as well-tested as quantum field theory is, there's some discrepancy between it and the real world that's best explained by assuming that the world isn't continuous and therefore the equations to describe it should be based on a discrete number system.  I haven't the foggiest idea how that could happen, but I don't see any fundamental logical reason to rule it out.

For now, however, it looks like the world is best described by differential equations like the Schrödinger equation, which is built on the complex numbers, which in turn are derived from the reals, with all their limits and infinities.  The (provisional) verdict then: the real numbers are real.

* One crude way to see that the rational numbers are countable is to note that there are no more rational numbers than there are pairs of numerator and denominator, each a natural number.    If you can count the pairs of natural numbers, you can count the rational numbers, by leaving out the pairs that have zero as the denominator and the pairs that aren't in lowest terms.  There will still be infinitely many rational numbers, even though you're leaving out an infinite number of (numerator, denominator) pairs, which is just a fun fact of dealing in infinities.  One way to count the pairs of natural numbers is to put them in a grid and count along the diagonals: (0,0), (1,0), (0,1), (2,0), (1,1), (0, 2), (3,0), (2,1), (1,2), (0,3) ... This gets every pair exactly once.

All of this is ignoring negative rational numbers like -5/42 or whatever, but if you like you can weave all those into the list by inserting a pair with a negative numerator after any pair with a non-zero numerator: (0,0), (1,0), (-1,0) (0,1), (2,0), (-2, 0), (1,1), (-1,1) (0, 2), (3,0), (-3, 0) (2,1), (-2, 1), (1,2), (-1,2) (0,3) ... Putting it all together, leaving out the zero denominators and not-in-lowest-terms, you get (0,1), (1,1), (-1, 1),(2,1),(-2,1),(1,2),(-1,2),(3,1),(-3,1),(1,3),(-1,3) ...

Another, much more interesting way of counting the rational numbers is via the Farey Sequence.

Science on a shoestring

On the other blog I would occasionally put out short notices of neat hacks (as always, "hack" in the "solving problems ingeniously" sense).  I recently ran across one that didn't have much to do with the web, so I thought I'd carry that tradition over to this blog.


Muons are subatomic particles similar to electrons but much heavier.  They are generally produced in high-energy interactions in particle accelerators or from cosmic rays slamming into the atmosphere.  Muons at rest take about 2 microseconds to decay, actually a pretty long time for an unstable particle.  Muons from cosmic ray collisions are moving fast enough that they take measurably longer to decay (in our reference frame), which is one of the many pieces of supporting evidence for special relativity.

The GRAPES-3 detector at Ooty in Tamil Nadu, India detects just such decays using an array of detectors set into a hill 2200m (7200 ft) above sea level.  The detectors themselves are made largely from recycled materials, particularly square metal pipes formerly used in construction projects in Japan.  The total annual budget for the project is under $400,000, but the team has already produced significant results.  Auntie has more details on the construction of the instruments here.

There are a couple of narratives that are often spun around stories like this.  One is a sort of condescending "Isn't that cute?" with maybe a reference to the Professor on Gilligan's Island building a radio out of coconuts.  Another is "Look what people can do without huge budgets.  Why do we need all these multi-billion-dollar projects anyway?"

I'd rather not tell either of those.  What I see here is highly skilled scientists making use of the resources they have available to produce significant results.  Their counterparts at CERN or whatever are making use of different resources to produce different significant results.  Both are moving the ball forward.  There have been plenty of neat hacks at CERN, including something called "HTTP",  but today I wanted to call out GRAPES-3, mainly because it's just plain cool.

How natural is nature?

Physics has produced several amazingly elegant theories that reduce a huge variety of phenomena to a few basic causes and concepts.  Even if the basic concepts are just a wee bit math-heavy and the results can be a just a wee bit mind-bending, a great number of important discoveries in physics can be reduced to fairly short descriptions.

• Thermodynamics uses a handful of laws to explain things like why perpetual motion can't happen, how engines work or why Play-Doh™ always ends up looking gray-brown if you mash it together long enough.
• Newton's laws explain things like why the Moon goes around the Earth, how you can tell if a car in an accident was speeding or how to sink the 8-ball in the corner pocket.
• Nöther's theorem demonstrates (in a way I've never quite completely grasped) a deep relation between symmetry and conservation -- if, for example, the equations describing motion don't care about direction then angular momentum is conserved and that figure skater spins faster and faster as the arms come in.
• General relativity holds that, left to themselves, objects travel in a straight line, the simplest possible path.  It just doesn't always look that way because space-time isn't flat, but this is why, for example, Mercury's orbit moves just a bit every time around.
• Quantum physics ... yeah.  Quantum physics.

It's not that quantum physics lacks elegance.  The idea that all matter and energy, basically everything we can measure, can be explained by equations similar in form to those that describe a vibrating string is pretty astounding if you think about it.  The Standard Model of quantum physics has built on this to make a large number of predictions, including predictions of new particles, that have been confirmed with outstanding accuracy.

You'd think this would be good news.  Instead, a certain uneasiness has developed around the Standard Model.  The basic framework is nice enough, but it can't completely describe what we know until you plug in several parameters.  There are 19 in all, ranging from m_e  (the mass of the electron, 511 keV), to θ₂₃ (the "CKM 23-mixing angle", 2.4°) to the recently established m_H, (the Higgs mass, tentatively 125.36±0.41 GeV).  There aren't just infinitely many other ways to tune the knobs, there are not one, not two but 19 knobs to tune.

Tweak a few of them the wrong way and stars can never form, or worse, no kind of solid matter can form at all.  We seem to be in some sort of special regime where the parameters just happen to have the right values for us to be here to observe them.  Even if you adopt the view that there may be infinitely other universes out there where the knobs aren't tuned right, so where else could we be (the "weak anthropic principle"), it's still all pretty unsatisfying.  Our universe is some point in a 19-dimensional space that's suitable for life forms like us to develop?  That's it?


Particle physicist Sabine Hossenfelder  argues in a piece called The LHC “nightmare scenario” has come true that yep, that's it, get over it.  As I read it she makes two points.  The smaller one is that the Large Hadron Collider which was instrumental in finding the Higgs boson has likely found all the particles it's going to find, and maybe it's time to stop trying to build bigger and bigger particle accelerators.

Fellow particle physicist Matt Strassler argues that there's no nightmare regardless of whether there are any other new particles.  The LHC has produced ridiculous amounts of data which won't be thoroughly examined for years, and it can easily produce more.  There might be, indeed probably are, interesting discoveries to be pulled out of that data now that it's pretty well established that the Higgs exists.

This seems reasonable, but it's more an argument against Hossenfelder's headline than the substance of the article.  Disputes over what experiments to do (and, more to the point, what experiments to fund) are by no means new.  Hossenfelder's and Strassler's are by no means the only views on the subject, and they may not even be particularly divergent, but in any case whether to keep building bigger particle smashers is of greatest concern to particle physicists and those who fund them.

Public policy and the sociology of science are worthy topics, but I won't be conjecturing any further about them here.  I'm more interested in Hossenfelder's larger point, which as I understand it is about what makes a good theory of physics.

When people started taking a close look at Newtonian mechanics, heat transfer and other fields they started to find anomalies under extreme conditions that eventually led to the discovery of relativity and quantum physics.  This is just part of a long history of progress in physics.  For example:

• Ptolemy explained the motions of the planets with a system of cycles and epicycles centered around the Earth.
• Copernicus explained those motions more simply with a system of cycles and epicycles centered around the sun.
• Kepler did away with epicycles using the notion that the planets moved in ellipses, not circles
• Newton explained elliptical orbits in terms of a universal gravitational force following an inverse square law
• and Einstein explained gravitation as a property of space-time itself

(I'm always a bit leery about ascribing a particular landmark result to a particular person, as in "Ptolemy explained ...".  There is more to each of these than a single person making a single discovery even when we know a particular person had a particular key insight.  But this will do for now.)

In all these cases, the new theory didn't just explain everything the old theory did, albeit in a new way.  It either made sense of something that had seemed arbitrary in the old theory, explained new things the old theory couldn't, or both.  Copernicus and Kepler dealt with epicycles, first simplifying them and then doing away with them altogether.  Newton's mechanics explained why the planets followed elliptical orbits as described by Kepler's laws and not some other shape.  It also explained why the Moon doesn't actually follow an exactly elliptical orbit, why the daily tides rise and fall, and much more.

Einstein's theory of relativity did away with gravitation as a force.  Objects under the influence of gravity still follow Newton's first law, just in a more subtle form.  It also gave better predictions for the motions of the planets and made a number of new predictions that were later confirmed, such as the direction and frequency of light being affected by gravity and why the orbits of stars in a binary system containing a pulsar can be seen to be slowing.

It's not just that the new theories were more powerful than the old ones.  That's to be expected.  Otherwise why adopt them?  In all these cases, and many others, the new theory was also, in some sense, more elegant than the old.  Elegant, in this sense, largely means simpler.  Fewer epicycles.  One universal force.  No universal force at all.  There is also a sense of reducing seemingly unrelated things to different aspects of the same thing.  The tides and the motions of the planet are both just effects of gravity.  Space and time are just components of a the space-time continuum.

Which brings us back to the Standard Model.

So far no one has come up with a theory-breaking anomaly for the Standard Model analogous to the precession of Mercury's orbit, or some new phenomenon, say an unpredicted particle or force, that the Standard Model could have been expected to predict but didn't.  There are a few candidates, but even after decades of effort nothing has really panned out.  The experiments at the LHC found the Higgs, at an energy consistent with the Standard Model, and nothing, or at least nothing definitive, inconsistent with it.

So the Standard Model is it, right?  We've described the fundamental forces and elementary particles of the natural world.  There's plenty of work, probably an endless amount, to be done working out the ramifications of that, and how it all fits in with relativity, what exactly it means to "measure" a system described by a wavefunction, and on and on, but as to explaining the basis for particle physics, we're done.  Right?

As I understand it, Hossenfelder's answer to that would be "looks like we could be", but that answer doesn't sit well with everyone.  How can such an inelegant theory, with its 19 arbitrary parameters, be the final answer?  "They just do" can't be an adequate answer to "why do those parameters have the values they do?" can it? Hossenfelder would likely say "sure it can".

In the history of physics, power and elegance seem to go hand in hand.  Or at least, after enough anomalies with ad-hoc descriptions turn up, eventually someone comes up with a new framework where it all makes sense again.  The new theory is both more elegant and more powerful.  Some would even say more "natural" and claim that nature is itself elegant, and if it doesn't seem that way we must not understand it properly.

The Standard Model seems ready to be replaced with something better, except it doesn't seem to be producing the sort of "close, but not quite" results that led us from Newton to Einstein.  There may be more elegant theories around -- string theory gets a lot of attention in this regard -- but nothing, so far, clearly more powerful.  If there's a more "natural" theory, nature doesn't seem keen to lead us to it.


This feeling that the world has to be more elegant than our current theories may just be an occupational hazard of physicists, and not necessarily the majority at that.  Plenty of working particle physicists are content to "shut up and calculate" without worrying too much about what it all might "mean" or whether the universe has some deep hidden simplicity.

Many chemists would shake their heads at the whole business.   There are around a hundred elements one can do meaningful chemistry with, each with its own particular properties.  That's not going to change with a new theory of chemistry.  There is a theory, namely the Standard Model, which explains why those elements are the way they are, and quantum effects definitely come into play in chemistry, but from a chemist's point of view it doesn't matter how many parameters the Standard Model has.  It matters what the electrons are going to do in a particular situation.

In my own field there are several models that can define the behavior of computers, and we do refer to them (particularly state machines and stack machines) from time to time, but there is not and is never likely to be a unified theory of software engineering.  And yet the servers still run.  Mostly.

Even mathematics, which can almost be defined as the relentless pursuit of elegance, is full of quirky, inelegant results.  What's so special about manifolds in four dimensions?  Why are there 26 sporadic groups?  Why is the 3N+1 problem so hard?  And let's not even get started on the prime numbers.



Suppose that everything in the universe could be precisely described by three simple rules ... and a table of three quadrillion quadrillion seven-digit numbers.  Even storing such a table would be completely infeasible using today's technology, but suppose we meet up with an alien race with full access to it.  Our best physicists pose them questions, they consult the table and deliver a verifiable answer every time (how to reduce any measurable question and its answer to an invocation of three simple rules is an interesting question, but roll with it).  Would we say the aliens have a good theory?

On the one hand, of course they do.  The hallmark of a good theory is making testable predictions that hold up.  On the other hand, there's something less than satisfying about a planet-sized table of numbers, each essentially its own arbitrary parameter.  What happens if our aliens go away or decide that we're not worthy of True Knowledge?  Maybe we should start asking questions that will reveal the nature of the magic number table and, ideally, allow us to reduce it to something our puny minds and computers can handle.

A good theory doesn't just have to be true in the sense of making true predictions.  It also has to be comprehensible and usable.  To this end, a theory with a thousand fairly simple rules and three or three hundred parameters with values we just have to accept is far better than the one I just described.  But this is not saying anything about nature.  It says something about us.  Our "natural" theories are the ones that work best for us, not just in aligning with nature, but with our resources and the way our minds work.

From that point of view, 19 is not a prohibitive number of parameters the way three quadrillion quadrillion would be.  If that's really how it is, we can probably live with it.  But the distinction is of degree, not kind.  The problem is not with arbitrary parameters themselves, but with having an intractable number of them.  Consulting our hypothetical aliens with knowledge beyond our ability to process is really just another kind of experiment from our point of view.   Consulting the Standard Model with its human-friendly list of parameters is better, and it would be even if its predictions weren't quite as good as they are.  It's certainly better than a more "elegant" theory that doesn't fit experiment as well as it does.

Nature is what it is.  A theory is only "natural" if it fits with our nature in particular as well as nature at large.

[Re-reading this, I realize I neglected to say that, although the basic equation of the standard model is fairly compact -- you can get a T-shirt with the Standard Model Lagrangian on it -- actually finding solutions for all but the simplest conditions is generally far beyond our computing ability.  In one sense this is more than a bit like the aliens-with-the-numbers scenario, but instead of a hidden table of numbers we can't begin to access, we have an equation we can barely begin to compute.  Except maybe with quantum computers ... --D.H.]


Re-reading Hossenfelder's piece, I see one more subtle point.  The main argument doesn't seem to be that there can't possibly be an elegant theory unifying quantum physics with relativity, or even a better way of explaining the results of the Standard Model.  Rather, a search for "elegance" or a "natural" theory is no longer a good way -- if it ever was -- of deciding what particle experiments to run next.  If we do find such a unified theory, it's probably not going to be because we found a more elegant replacement for the Standard Model, or because we found an unexpected particle with a new, more powerful accelerator, but because we found something else entirely and a theory to explain it that happens to subsume the Standard Model.

Primitives

Non sunt multiplicanda entia sine necessitate.

This is one of several formulations of Occam's razor, though Wikipedia informs us that William of Ockham didn't come up with that particular one.  Whatever its origins, Occam's razor comes up again and again in what we like to call "rational inquiry".  In modern science, for example, it's generally expressed along the lines of "Prefer the simplest explanation that fits the known facts".


If you see a broken glass on the floor, it's possible that someone took the glass into a neighboring county, painstakingly broke it into shards, sent the shards overseas by mail, and then had a friend bring them back on a plane and carefully place them on the floor in a plausible arrangement, but most likely the glass just fell and broke.  Only if someone showed you, say, video of the whole wild goose chase might you begin to consider the more complex scenario.

This preference for simple explanations is a major driving force in science.  On the one hand, it motivates a search for simpler explanations of known facts, for example Kepler's idea that planets move around the Sun in ellipses, rather than following circular orbits with epicyclets as Copernicus had held.  On the other hand, new facts can upset the apple cart and lead to a simple explanation giving way to a more complicated revision, for example the discoveries about the behavior of particles and light that eventually led to quantum theory.

But let's go back to the Latin up at the top.  Literally, it means "Entities are not to be multiplied without necessity," and, untangling that a bit, "Don't use more things than you have to", or, to paraphrase Strunk, "Omit needless things".  In the scientific world, the things in question are assumptions, but the same principle applies elsewhere.



The mathematical idea of Boolean Algebra underlies much of the computer science that ultimately powers the machinery that brings you this post to read.  In fact, many programming languages have a data type called "boolean" or something similar.

In the usual Boolean algebra, a value is always either True or False.  You can combine boolean values with several operators, particularly AND, OR and NOT, just as you can combine numbers with operations like multiplication, addition and negation.  These boolean operators mean about what you might think they mean, as we can describe them with truth tables very similar to ordinary multiplication or addition tables:

AND	True	False
True	True	False
False	False	False

OR	True	False
True	True	True
False	True	False

NOT	True	False
	False	True

In other words, A AND B is true exactly when both A and B are true, A OR B is true whenever at least one of the two is true, and NOT A is true exactly when A is false.  Again, about what you might expect.

You can do a lot with just these simple parts.  You can prove things like "A AND NOT A" is always False (something and its opposite can't both be true) and "A OR NOT A" is always True (the "law of the excluded middle": either A or its opposite is true, that is, A is either true or false).

You can break any truth table for any number of variables down into AND, OR and NOT.  For example, if you prefer to say that "or" means "one or the other, but not both", you can define a truth table for "exclusive or" (XOR):

XOR	True	False
True	False	True
False	True	False

If you look at where the True entries are, you can read off what that means in  terms AND, OR and NOT: There's a True where A is True and B is False, that is, A AND NOT B, and one where B is True and A is False, that is, B AND NOT A.  XOR is true when one or the other of those cases hold. They can't both hold at the same time, so it's safe to use ordinary OR to express this: A XOR B = (A AND NOT B) OR (B AND NOT A).  The same procedure works for any truth table.

That whole neutrino thing

The past few days have been rife with headlines of the form "CERN Scientists Claim Neutrinos Travel Faster Than Light."  Sometimes the story underneath explains what's really going on, but too often it veers off into time travel and speculation that Einstein Was Wrong.  In fact, no one at CERN (which produced the neutrinos) or OPERA (which detected them) is claiming any such thing.  As their actual article makes clear, they have a measurement they can't explain and they're asking for help explaining it.

This being a particle physics article, the list of authors takes up the first two pages, but there's plenty of solid information after that.  I haven't read it in detail and, not being a particle physicist, probably wouldn't understand all the detail, but they talk a lot about how they measured both the times and distances involved, including taking into account the movement of the earth's crust and the 2009 Italian earthquake, along with a host of other factors.

They present their data and explain how they analyzed it, but specifically don't make any claims about faster-than-light particles or anything else.  They simply claim they have an anomaly they don't know how to explain and they're still looking into it.  Their precise words:

Despite the large significance of the measurement reported here and the stability of the analysis, the potentially great impact of the result motivates the continuation of our studies in order to investigate possible still unknown systematic effects that could explain the observed anomaly. We deliberately do not attempt any theoretical or phenomenological interpretation of the results.

Statements I've seen in the press from the actual physicists involved reflect this.  A better headline would be "Scientists Can't Explain Neutrino Speed Measurement."

There is ample reason to doubt that OPERA observed neutrinos traveling faster than light.  First, the measurements underpinning special and general relativity are quite solid by now.  Relativity predicts not just that nothing travels faster than light, but a large number of other effects -- for example that clocks run faster in weaker gravity than stronger -- that have been measured to great accuracy.  The odds that those measurements are wrong are very small.  Much more likely that we just haven't found the flaw in the neutrino measurement.

Second, there is strong evidence from astronomy that neutrinos do not travel faster than light.  Supernovae put out both neutrinos and light, and they arrive here at essentially the same time, having travelled for hundreds of thousands of years.  The OPERA anomaly of one part in 4,000 or so would accumulate to 25 years or so over 100,000 years.  In practice, the neutrinos from a supernova do arrive sooner, but only on the order of hours, and astronomers have good reason to believe this is because they leave about that much sooner.  Physicist Matt Strassler has a good summary on his blog Of Particular Significance.

Even if the measurements did hold up, and it turned out that neutrinos can travel faster than the observed speed of light, we're quite a way from time travel.  It might not even be evidence that relativity is wrong.  I've seen speculation that the photon, as we already know the neutrino is, might actually be ever-so-slightly massive.  This would leave relativity's absolute speed limit intact and imply that we just hadn't had the tools to measure the difference between the speeds of photons and the actual upper limit.  I'm not sure I quite buy that that squares with all the observations of light over the last several decades, but I haven't looked at the details (and I'm still not a physicist).

Failing that, it's quite possible that relativity is only mostly right and breaks down in some extreme cases, the same way that Newtonian physics breaks down at extreme speeds and other places.  Who knows?  Such a breakdown might even clear the way for unifying gravitation and quantum mechanics.

But again, no one involved is claiming we're anywhere near that point.

[Prof. Strassler has added a post about the OPERA anomaly.  Among other things, he says that the speed of light not quite being the ultimate speed limit -- that is, not quite the c in e = mc² -- would be a plausible explanation for slightly-faster-than-light particles.  Since he really is a particle physicist, I'm going to bow out and suggest that non-physicists interested in the subject follow his blog (if you are a physicist, I'm sure you already know where to go, but then what are you doing reading this?) -- D.H.][I had originally referred to the "CERN/OPERA" anomaly, but I've changed that.  Although CERN did produce the neutrinos and its name is now associated with the results, it did not conduct the measurements in question. -- D.H.]
[And, of course, it now appears the measurements were wrong. due to a faulty cable.  Kind of anticlimactic, except to two of the project leads involved, who resigned -- D. H.]