Powers of 2

The relationship between probability and information is interesting and fun.  The table below is a work in progress, but I think it’s kind of cool already.  The idea is to compare information quantities with the probabilities of unlikely events.  For instance, if I flipped a coin for every bit on my hard drive and they all came up heads, it would be pretty unlikely.  But what else would be that improbable?

The leftmost column has quantities of information, ranging from one bit upwards.  The next column is the same thing, but in Joules per Kelvin, the historical physical unit of entropy.  Then there’s an illustrative example from the world of computing.  The next column is a probability.  This can be thought of as the probability of flipping heads on N coins, where N is the number of bits, though if you’ve studied information theory you’ll know the connection between information and probability is somewhat deeper than that.  Finally, there’s a bunch of examples of other things that have a similar probability.

It’d be pretty cool to make this table more complete.  I’d quite like to have every power of two on the left-most column, all the way from one bit up to around , and even beyond.  The main problem is thinking of interesting rare events with the appropriate probability – so if you think of any, let me know in the comments and I’ll see if I can fit them into a later version of this table.  (You don’t have to do the maths, though if you can it would help.)

Bits	Physical units	Computer storage capacity	Probability	Event with approximately the same probability
1 bit		1 bit	1/2	flip heads on a coin.
2 bits			1/4	flip heads on 2 coins, or roll four on a 4-sided die
8 bits (1 byte)		One ASCII character	1/256	flip heads on 8 coins (1/256), roll three sixes on three dice (1/216), or die in an accident* (1/215)
16 bits (2 bytes)		A 16-bit integer	1/65536	die by being bitten by a mammal other than a dog* (1/54794), match 5 numbers in the UK national lottery, which wins about £1,500 (1/55491), or be dealt a straight flush (1/64974)
32 bits (4 bytes)		A 32-bit floating point number	1/4,294,967,296	get dealt two straight flushes in a row** (1/4,198,262,436), be the next person in the world to blink (1/6,800,000,000)
64 bits (8 bytes)		A 64-bit double-precision number		You win the UK national lottery and your house gets hit by a meteorite in the same week
160 bits		An SHA-1 hash		An SHA-1 collision occurs between a given pair of files; or get struck by lightning on your birthday 6 years in a row***; or get dealt 10 straight flushes in a row.
8192 bits (1 kilobyte)		1/16th of a Sinclair ZX’s Spectrum’s RAM		Win the UK national lottery every week for six and a half years ****
bits (125 gigabytes)		Small-ish modern hard drive		A fully functioning E. coli cell assembles by chance in an inert mixture of chemical elements with no energy supply*****
bits (so big there isn’t a word for it)		If this were ever to be built it would take up a sizeable fraction of the visible Universe		Roger Penrose’s (probably rather silly) estimate of the probability of a universe being created with similar physical properties to ours.5
bits		The information capacity of our universe – this would take up the entire visible universe.  It’s as big as you can get, probably.		The probability of finding yourself in our exact universe, as opposed to one of the many others that are compatible with the laws of physics as we currently understand them.6

* cause of death probabilities are based on the 2009 totals from this UK data set, and should be taken with a large pinch of salt.  So should everything else on this page, in fact!

** more accurately, this is the probability of drawing two straight flushes if you’re dealt exactly two hands, which is less probable than just getting two in a row if you were playing poker all night (though that’s still pretty astronomically unlikely).

*** this site estimates the odds of being struck by lightning to be roughly 1 in 280000 per year.  I divided this by 365 and raised to the power of 6.  This is a bit of a sketchy calculation because there are quite a few things I didn’t take into account, including 1. you’re quite likely to die if you get struck by lightning, reducing your chances of being struck again (though my grandfather said he survived being hit three times); 2. some people are more susceptible to being struck than others (my Grandad was a farmer, hence he was often the highest point in a field); and 3. storms are more likely at some times of the year than others, which increases the probability of this happening.  I’m hoping these all roughly cancel each other out.

**** of course, is actually 51 times smaller than — but these numbers are so incredibly small that for many purposes even a few dozen orders of magnitude can be neglected.

***** life didn’t actually start this way, of course – it probably started with something quite a lot simpler than a cell, and there most certainly was an energy supply – so the actual probability of life forming under the conditions of the early Earth was almost certainly a lot higher.  I reckon it was probably pretty much 1.  The probability of spontaneous formation of an E. coli cell under these unlikely circumstances was estimated as  by Morowitz, 1968.

⁵ Penrose’s calculation would make much more sense if it was supposed to be the probability of a Universe like ours forming due to a thermal fluctuation in an equilibrium Universe that lacks a low-entropy past.  The probability is so small that it hardly makes a difference whether you put a 10 or a 2 at the beginning.

⁶ a reasonable limit on the information capacity of the Universe is the entropy of the de Sitter horizon, which is an event horizon that surrounds the observable universe like a giant inside-out black hole.  Blindly throwing the area of the de Sitter horizon into the black hole entropy formula gives an information capacity of the order bits, and that’s what I’m taking as the maximum number of bits that could fit into the Universe.  (But note that this number is actually increasing – it must do so more-or-less linearly as the universe expands exponentially due to “dark energy”.  It’d be interesting to know the rate of its increase, but I haven’t worked that out yet.)