A modified prisoner's dilemma simulation

Recap

Over a year ago, I wrote a series of posts about the evolution of iterated prisoner's dilemma strategies.  Since it was so long ago, I will briefly recap, although reading the previous posts may be necessary.

1. I explained what the iterated prisoner's dilemma game is.  Then I program a simple evolutionary simulation where there is a population of individuals who play iterated prisoner's dilemma games against each other.  Rather than intelligently choosing strategies, the individuals blindly follow strategies determined by their "genes".


2. I explain an important scientific paper which demonstrates the existence of so-called "Zero-determinant strategies" (ZD strategies).  These strategies allow one player to unilaterally determine the score of their opponent, or unilaterally enforce a linear relationship between their opponent's score and their own.  The paper claims that this should lead to extortionate strategies in evolution.


3. I explain a response to the ZD paper.  In some sense, ZD strategies may "win", but this does not mean they are evolutionarily stable.  The paper explains conditions for evolutionary stability, and shows that several key ZD strategies are not evolutionarily stable.

In this, the final post of the series, I will modify my evolutionary simulation in light of what I learned from the papers.  I actually modified the simulation a long time ago, but I stalled while trying to figure out what parameters to use.  It was difficult to find a mutation rate which was low enough to allow for population stability, but high enough that the simulation would operate in a reasonable amount of time.

My primary conclusion, therefore, is that researchers in this field have their work cut out for them.  I could see using a supercomputer on this problem, and more fully exploring different variations.  But I'm not trying to contribute to the field, I'm just doing this for fun, and to better understand the field.

Modifications in the simulation

Here are the things I changed in my simulation in light of what I read in the literature:

• Previously, each individual was specified with three parameters.  Now I use four, because these individuals remember not only whether their opponents cooperated or defected in the previous round, but also remember what they themselves did.


• Previously, I calculated the average outcome after ten iterations of the prisoner's dilemma.  Using mathematical techniques from the literature, it now calculates the average outcome after an infinite number of iterations.  This modification eliminates the need to know what individuals do in the first iteration.  Also, the calculations don't work with "pure" strategies, so every individual always has at least 0.1% chance of cooperation and 0.1% chance of defection.


• I switched to a payoff matrix of 5/3/1/0, which is the standard in literature.


• Previously, the individual with the highest score reproduced, and the individual with the lowest score died.  In the literature, the chance of reproduction is proportional to score, and a random individual dies.  I found that the reproduction process makes a huge difference, because it's the difference between trying to be better and trying to be the best.  I changed it so that reproduction rate is proportional to score, and death rate is proportional to five minus the score.

I note that the literature seems to prefer a different mutation process than what I used.  I had individuals constantly mutating their parameters by small amounts.  They instead gave each individual a small chance of mutating a single parameter to a random value.  Both methods make sense.  If behaviors are controlled by many genes, there will be a large chance of small mutations; if behaviors are controlled by just a few genes, there will be a small chance of large mutations. I tried both ways, but stuck with my method because it was easier to make the simulation stable.

The results

The following comes from a population of 40 (which is far smaller than the populations used in the literature).  Each generation, each player plays against two randomly chosen opponents.  Then reproduction and death is assigned by the scores, and the genes of each individual are mutated by a random number between -0.005 and 0.005.

A million generations are shown.  It takes about seven minutes to run.

Here I plot values as they change over the generations.  The thick black line shows the average score (right axis).  The other four lines show the probability (left axis) of cooperating given the outcome of the previous iteration.  For example the CC line shows the probability of me cooperating if we both cooperated in the previous iteration.  The CD line shows the probability of me cooperating if in the last iteration I cooperated and you defected.  All of these values vary through the population, but what's shown are the population averages.

You could be forgiven for thinking that it all looks like a bunch of noise.  This is basically why my simulation stalled.  Is it all just noise from genetic drift?  Do I need a lower mutation rate or larger population?  Or am I biasing results by changing the parameters until the noise is reduced to my satisfaction?  I'm not paid enough to figure it out.

But you can see a few strategies showing up over and over:

• Defection.  All colored lines are near zero, and the average score is near 1.

• General cooperation.  In the literature, evolutionary simulations converge on the so-called general cooperation strategy, (0.935, 0.229, 0.266, 0.42).  My simulation is a bit different and not very stable, but you can see similar results whenever the green line is high and the other colored lines are low.

• Tit-for-tat.  This is the (1,0,1,0) strategy, when the green and orange lines are high, and the others are low.  It only appears a couple times, but seems stable.

 

• Other hybrid cooperation strategies (green plus blue, or green plus aqua) occur repeatedly, and generally precede a descent from cooperation to defection.  If I were to give this a narrative, it's sort of like the population goes soft on crime, and then the criminals take over?

•  I find myself very puzzled by the aqua only strategy, which occurs twice.  I call it the "submission" strategy, since it means that if you cooperate and your opponent defects, then you may just choose to cooperate again.  I don't understand how this could possibly be evolutionarily stable.  This was the sort of thing that made me wonder if my simulation was being wonky.  But here it is:

I also wanted to do a quick test to see how much the different strategies resemble ZD strategies.  I didn't figure out how to define resemblance though, so that was a dead end.

In summary, you can see the results are complicated, and that's why you have to pay researchers money instead of making them work for free.

Extra: Battle of the Sexes

My boyfriend was really curious what happens if you try the same simulation in an iterated Battle of the Sexes game.  In Battle of the Sexes, there is a married couple, and the husband wants to go to a monster truck rally, and the wife wants to go to the opera, or something ridiculously gendered.  Anyway, each person prefers to get their own choice, but going together is still preferable to going separately.

This is pretty easy to do with the same script I already wrote, I just need to replace the payoff matrix 5/3/1/0 with 5/0/0/3.

This is largely an afterthought, so I'm just running it for 100,000 generations.  The left axis shows the probability of giving in, given different outcomes of the previous iteration.

(ETA: I realized the legend is ambiguous.  "My choice" represents the probability that I will give in if in the previous iteration both me and my spouse went to my preferred event.  "Your choice" is the probability that I will give in if in the previous iteration both me and my spouse went to my spouse's preferred event.)

That was really straightforward.  If you're playing an infinite number of iterations of Battle of the Sexes, the evolutionarily stable strategy is to mostly avoid giving in.  But once somebody finally does give in, they continue to give in.  It's funny that the average score hovers around 3, instead of 4.  I guess it's common to have evolutionary players who just never give in.

I suppose this supports the idea that the Iterated Prisoner's Dilemma is really complicated, and it's not just that my simulation is noisy.

---------------------------------------------

This is the last part of a miniseries on the evolution of Prisoner's Dilemma strategies.
1. Evolution of prisoner's dilemma strategies
2. Extortionate strategies in Prisoner's dilemma
3. Prisoner's Dilemma and evolutionary stability
4. A modified prisoner's dilemma simulation

A little welfare math

In an earlier post, I mentioned an argument that welfare decreases wages. The idea is that employers don't need to pay as much, since the wages they provide will be supplemented by welfare. However, in a simple analysis, welfare clearly increases wages. It could be that the simple analysis is wrong (as I showed is the case for a simple analysis of minimum wage), but the simple analysis should still be a beginning for understanding.

Since this is only a personal blog, I did just the bare minimum of research into US welfare systems. I immediately learned that US welfare is shit. I can't believe how little safety net there is in this country. Here are some basic facts:

• The major programs are Social Security, Medicaid, TANF, and SNAP. I ignored the first two because those are more complicated and conditional. 
• TANF (Temporary Assistance for Needy Families) is for extremely poor families with children. For example, a household of four people in Indiana must have income under 36% of the poverty level, and the maximum payment is 17% of the poverty level (and that's assuming your income is zero). 
• Additionally, TANF has a lifetime limit of 5 years, and only 2 years without a job. 
• TANF websites are horrible and most of them won't give the relevant information in an accessible manner. 
• SNAP (Supplemental Nutrition Assistance Program, or food stamps) applies to a much broader class of people, and pays more. For a household of four, you must have gross income under $2552 a month to qualify (or 130% of the poverty level), and you are allotted $632 (with 30% of your "net income" deducted). 
• People in the SNAP program must get a job within three months.

For the sake of analysis, I will take a simplified SNAP as a model. Let's just say that we have a four-person household, and the payments are $630 a month minus 25% of gross income.

The first thing you should notice is that SNAP effectively creates a 25% tax on the first $2520 of your household's income. Already we can see why welfare should increase wages. Workers will demand more money for the same amount of labor, because they're only seeing 75% of that money. Of course, there's a tradeoff between wages and employment. If workers only see 75% of their wages, more of these workers may choose not to get a job at all. Or employers may choose not to hire them in light of the higher wages.

This is all complicated by SNAP's requirement that people get a job. Now that may decrease wages, if it motivates people to look for some job, any job, in order to maintain eligibility.

There's a second way that welfare can increase wages: the decreasing marginal utility of money. I'm making these numbers up, but say you need $100 a month to eat, and $200 a month to eat well. The first $100 prevents you from starving, whereas the next $100 only prevents you from hating your food. If there is no welfare, you might be willing to work a shit job for low pay because it's still better than starving. If you're already getting $100 a month from welfare, you would demand more money for your labor.

------------------------------------------

The rest of this post will do a simple mathematical analysis of welfare. You must have javascript enabled in order for MathJax to render the *\LaTeX* equations.

In a simple analysis, the number of low-wage workers employed and the amount of wage they earned is determined by the intersection of supply and demand curves. The demand curve is the number of workers that employers are willing to hire. The supply curve is the number of people willing to work for a certain wage.

 

In panel a I show typical supply and demand curves. However, there are arguments that the demand curve for low-wage workers is unusual in that it has "zero elasticity" (panel b) or "negative elasticity" (panel c). These arguments are complicated and controversial, so I'm just going to ignore the demand curve for now.

Welfare affects the utility function of workers. Normally, we'd just say

**U = L w - c(L)**

 where U is the utility function, L is the fraction of laborers employed, w is the wage, and c(L) is the total cost of all those people working. c(L) is basically a measure of people's subjective preferences. But now the total earnings of each worker is the wage plus welfare, which we'll call t(w). So we have

**U = L t(w) + (1-L) t(0) - c(L)**

 A further complication is the decreasing marginal utility of money discussed previously. If u(w) is the utility of wage w, the the total utility function is

**U = L u(t(w)) + (1-L) u(t(0)) - c(L)**

Here I plot the t(w), u(w), and u(t(w)) that I used. t(w) is wages plus the simplified SNAP payment described previously. We know very little about u(w), so I just used *u(w) = Log(\frac{w + $500}{$500})*.

The supply curve is defined by the point where the marginal worker no longer benefits from working. In other words, it's the point where U no longer increases with L. The derived supply curve is

**u(t(w)) = u(t(0)) + \frac{dc}{dL}(L)**

 Below I show the supply curves before and after applying welfare (assuming *\frac{dc}{dL}(L)* is just proportional to L).

So as far as I can tell, the idea that welfare is "subsidizing" employers makes no sense. Welfare appears to increase wages.  The number of workers may increase or decrease, depending on the elasticity of the demand.

There are a million caveats and complications here, and you should not trust the basic analysis.  For instance, note that the wages payed by the employer are not the same as the profit earned by the employees, since employees miss out on the welfare they could have gotten by being unemployed.  This is just the effective 25% tax from welfare.

One way to get rid of the effective tax is to guarantee everyone a base income. That is, give everyone $630 a month, regardless of their other earnings. This would still raise wages, because of decreasing marginal utility. But then we'd probably have to worry about inflation. I'm not even gonna go there.

I couldn't find any studies (that I understood) on the impact of welfare on wages, so I have no way of knowing whether any of my conclusions are correct or incorrect. I think, however, it was good to learn about our welfare system. If it took a math problem to motivate me to do a bit of research, it was worth it.

More literal spaghetti

Yesterday, I wrote about the spaghetti metaphor for Everettian Quantum Mechanics (aka Many Worlds).  Unfortunately, the metaphor breaks down because different worlds in quantum mechanics can constructively and destructively interfere with each other on a microscopic level, although we generally don't need to worry about this on a macroscopic level.

But there is another interpretation of quantum mechanics for which the spaghetti metaphor is more exact.  I'm speaking of the Bohmian interpretation of quantum mechanics.

The Bohmian interpretation is usually the go-to example for how we can have a deterministic theory of quantum mechanics.  In Bohmian theory, every particle has a well-defined trajectory, and only occupies one position at any given time.  The fact that we can't predict exactly where the particle will be just has to do with the fact that we do not (and cannot) know with certainty the particle's initial position.  Bohmian theory makes all the same predictions as the other major interpretations of quantum mechanics.  But this comes at a cost: faster-than-light information transfer.

I'm largely a Many Worlds partisan, but I give the Bohmian interpretation credit, because it basically starts with the Many Worlds interpretation.  In Bohmian theory there is no wavefunction collapse.  Instead, the wavefunction simply evolves according a single equation, and splits into many worlds just like in the Many Worlds interpretation.

The difference is that the wavefunction is not interpreted as a description of reality (and therefore there aren't really many worlds).  While the wavefunction is a real object, it is seen as distinct from all the particles we see around us.  The wavefunction is interpreted as a pilot wave which merely guides the motion of particles.*  All particles have a definite position, and we just need is this complicated pilot wave object to determine their motion.

*For those who have studied quantum mechanics, it's actually quite simple to understand.  Even in standard quantum mechanics, we speak of the probability current.  We can obtain the probability "velocity" by dividing the probability current by the probability density.  Bohmian theory interprets this literally, by having the particle velocity equal the probability velocity.

To relate this to the spaghetti metaphor, let me consider the classic double slit experiment.  We send light through two slits, and the waves coming from the two slits interfere with each other.

 Waves of light go through two slits, located at the bottom, and travel upwards.  (Technical details: I'm just showing the real part of the wavefunction, with blue positive, red negative, and green zero.)

At some points, the waves interfere constructively, and at other points they interfere destructively.   This creates alternating dark and light fringes.  In more ordinary quantum interpretations, this is because the wavefunction of the light determines the probability that the light is in any particular location.

 The colors here show the probability that light is in any particular location.

But in the Bohmian interpretation, we do not interpret the wavefunction as probability.  Instead, we interpret it as a pilot wave.  The light goes on a well-defined trajectory, it's just hard to predict which particular trajectory it's on.

Here I show many possible trajectories for the light, as calculated using Bohmian theory.  As you can see, particles don't respect conservation of momentum, and even though light only goes through one slit, it is obviously affected by the presence of the other slit.  (You can find lots of similar images on the net, but this whole post is really an excuse for me to write a Bohmian calculator, so I'm giving you my image.)

Perhaps now you can see how this follows the spaghetti metaphor.  Each possible trajectory for the light is a single strand of spaghetti.  When we speak of probabilities in quantum mechanics, we're really talking about our degree of belief that we are in any particular strand of spaghetti.

The difference is that in the Bohmian interpretation, there is only one strand of spaghetti, the one that includes us.  And in my Literal Spaghetti interpretation, there are many strands of spaghetti, of which we are just one.

An atonal earworm

I have a fondness for 20th century classical music, because that's when composers really start to peel off the layers of conventional tonality.  That said, I am not sure I'm a fan of twelve-tone serialism, which is the musical movement that went the farthest in the quest to defeat tonality.

Serialist composers would constrain their music such that it would go through every one of the twelve tones before coming back to the first one.  The sequence of twelve tones is called a "tone row".  This was intended to prevent it from being in any particular key.  For example you couldn't say it's in the key of C, because C is only ever played within a tone row with other equally important notes.  But oddly, this constraint often isn't sufficient to defeat tonality.  Even when you have a random sequence of twelve notes, our minds tend to pick out some pattern, and fit it into a key.

Mathemusician Vi Hart has a great video about serialism, which includes a few serialist compositions that are intentionally tonal:



Vi Hart talks a bit about The Owl and the Pussycat, a song by Igor Stravinsky.  But she doesn't play it because it's copyrighted.  But there are other people are willing to violate copyright, so here it is:



But before I move on, I have to talk about these silly lyrics. I was disappointed to learn that Stravinsky didn't write them himself.  Instead, they come from a poem by Edward Lear:

The Owl and the Pussy-cat went to sea

   In a beautiful pea-green boat,

They took some honey, and plenty of money,

   Wrapped up in a five-pound note.

The Owl looked up to the stars above,

   And sang to a small guitar,

"O lovely Pussy! O Pussy, my love,

    What a beautiful Pussy you are,

         You are,

         You are!

What a beautiful Pussy you are!"

The rest of the poem describes how the owl and pussycat buy a ring from a pig and get married by a turkey.  Mysteries in the poem abound:  How can they spend a year and a day searching for a ring right after the pussycat delivers the line, "too long we have tarried"?  Why do they only get one ring: which of the two will wear it?   What is a runcible spoon?  I thought the incongruities were just too absurd, until I remembered that "rockabye baby" is a lullaby about a baby falling to its death.

As for Stravinsky's music, it sounds like all the notes are wrong.  But I may have an unusual opinion: I think it is catchy.  As in, I literally caught the singer's melody in my head.  I had a serialist earworm.

This raises the question of whether my earworm accurately reflects the song.  Is it truly an atonal earworm?  Or is my mind interpreting the music as being in a particular key?

This can be tested with an experiment!  A couple days after listening to the song, I transcribed it from memory.  Then I transcribed the original song, almost finishing the first stanza before giving up.  Here are the results:

Each line represents one of Stravinsky's tone rows.  In blue is Stravinsky's tone row (I ignore repeated notes for simplicity).  The other colors show the corresponding notes that were in my earworm.

Perhaps unsurprisingly, my earworm is full of inaccuracies.  Besides getting a bunch of notes just wrong, there are also a few sections that were shifted up or down by an interval.  There are also some missing parts.  I hadn't at all remembered the line, "And sang to a small guitar."  And lastly, my earworm gets completely derailed in the last line ("What a beautiful pussy...").

The last line is different from the others, because it's not really a tone row.  The note F is repeated three times!  D# and F# are also repeated.  Perhaps Stravinsky didn't follow tone rows strictly.  Or I transcribed the notes wrong (transcription is hard).  Or the singer sung the wrong notes (who would know?).  In any case, there's one part of the line that stands out as having conventional tonality: the sequence C#, A#, B ("-ssy you are!").  This is called an "authentic cadence", and it tends to establish a key of B.

The authentic cadence is such a strong structure that it appeared in my earworm, albeit shifted to the key of D.  And then it seems like the entire line got derailed into a key of D major.

No wonder my earworm is so inaccurate in this spot.

This leads me to conclude that while some parts of my earworm roughly represent the original, there's also a tendency for my mind to substitute the atonal melody with a tonal one.  This is more likely to occur as we get further into the song, since I tend to have a poorer memory of those parts.

I wonder if there is any research on how well earworms tend to reflect the songs that they come from.  I daresay that I would have a much easier time transcribing Lady Gaga from memory.

Evolution of prisoner's dilemma strategies

While I was in France, I couldn't spend all my time going to the beach and talking about physics.  I mean, sometimes the beach had too many jellyfish.  And we had no internet for a time.

Our arch-nemesis, the jellyfish

So I might have spent an hour or two programming an evolutionary simulation of Prisoner's Dilemma strategies.  The inspiration was that my boyfriend described to me an awful article on BBC news which completely distorted a paper (update: corrected link) on evolutionarily stable strategies for the iterated prisoner's dilemma.

This is all based on multiple hearsay, since I have read neither the news article nor the paper.  So my evolutionary simulation may be completely off the mark.  It's okay, this is just for fun.  Later I will look at the paper and compare what they did to what I did.

Introduction

I assume readers are familiar with the prisoner's dilemma.  The prisoner's dilemma contains the basic problem of altruism.  If we all cooperate, we all get better results.  But as far as the individual is concerned, it's better not to cooperate (ie to defect).  It gets more complicated when two players play multiple games of prisoner's dilemma in a row.  In this case, even a purely selfish individual may wish to cooperate, or else in later iterations of the game, their opponent might punish them for defecting.

The iterated prisoner's dilemma can be analyzed to death with economics and philosophy.  But here I am interested in analyzing it from an evolutionary perspective.  We imagine that individuals, rather than intelligently choosing whether to cooperate or defect, blindly follow a strategy determined by their genes.  I parametrize each person's genes with three numbers:

x: The probability of cooperating if in the previous iteration, the opponent cooperated.
y: The probability of cooperating if in the previous iteration, the opponent defected.
z: The probability of cooperating in the first iteration

I believe everything is better with visual representation, so here is a visual representation of the genespace:

Each square represents an individual.  Each individual is described by three numbers x, y, and z, which are visually represented by the position and color of the square.  The major strategies of the iterated prisoner's dilemma are associated with different quadrants of the space.  In the upper right quadrant is the cooperative strategy.  In the lower left quadrant is the defecting strategy.  In the lower right, is the "tit for tat" strategy, wherein you cooperate if your opponent cooperate, and defect if your opponent defects.  The upper left quadrant... I don't think there's a name for it.

And of course, there are hybrid strategies.  For example, between cooperation and "Tit for Tat", there's "Tit for Tat with forgiveness".  In the evolutionary algorithm, the only way to get from one strategy to another is by accumulating mutations from generation to generation, so the hybrid strategies are important.

The simulation

I tried to perform the evolutionary simulation in the simplest way possible.  Start with some number of individuals (here I use 20 individuals).  Have each pair of individuals play an iterated prisoner's dilemma (here I used 10 iterations).¹ Afterwards, I total up all the scores, and remove the individual with the worst score while duplicating the one with the best score.  Then every individual mutates by randomly making small changes in x, y, and z.

Upon testing, I found that this simulation favors defection far too much.  The trouble is that it's really a zero sum game.  There is no absolute standard for what sort of scores are good or bad, it's just a comparison between the scores of different individuals.  Even if defecting doesn't benefit you directly, it's still beneficial to hurt your opponents so that you come out ahead.

I'm not really sure how to solve this problem, and I'm very interested to see how researchers solve it.  I tried several things, and settled for a simple modification.  Rather than playing against every other player, each individual will play against two random individuals.²  It's still beneficial to hurt your opponents, but not that beneficial since you are only hurting a few of them.  It's also possible that an individual will play against itself, and by defecting hurt itself.

I believe that researchers typically use the following payoffs: mutual cooperation is 3 points, mutual defection is 1 point, and if only one player defects then they get 5 points while the cooperating player gets 0.   However, I wanted to find some interesting behavior, so I adjusted the parameters to better balance the cooperation and defection strategies.  I eventually settled on 4/3/1/0 instead of 5/3/1/0.

Results

Interestingly, rather than there being a single preferred strategy, multiple strategies can be evolutionarily stable.  Actually, this may not be so surprising, since I basically adjusted the parameters of the simulation until I saw interesting behavior.  But it's still fun to watch.

Often the population will settle into the defection strategy and not move for a thousand generations.  But sometimes there is a shift to a cooperative tit-for-tat hybrid strategy.  This hybrid strategy is not very stable, and always seems to eventually fall back to the defection strategy.  Here I show an example of this happening:

Why does this happen?  I'm not quite sure.  But I think that when everyone is defecting, there's no real disadvantage to taking a tit-for-tat strategy.  So genetic drift will sometimes cause the population to shift towards tit-for-tat.  Once there is a critical mass of tit-for-tat, it becomes advantageous to cooperate, since you will get back what you paid for.  Soon everyone is cooperating or using tit-for-tat.  But then, perhaps by genetic drift, we lose too many tit-for-tat individuals, and it becomes advantageous to defect again.  And so begins the cycle again.

Here I also show the average scores of the population over time:

If the population is defecting, then the average score will be 1.  If it is cooperating, the average score will be 3.  Here you see a punctuated equilibrium behavior.

Of course, you cannot draw any real conclusions from this simulation, since I did a lot of fine-tuning of the parameters, and because I'm not really interacting with the established literature.  The conclusion is that this is an interesting topic, and now I feel motivated to read some papers.  When I do, I'll report back.

--------------------------------
1.  Actually I don't have individuals really play an iterated prisoner's dilemma against each other.  Instead, I calculate the average result of an iterated prisoner's dilemma between those two players.  There is no chance involved in this calculation.
2. This means that each individual plays four games on average.  Two games where the opponent is chosen randomly, and two games where that individual was chosen as the opponent.

This is part of a miniseries on the evolution of Prisoner's Dilemma strategies.
1. Evolution of prisoner's dilemma strategies
2. Extortionate strategies in Prisoner's dilemma
3. Prisoner's Dilemma and evolutionary stability
4. A modified prisoner's dilemma simulation

How well did Nate Silver do?

The news is saying that Nate Silver (who does election predictions at FiveThirtyEight) got fifty states out of fifty. It's being reported as a victory of math nerds over pundits.

In my humble opinion, getting 50 out of 50 is somewhat meaningless. A lot of those states weren't exactly swing states! And if he gets some of them wrong, that doesn't mean his probabilistic predictions were wrong. Likewise, if he gets them right, that doesn't mean he was right.

I thought it would be more informative to look at Nate Silver's state-by-state predictions of Obama's vote share. That way, Nate could be wrong even in states like California.  So here's what I did for each state: I took the difference between the final prediction of FiveThirtyEight, and the vote share reported by Google this morning.  Then I divided this difference by Nate's margin of error.  See the results in a histogram below.

What the figure shows is that Nate's predictions were more accurate than Nate himself claimed!

The mean of the actual distribution is -0.14, which means that Obama did slightly worse than Nate predicted, but by an amount that can be explained by random error.  The standard deviation of the distribution is 0.5, which means that Nate predicted an error that was twice the actual error.

Of course, Nate's reported error is likely due to expected systematic error.  For example, if all states were slightly more in favor of Obama, that would be a systematic error.  Assuming that Nate Silver predicted a spread of 0.5, he must have expected a systematic error of about 0.85 in one direction or the other.

Humanists on race

I'm interested in determining how well (secular) humanists do social justice.  Many people want to support social justice, but whether their actions and positions are really in favor of social justice is a different question.  Therefore, I proposed that I look at humanists' expressed opinions on a particular concrete social justice issue: colorblindness.

So here's my definition: Colorblindness is the view that the solution to racism is to immediately stop making any distinctions based on race or ethnicity.  It is the view that the only racism worth fighting is the explicit kind, rather than the systemic, structural kind.

In my opinion, colorblindness is counterproductive, because it leads to ignoring problems rather than solving them.  I am far from alone in this view (here's one example).

I will conduct this sort of like a controlled experiment, not because I'm being all precise and scientific.  Judging whether something is expressing colorblindness is hopelessly subjective.  I'm imitating experimental procedure because it is more fun that way, and because it has the best chance of changing my mind (since, after all, I will agree with my own subjective judgments).

The procedure: I will search for race and ethnicity on The Humanist (a magazine by the American Humanist Association), the Council for Secular Humanism (CSH) website, and the AtheismPlus forums (for control).  I will select articles at random from each site, and try to categorize them as "The colorblind", "Colorblindness rejectors", and "other".  This will not be just based on explicit positions, but implied ones.

The Humanist

After trying a few search terms (race, racial, racism, ethnicity, ethnic), I decided that The Humanist's search engine is not very good.  I settled on "racial", because it seemed to have the most meaningful results.  There were 10 results, and I picked 5 of them with a random number generator.  One article only used the word "racial" once, without discussion so I replaced that one with another random one.

The colorblind:
None

Colorblindness rejectors:
1. The Humanist interviewed Gloria Steinem, and in my opinion she indirectly rejected colorblind ideology several times.  For instance, she talks about how black women are more likely to be feminist; this is inconsistent with the view that black and white makes no difference.
2&3. Two articles are by Sikivu Hutchinson, who I already know would oppose colorblindness.  I think she founded the Black Skeptics group.
4. In an article about Thomas Jefferson, the author says, "As a society, we routinely deplore racial violence and say we are not prejudiced, but racism still exists."  Sounds like an explicit rejection!

Other:
1. A review of Less Than Human talks about race, but I don't think it mentions colorblindness in any way.

The Council for Secular Humanism

A search for "Racial" gave me 100 results!  I tried picking them at random, with little success.  I looked at 15 articles, and only one seemed relevant.  It seems that many writers like to list race alongside other things like class, ethnicity, and sex, without elaboration.  Since this wasn't working, I picked the last articles by looking at those with the top relevance scores (skipping 1 which I felt was not actually relevant).

The colorblind:
1. One article compares anti-atheism to historical racism, and I got a colorblind vibe based on what things it emphasized.  When anti-atheism is compared to racism, it's only the discrimination aspect of anti-atheism that comes up.  It's also sort of triumphalist about the nearing end of racism, speculating that "anti-atheism may well go the way of racism."
2. A manifesto by Paul Kurtz (founder of CSH) calls for "a new global ethics that transcends the ancient religious, ethnic, national, and racial differences of the past."  It asks us to "rise above parochial national and multicultural perspectives."  It also recommends "concrete practical reforms to achieve these aims", none of which include explicitly addressing race.  I am not sure how Kurtz intends to transcend issues of race without addressing them explicitly.

Colorblindness rejectors:
1. A short article credits the "growth of justice" to "the boldness of minorities that have demanded rights instead of merely bowing to commandments."  That sounds a bit like a rejection, since the author thinks speaking up, rather than ignoring differences, solves problems.
2. An article announces the founding and endorsement of African Americans for Humanism.  I consider creating specialized groups to be a rejection of colorblindness.

Other:
1. An article laments the "regressive" view that we should censor speech " deemed offensive and hurtful to presumptively vulnerable or historically oppressed groups".

AtheismPlus

Because AtheismPlus is a forum, rather than a series of articles, I can't use exactly the same procedure. Instead, I searched for "racial", and got 76 posts. I randomly picked a few posts, and looked at the pages containing them.  I tried to enumerate the number of unique users who seemed to be espousing colorblindness, vs those who were not.  As usual, I ignored a few threads which came up in the search but I felt were irrelevant.

1. Does "We are all Africans" Co-opt Black Identity?: For the sake of this experiment, I won't take a position on that question.  But some people criticized the statement for colorblindness.  I counted 4 who explicitly rejected colorblindness, and 11 who didn't explicitly mention it.
2. How to be more than a "White People's Movement": One commenter endorsed colorblindness, saying "As long [AtheismPlus] is inclusive and openly so, then there is no problem," and, "instead of thinking of it as a white people's club, just think of it as a people's club."  11 unique commenters made a point to disagree with this.  Only a couple commenters I wasn't sure about.

Conclusions (TL;DR)

Because of my prior negative view of Humanist organizations, I thought that they would take the naive, colorblind stance towards racism.  I was mostly wrong.  The Humanist takes a very progressive stance.  CSH has a somewhat regressive stance (and coming from the founder too), but perhaps this is a characteristic of CSH rather than humanism in general.  AtheismPlus got highest marks, as expected.

Perhaps- perhaps I was wrong about humanism.  When I reflect back on why I have such a negative view of humanism, one of the things I recall was reading an article by Paul Kurtz which I deeply disagreed with.  Maybe I just had a problem with Paul Kurtz and CSH!  What an epiphany.

The game theory of profiling

Previously, I read a paper that mathematically modeled the search for a malfeasor using profiling techniques.  I concluded that the model does not apply to security against airplane hijackers (though it may apply to other security situations).  You know what, I could build a better model than that!  I'm a physicist, dammit!

This comic seemed appropriate.

So here's my model.  Suppose that there is a small minority of fliers who are "marked" as especially suspicious.  (In Harris vs Schneier, the marked group are Muslims, though Schneier points out that being Muslim usually isn't a visible characteristic, and you'd have to use some proxy, such as "Arab-looking".)  There is a fixed percentage of fliers that airport security can search, but if they like they can choose to search people in the marked group more often.

Security's adversaries are the terrorists.  They have a certain amount of recruitment resources, which they can use to recruit hijackers in the marked group, or outside the marked group.  However, if they recruit outside the marked group, it costs more resources, and thus they can recruit fewer people.

Security plays to minimize the number of successful attacks.  To do this, they must minimize the average number of hijackers not searched.  Terrorists play to maximize the number of successful attacks.  The question: How much should security focus on searching the marked group, and how much should the Terrorists focus on recruiting from the marked group?

Parameters in this model:

m = the ratio of the number of people in the marked group to the number of everyone else

λ = the percentage of fliers that security can search

c = the ratio of the cost of recruiting a hijacker from the marked group to the cost of recruiting elsewhere

Assumptions I will make:

m is very small (the marked group is a very small minority)

λ > m (security can search every single person in the marked minority if they want)

λ < 1/(1+m) (security cannot search every single person outside of the marked group)

c < 1 (It is cheaper to recruit hijackers in the marked minority)

The number of hijackers the terrorists can recruit is smaller than the number of people in the minority.

Lastly, I will ignore the fact that people come in discrete quantities.

The game:

Both airport security and terrorists have a choice to make.  And it's not one of those either/or choices, they have a whole sliding scale to choose from.  Let y be the position of the sliding scale for airport security, and let x be the position of the sliding scale for terrorists.

Legend text: "Percentage of minority searched"; "Percentage of other people searched"; "Hijackers recruited from minority"; "Hijackers recruited elsewhere".  n is the maximum number of hijackers that can be recruited, but this parameter does not figure into the solution.

x and y are each numbers between 0 and 1.  The greater x is, the more the terrorists focus on recruiting from the marked group.  The greater y is, the more airport security focuses on recruiting from the marked group.

The number of successful attacks is a function of x and y.  I will represent this function as the height in the following graph:

Terrorists control the x coordinate, and want to maximize the height.  Airport security controls the y coordinate, and wants to minimize the height.  Assuming both players are rational[1], and know each other to be rational, they will choose the single Nash equilibrium.  This is a saddle point, which I've marked in the above graph.

Results:

I could give you the coordinates of the solution[2], but this would be meaningless because x and y are just abstract quantities.  Here are some more meaningful quantities.

Click for larger

Yes, that means the terrorists should be equitable in their recruitment process.  Even if it is easier to recruit from the marked group, they should still make no effort to focus on the marked group.  This solution is exact[3].

The results for the airport security, on the other hand, are approximations only valid for small m.  Basically, security should search a percentage of marked people such that the terrorists get just as much bang for their buck regardless of where they recruit.

If I were to plug in "reasonable" numbers, I would say c = 3/4, and λ = 1/5.  With these numbers, airport security should search 2/5 of people in the marked group, and 1/5 of everyone else.

Applicability of this analysis:

This analysis is inapplicable, because it ignores the additional risk and cost associated with determining a profiling scheme, determining the parameters, and training security personnel to implement it.  There are probably other complications too.  Lastly, it assumes that airport security is rational. So maybe applications are a lost cause, but at least I got to do some math.

[1]This is "rational" in the game theory sense, which is frequently irrational in the colloquial sense.

[2]The exact coordinates are ( cm/(1+cm) , 1 - c(1-λ)(1+m)/(1+cm) ).

[3]That is to say, it does not use the assumption that m is very small. However, it does use the other assumptions.

Profiling: a math experiment

In an earlier post, I summarized a paper that used a simple mathematical model of profiling to determine the best strategy.  I also argued that this paper was too idealized to apply to real-world situations, and it does not apply to the case of airplane hijackers at all.

Here I will expand on the paper just slightly.  The paper still won't apply to real-world situations, and still won't apply to airplane hijackers.  Basically this is just math for its own sake.

Here is a re-summary of the paper's model:  We have a large number of people with one unknown malfeasor.  Different people have different prior probabilities of being the malfeasor, and the government knows these probabilities.  The government must search people one by one, but has no memory of who it has searched before.  Therefore, the government must sample people randomly with predetermined sampling probabilities.

The paper chooses the strategy which minimizes the mean number of searches before the government catches the malfeasor.  However, I think it makes more sense to maximize the probability of catching the malfeasor with N searches.  This is a subtle difference, but it completely changes the math.

Math ahoy!

Let p_i be the prior probability that person i is the malfeasor.  Let q_i be the probability that the government will search person i in any given search.  The goal is to choose q_i, given p_i so that we maximize the probability of catching the malfeasor within N searches.  Let M be the total number of people.  We'll assume that M and N are very large numbers.

Consider the case where person j is the malfeasor.  The probability of not catching them in any given search is (1-q_j).  The probability of not catching them at all in N searches is (1-q_j)^N.  This is not quite our probability of failure, because we still have to average over all possible people who could be the malfeasor.  And when we do the average, we have to weight the average by the probabilities p_j.  Here is our probability of failure:

But it's not as easy as picking the q_j which minimize this figure.  q_j must also obey additional constraints (eg their sum total is 1).  Fortunately, there is a technique to account for these constraints.  It's called the Lagrange multiplier method, and I don't have the space here to explain it.  Here's a solution:

Oh no, I chased away all my readers with a scary equation!  But it gets worse... this is only sometimes the solution.*  In certain situations, the above result will give negative values for q_j.  The physical meaning of this is that sometimes there are people who are best ignored completely, even if there is a small chance they are the malfeasor.  Naturally, the larger number of searches you can make, the fewer people who should be ignored.

A simple case

Suppose that we have two groups of people, one which composes 99% of the population, and the other which composes 1%.  And suppose that any given person in this 1% is 100 times as likely to be a malfeasor as any given person in the 99%.  Finally, let's say that the number of searches is equal to the number of people (but recall that we can't prevent redundant searching).  What should the sampling probabilities be?

The answer: You should check on people in the 1% more often by a factor of about 5.6.**  (This is proportional to the log of 100.)  Since there are far more people in the 99%, you should be performing 5.6 searches on the 1% for every 99 searches on the 99%.

This is only one particular limit of a very complicated solution, and you cannot draw more general conclusions from the results of this case.  I tried a bunch of things, and one general trend is that the larger number of searches you can make, the more equitable your searching should be.

Sweet FSM, this was useless, but it sure was fun.  There's one more post on profiling math coming.

*Details for those understand the math: Applying Lagrange multipliers is somewhat tricky because I don't know which constraints to apply.  Some q_j will have values of 0, bumping up right against the constraint that all q_j must be positive.  But for other q_j, the constraint is irrelevant.  The form of the solution depends on which constraints apply, and which don't.  Mathematicians might have a better way of handling this, but I can only handle it by "guessing" which constraints are important.
**The general formula, when N and M are very large, is ( N + M(1-K)Ln(a/b) )/( N - M K Ln(a/b) ), where K is the fraction of the population in group A, a is the prior probability that a person in A is a malfeasor, and b is the prior probability that a person not in group A is a malfeasor.  Unless I made an error.

Science vs Acne: followup

Some time last year, I talked about doing "an affectionate parody of science" by half-assing an experiment with acne treatments.

On one side of my back, I tried using hot pads.  On the other side I used some acne medication.

So it turns out that hot pads might be effective, but they're too much work!  I'd have to heat up some water, then pour it on the pad, and hold it against my back for the small window of time when the pad was neither too hot, nor too cold.  This window was shorter than three minutes, so I'd have to repeat a few times.  And then I'd have to do it again for each pad-sized unit of area on my back.  In practice this meant that I gave up on hot pads entirely after a few weeks.

The acne medication, on the other hand, appears moderately more effective than hot pads (or no treatment).  The medication I was using was tea tree oil.  That sounded really dubious to me, so I didn't expect it to work, but it did.  Apparently, it's a legitimate acne medication, shown to be about as effective as benzoyl peroxide.

As I said before, I drank some "skin detox" tea so that I could later find some excuse to credit the tea instead of crediting the actual acne treatment.  It seems hard to believe that drinking this tea could continue to have an effect for two months, (but only on the side of my back with the tea tree oil, and not during the winter break when I stopped using tea tree oil).  But the facts speak for themselves!  A single empirical observation trumps all the theoretical arguments in the world.  It's herbal tea, man.  Goodness is real nature, or so the tea told me.

More seriously, I drink a lot of herbal tea, but not because I think it's healthy.  Technically speaking, herbal tea is just tea that's not made from the herb Camellia sinensis.  It's made from other plants, such as chamomile or chrysanthemum.  Chrysanthemum tea is great.  In an alternate universe, we would call chrysanthemum tea tea, and all other teas herbal teas.

Math-ive attack

Warning: this post is part of my quest to find increasingly ridiculous excuses to talk about math.  But first, some music.



When listening to Massive Attack's song "Future Proof", the part that stands out to me are those beeping sounds that continue through the entire song.  Most of the time, the beeps cycle quasi-randomly between three pitches in some kind of inscrutable arpeggio.

So the question is, does the sequence of pitches actually come from a random number generator, or does it just sound random because it goes by so quickly?

I wouldn't put it past artists to use randomness in their music.  I swear, some of the other bands I listen to must compose lyrics by pulling words out of a hat.  And if Wikipedia is to be believed, there is a long history of using chance in musical compositions, and the practice is known as aleatoric music.

On the other hand, humans are terrible at generating and recognizing randomness.  Maybe the composer wrote a sequence of pitches that seemed random to them, but which does not resemble a typical randomly generated sequence.  Or maybe it's not even meant to sound random, but sounds random anyway because humans are so terrible at recognizing it.

Perhaps this can be cleared up if I transcribe a small segment of the music.  The following represents the sequence of pitches starting at 3:35, until the chord change at 3:41:

3223 2132 1232 1132 1232 1323 2123 2132

(Note: If a note is of double length, I'm transcribing it as a repeated note.  I can't guarantee that my transcription is free of errors.)

Now that we have a transcription, it's an open-and-shut case.  This sequence was probably not created by a random number generator.  If it were randomly generated, you would expect about a third of the numbers (give or take a few) would be followed by a duplicate.  In other words, there should be a lot more double-length notes, and probably even some triple-length notes.  Here is a randomly generated sequence for comparison:

1131 2131 3133 3121 3133 1112 3311 1213

Observe that there are three doubles and three triples.  Massive Attack's sequence, on the other hand, only has two doubles.

You might also observe that the randomly generated sequence has sixteen 3s, four 2s, and twelve 1s.  This may seem strange, since you would expect each digit to appear about ten or eleven times.  But that's just the way randomness goes sometimes.  But if we accept that randomness sometimes produces outliers, shouldn't we also accept the possibility that randomness produced Massive Attack's sequence even though it includes only two doubles?

The answer is that we are using trickier reasoning here than it may first appear.  A true random number generator is equally likely to produce any sequence of 32 digits.  A random number generator is no more likely to produce the sequence I showed than it is to produce Massive Attack's sequence.  Our reasoning really has to do with what sequences are most likely to be produced by a human.

And the thing is, we don't really know the probability that a human will generate any given sequence.  There are 3^32 sequences, and it's just not possible to collect that many statistics.  So the first thing we do is we classify those sequences by some simple property.*  For example, I chose to classify the sequences by the number of doubles and triples.  The idea is that instead of 3^32 different sequences, we just keep track of the set of sequences with one double, the set of sequences with two doubles, and so forth.  It's (somewhat) well-known that when humans try to imitate random number generators, they tend to underestimate the typical frequency of doubles and triples.  So if a sequence has relatively few doubles and triples, that tends to support the hypothesis that it was a human imitating randomness.

*This is much like the way we group microstates together into macrostates in order to define entropy. [/physics]

Note that we can come up with more hypotheses to explain the sequence.  For example, perhaps they used a random number generator, but ignored most repeats.  Or they could have used a random number generator in some other way.  This would be difficult to disprove.  However, I believe in a fourth hypothesis, which is that it's meant to sound wandering and mysterious, but is not meant to imitate randomness.  I observe that the transcribed sequence has three copies of the sequence 3212321, which is the kind of pattern that seems very unlikely to be produced by a random number generator, but much less unlikely to appear in deliberately composed music.

Science vs Acne

I have pretty bad back acne.  No, I will not show you photos.  This is not that kind of blog.

I don't really care about my acne.  I've shown it scientifically!  In the past when I've tried acne medication, my conclusion was that I don't have the motivation to apply the medication on a regular basis, and that medication applied on an irregular basis is ineffective.

But I have new motivation.  I have what I'm worried is another abscess.  After how painful it was last time, I'm taking someone's suggestion that I apply hot packs to kill the bacteria before it grows big.  As long as I'm boiling this water, I might as well kill some acne!

Of course, I'm not actually going to do any science.  I am participating in what has been called the "coffeeshop fallacy" (via The Thinker).  I like the idea of doing science, but I'm not actually willing to put in the effort.  I'm a PhD student, and I have real science to occupy my time!  So what I'm actually going to do here is an affectionate parody of science, whatever amuses me.

The hot pack idea comes from Brian Dunning.  He cites a study which says you can treat acne by applying 120 farenheit for three minutes twice a day.  There's no way I will use such stringent protocols.  Brian suggested using a laptop power adapter, but I'm going to use a rag with a bit of boiling water poured on it.  I can use the extra boiling water for tea!

The other day I had some Yogi tea called "skin detox".  On the label, it said:

Goodness should become human nature because it is real nature

This so that I can credit the tea later if I get rid of the acne.

I'm taking another idea from XKCD:



On one side of my back, I'll apply the hot pack.  On the other side, I may just try some acne medication.  Then I will ask my boyfriend to say which side looks better, without telling him which is which.  Actually, I will probably use really shoddy blinding, and he'll find out which side is which.  But he will appreciate the excuse for me to be topless.

And my excuse for writing about the experiment before it's done is to avoid reporting bias.  As you know, scientists are less likely to report negative results than positive ones, which can lead to systematic errors.  Therefore, if I never write about this again, you may assume that the results were negative, or that I lost all motivation.

One: the universe's favorite digit

Out of all the digits, from zero to nine, one is the most common.  This has to do with the log scale.

The log scale captures an important fact that is true of many quantities in life.  Take money for instance.  If you have one dollar, then earning another dollar is great because you've doubled your money!  If you have a million dollars, earning another dollar does not make much of a difference.  Small changes matter less the more you already have.

This is true on a log scale too.  On a log scale, 1 is the same distance from 2 as 100 is from 200.  The higher you go up, the more the numbers all get smooshed together.  What does that mean for the digits from zero to nine?

In the above picture, I show a log scale.  And on that scale, I highlighted in blue all the regions where 1 is the first digit of the number.  You should see that the blue regions cover more than one tenth of the log scale.  In fact, they cover about 30%.  And so, if we pick numbers randomly on the log scale, about 30% of those numbers will have 1 as their first digit.

Just for fun, let's apply this concept on the fundamental constants of nature.  I will compare two hypotheses:

1. The fundamental constants of nature are distributed on a log scale.  About 30% of the constants will have 1 as their first digit, 18% will have 2 as their first digit, and so on.
2. Each digit from 1 to 9 is equally likely to be the first digit of the fundamental constants of nature.

Of course, it could be that both hypotheses are wrong.  It's difficult to say that these constants are really random.  To talk about randomness, we'd need a whole collection of universes with different sets of fundamental constants so we can do some statistical analysis.  But clearly we have only one universe.  Luckily this one universe has 26 fundamental constants (as far as we know), which may be a big enough collection of numbers to do some statistical analysis.

The caveat is that there is more than one way to choose our 26 fundamental constants!  For instance, the mass of the u quark in planck units could be considered a fundamental constant.  But instead of this constant, we could use a different constant: the ratio of the mass of the u quark to the mass of the electron.  It's possible I could bias my results by "choosing" the fundamental constants that confirm my hypothesis.

Therefore, I will not choose which constants to use.  I will simply use the list compiled by John Baez and David Black.*  Note that 6 of the constants are unknown, so we'll have to make do with the remaining 20.

*Baez offers two sets of equivalent constants for the CKM matrix.  I just used the first set.

Results

The frequency of first digits more resembles hypothesis 1 than hypothesis 2.  I also tried doing some Bayesian analysis.  If our prior belief is that the likelihood of each hypothesis is 1:1, then this evidence increases the ratio to 15.8:1.  In other words, if we previously thought they were equally likely, then after seeing this evidence, hypothesis 1 is almost 16 times as likely.  As far as evidence goes, this is fairly weak evidence, but you're not going to get much better with only 20 fundamental constants.

Does this really mean fundamental constants are randomly distributed on a log scale?  Gee, I don't know.  Probably not really.  What's your interpretation?

Wealth distribution: a model

The above chart shows the United States wealth distribution, alongside with what people think the wealth distribution is, and what people think it should be.  It comes from a paper by Norton and Ariely earlier this year, and it's made the rounds in the news.

That's pretty interesting, but I always find myself wondering how to interpret this.  Is the wealth distribution unfair, or are people just clueless about wealth distribution?  Probably both.  I feel pretty clueless about it myself.

To be honest, the very way the data is presented seems unintuitive to me.  I mean, I'm a physicist, so what I consider "intuitive" is all out of whack.  But seriously, the wealth owned by the top 20%?  I have very little sense of what to expect from such a number.  It has to be somewhere between 20% and 100%.  84% seems like too much, but what do I know?

If I were to pick a wealth distribution, I would start with a model, and from there calculate the percentage owned by the wealthiest 20%.  So here's my model: wealth has a normal distribution on a log scale.*  People who are one standard deviation above the mean own N times more than the median.  People who are one standard deviation below own N times less than the median.  N is a number that we can choose.  For graphing purposes, I am choosing N=2.718 (Euler's number).

In the above graph, the median wealth is 1, meaning that half of people earn more than that and half earn less.  But the mean wealth is actually more than that, since the distribution is skewed.  And the most common amount of wealth (called the "mode") is about 40% of the median.

The next step is to translate this to the amount of wealth owned by the top 20%.  So that's what I did for a few values of N:

(Note that though my plot has similar colors to the one at top, they aren't exactly the same since I split the top 20% into three groups.)

I showed N=6.5 because that's the number that seems to correspond to reality, according to Norton & Ariely.  I showed N=2.718, because that's the number I would have guessed if I had never seen the real data.  I showed N=1.5, because that's what strikes me as a "fair" distribution.  In other words, I would think it was fair if people who are one standard deviation above the median own 50% more wealth.  But in reality, they earn more than six times as much wealth.

I was surprised how similar my bar graph is to the one in Norton & Ariely.  I'm quite sure that most people answering this poll have no understanding of normal distributions or log scales, and I was all set to conclude that people are clueless.  I'm surprised to find that I agree with the popular opinion, because I think N=1.5 seems ideal.

Of course, the "ideal" value of N is completely arbitrary.  What do you think is an ideal value of N?  If you like, I can calculate the resulting wealth distribution.

*Some technical details: A normal distribution means that the probability density is exp[-y²/(2*log(N)²)].  But here, y is not the amount of wealth, but the log of the wealth.  When I transform from a log scale to a linear scale, the probability density becomes exp[-log(x)²/(2*log(N)²)]*1/x, where x is the ratio of the wealth to the median wealth.  This is the function I have plotted in the graph.  This is, by the way, just about the simplest model imaginable.