The Two Envelope Paradox
In 1993, when I was a graduate student, fellow student Josh Dever introduced me to a simple puzzle in decision theory called the "exchange problem" or the "two envelope paradox". It got under my skin.
You are presented with the choice between two envelopes, Envelope A and Envelope B. You know that one envelope has half as much money as the other, but you don't know which has more. Arbitrarily, you choose Envelope A. Then you think to yourself, should I switch to Envelope B instead? There's a 50-50 chance it has twice as much money and a 50-50 chance it has half as much money. And since double or nothing is a fair bet, double or half should be more than fair! Using the tools of formal decision theory, you might call "X" the amount of money in Envelope A and then calculate the expectation of switching as (.5)*.5(X) + (.5)*2X = 5/4 X. So you switch.
Of course that's an absurd result. You have no reason to expect more from Envelope B. Parity of reasoning -- calling "Y" the amount in Envelope B -- would yield the result that you should expect more from Envelope A. Something has gone wrong.
But what exactly has gone wrong? I've never seen a satisfying answer to this question. Various authors, like Frank Jackson and Richard Jeffrey, have proposed constraints on the use of variables in the expectation formula, constraints that would prevent the fallacious reasoning above. However such constraints are impractically strong, since they would also forbid intuitively valid forms of reasoning such as: If I have to choose between (i.) a gift from Person A and (ii.) a coinflip determining whether I get a gift from Person B or Person C, and I believe that Person A would, on average, give me about twice as much money as Person B and half as much as Person C, I should take option (ii).
Terry Horgan and Charles Chihara have proposed less formal contraints on the use of variables in such cases, constraints that I find difficult to interpret and which I'm not sure would consistently forbid fallacious calculations (for example in non-linear cases).
Many mathematicians and decision theorists have written interestingly on what happens after you open the envelope and see an amount. For example, could there be a probability distribution according to which no matter what amount you see, you should switch? That's a fun question, but I'm interested in the closed-envelope case, in diagnosing what is wrong in the simple reasoning above. No one, I think, has got the diagnosis right.
For Josh Dever's and my stab at a solution, see here (simplified version) or here (more detailed version).
For a list of on-line essays on this topic, see this Wikipedia entry. (This entry gives Josh and me credit for "the most common solution" -- does this mean that our unorthodoxy has become the new orthodoxy? -- and then shifts focus to the open envelope version.)