Saturday, June 14, 2008

Occam's Razor: Don't claim too much

[Note: if you signed up for the Google Puzzle Championship, it's in two hours, 1 PM EDT!]

I've spoken on Occam's Razor before. And then I offered a different interpretation. If it seems like I'm trying to rationalize and salvage the few justifications for Occam's Razor, it's because I am. I'd like to emphasize that I don't think Occam's Razor is all it's cracked up to be. Its application is rather narrower than people think and its use is best avoided.

But there is yet another interpretation that I want to discuss. I said in a previous post that to do the mathematical calculations in Bayes' theorem, you need what is called the "prior" probability of a claim. The prior probability is the likelihood that the claim is true before we've looked at the evidence.

Estimating the prior probability of a claim is a fundamentally unsolvable problem. Sometimes you're lucky; if the claim is about people, you can simply test a bunch of people to get the probability that it is true for any one person. But if the claim is about the universe, you can't really test a bunch of universes--we can only see one! That's why it's generally a bad idea to give the prior probability an actual number. For an argument to be effective, it should not rely on questionable estimates of prior probabilities.

But sometimes there simply is no effective argument in either direction. What happens then? We start arguing over prior probabilities. We start arguing about how much "sense" the claim makes. The debate devolves into a matter of personal belief and incredulity.

Enter Occam's Razor. As I said before, the claim that "I have an apple" is more likely than the claim "I have an apple and a banana". That's because the latter necessarily implies the former. In general, any claim "A" has a probability equal to the sum of "A and B" and "A and not B". Therefore, by removing any mention of B, we've made our claim more likely. In fact, the claim becomes more likely the more elements you remove from it. The less you say, the less likely you are to be wrong.

Of course, the logical conclusion is that in order to avoid being wrong, the best claim is one with zero elements: pure agnosticism. But such a claim is useless. There is a certain balance here between usefulness and likelihood. To strike this balance, you want to remove any unnecessary elements from your claims and keep the necessary ones.

Some people try to compare claims by simply counting up the number of elements. This is a useful move, but it doesn't follow logically from the above. And how can you really count the number of "elements"? What constitutes a single "element"? While these questions are not unanswerable, they are a major obstacle.


Bad said...

I think you're really confusing the issue here. The Razor is not relevant to bare factual claims or observances, as you are treating it: not about the likelihood of apple vs apple+banana. The Razor is about explanations and the idea that we should make do with as few extra assumptions as possible. It's general statement is "do not multiply [explanatory] entities without need." In other words, if you can explain the evidence with known facts, that's to be favored over supposing the intercession of a mysterious conspiracy.

But the Razor still requires you explain the things in question. If you apply it to you apple banana question, it's not whether it's more likely that you have an apple or an apple and a banana. It's that, given that I can SEE an apple and banana in your hand, I can most easily explain this by the fact that you do have both fruit in your hand... and not that you have an apple, but that the banana is an illusion put there by a demon.

miller said...

Bad, I don't disagree. I realize I am being very harsh to Occam's Razor. This is because I am looking for a justification to use Occam's Razor in cases where there is little evidence on either side, and Occam's Razor is your only tool. Of course, if you see a banana, it's probably real rather than an illusion, but what about cases where you have little to no evidence of the banana?

As for explanations vs claims, I am merely looking at it from an unusual angle. In my previous interpretations, I talked about explanations; here I talk about claims. There is no single canonical way to interpret Occam's Razor.

Bad said...

The point though, is that the Razor was never meant to be used simply to decide between two bare possibilities. It's statement is not "prefer the simplest." It's "do not invent extraneous entities for stuff you can already explain without them."