If everything looks one-sided...

Earlier in the comments of The Uncredible Hallq, the topic was same-sex marriage, and someone was saying, "Policy Debates Should Not Appear One-Sided."  The point being that if same-sex marriage looked like such a one-sided issue, then we're obviously biased.

My reaction was, "Possibly true, definitely unhelpful."

I put "Policy Debates Should Not Appear One-Sided" in quotes, because this is the title of a well-known piece on Less Wrong.  In brief, you might expect issues of fact to be one-sided, because most of the strongest evidence will converge (though by chance there may be some weak contrary evidence).  But issues of policy generally should not be one-sided.*  Any proposed policy has costs and benefits.  The good policies are the ones where the benefits outweigh the costs.  But it would be very strange if it appeared that a policy literally had no costs whatsoever.  If policy issues look one-sided, then we must be biased.

*I recall one time Leah Libresco said "Arguments shouldn't look one-sided" in the context of atheism vs Catholicism.  This was an incorrect application of the idea, since atheism vs Catholicism is analogous to an issue of fact rather than an issue of policy.

I agree with the general point, but is this really something that can used in any specific argument?  As I've said before, accusing your opponent of cognitive bias is a pretty shitty argument, even if it's true.  I mean, we all know that we have our own cognitive biases.  But that doesn't mean that any particular belief is wrong.

The argument is made worse by the fact that "Policy Debates Should Not Appear One-Sided" is not a universal rule.  You'd expect that most policies would have costs and benefits, but who is to say that this is true of any particular policy?  One could imagine, for instance, an issue of policy that reduces to a single issue of fact.  Or you could imagine an issue of policy that only has a few distinct effects, so it's not so outlandish that the effects could all happen to be positive.  Or you could imagine an issue of policy which has one effect which is so big and important, that it's not really necessary to consider the other smaller effects.

Even when you're just trying to evaluate a policy for yourself, rather than trying to argue with someone else, "Policy Debates Should Not Appear One-Sided" still seems unhelpful.  If a policy debate looks one-sided, what are you supposed to do about it?  You can search for new evidence and arguments that oppose you.  But if you've already seen the opposing arguments and found them wanting, that's that.  There's no point in trying to fill a quota of disadvantages to your own side, just because you have the prior expectation that your side should have disadvantages.  You might as well just believe your prior expectations and ignore evidence.