Here's a strawman I'm sure you've seen before:
Why all this extra process? Shouldn't we trust people to do the right thing? Wouldn't you rather people do the right thing because it's the right thing to do? People are mostly doing the right thing anyway! Trust in the hacker spirit to triumph!
The question is, is this an objection to copyleft, or is it an objection to code of conducts? I've seen objections raised to both that go along these lines. I think there's little coincidence, since both of them are objections to added process which define (and provide enforcement mechanisms for) doing the right thing.
Note that I don't think copyleft and code of conducts are exactly the same thing. They aren't, and the things they try to prevent are probably not proportional.
But I do think that there's an argument that achieving real world social justice involves a certain amount of process, laying the ground for what's permitted and isn't, and (if you have to, but hopefully you don't) a specified direction for requiring compliance with that correct behavior.
Curiously we also have people who are pro copyleft and strongly anti code of conduct, and the reverse. Maybe examining the parallels between objections to both might help identify that a supporter of one might consider that the other makes sense, too.
Update: This was originally posted to the pumpiverse and since cross-posting to my blog, at least one interesting response to it has been made there.
Another update: Sumana Harihareswara pointed out that not only had we probably talked about this when we hung out in May, but that she had previously written on the subject even before then! I had forgotten about reading this (I forget about a lot of things), though honestly, my post is probably a reflection of Sumana's original thoughts. She goes into more detail (and probably much better) than I did here... you should read her writing on the subject! It's good stuff!