I can't down-vote anyone. I was actually quite disappointed when I first read that you can down-vote above some karma threshold.
Text/turn-based commenting system already ensure that one has to 'wait' for the other to finish their writing. Down-voting only seems like shouting louder and thus seems unnecessary. The only reasonable excuse is SPAM prevention. For that, however, I think that automated flagging system, that won't influence points, would do just fine. I also try to up-vote any opinions that I do not agree with, but that argument their point well.
Long story short, I think down-voting is unnecessary and even potentially harmful. I don't comment often, so it's not going to happen anytime soon, however I plan to not use my down-vote privileges, once I get them, for any reason at all.
(Actually, I think I'll create greasemonkey script to ensure I won't downvote anyone accidentally.)