A private key is not required to validate certs (private keys are for generating certs). A cert chain back to a trusted root cert is required for validation. A phishing site would just trust all certs.
I realize that now. One way to mitigate it would be to have the browser somehow tie a given client certificate with a particular website. That is, the client cert for news.ycombinator.com, would only be presented if I try to connect with that server and nothing else.
That way, if I go to a phishing website that pretended to be Hacker News, my client certificate would not be sent and my browser could warn me by saying that the connection is not using a client certificate. Right now, if we only rely on server side certificates, there's nothing stopping a phishing website from using Let's Encrypt to show the secure connection icon in the URL bar and tricking me into thinking it's a legitimate server.