What conversation? You don't have to rate the driver in their presence, and I've never once done so.

Occasionally I'll have a driver demand a 5-star rating, which of course earns them an automatic 1.