Volumes have been written about that question; you can find plenty on this very site, e.g.: http://searchyc.com/nosql
Without rehashing all that, I'll just assure you from my own experience using distributed key/value stores in production as an Amazon.com engineer, that the tradeoff (less expressive data storage engine but simpler scaling/operations/reliability) is often worth it. And many applications are actually simpler to express in a non-relational model.
(And before anyone asks: Yes, for other applications a relational database is a better solution, despite the pain of scaling it and operating it reliably. And I've worked on production systems that used both DHTs and SQL.)
