Volumes have been written about that question; you can find plenty on this very site, e.g.: http://searchyc.com/nosql
Without rehashing all that, I'll just assure you from my own experience using distributed key/value stores in production as an Amazon.com engineer, that the tradeoff (less expressive data storage engine but simpler scaling/operations/reliability) is often worth it. And many applications are actually simpler to express in a non-relational model.
(And before anyone asks: Yes, for other applications a relational database is a better solution, despite the pain of scaling it and operating it reliably. And I've worked on production systems that used both DHTs and SQL.)
Without rehashing all that, I'll just assure you from my own experience using distributed key/value stores in production as an Amazon.com engineer, that the tradeoff (less expressive data storage engine but simpler scaling/operations/reliability) is often worth it. And many applications are actually simpler to express in a non-relational model.
(And before anyone asks: Yes, for other applications a relational database is a better solution, despite the pain of scaling it and operating it reliably. And I've worked on production systems that used both DHTs and SQL.)