I used to use Psion Series 3 "palmtops". One of the best thumb-typing experiences I've had and it ran for days on 2 AA cells.

http://en.wikipedia.org/wiki/Psion_Series_3