For me the axis is not computer quality, it's IDE quality. My productivity in a large codebase jumped 10x when I switched from emacs to IntelliJ (for Java). Going back would be like blindfolding myself after I grew up blind and got my eyesight restored once.
Remote development is tricky indeed. Many people run the IDE locally and use its remote development functionality. I run it remotely over VNC.
I've been wanting to try out remote developing for multiple years now, it would allow me to get an iPad with a perfect battery life or an Macbook Air that would tag along all day without charging. But because I cannot work with vim/emacs and my IDEs (IntelliJ) do not support anything like remote development I am stuck with a huge Macbook Pro including a battery draining IDE that cannot be ran remotely. VNC added too much lag for me to use all day.
Latency is a problem indeed, I am lucky to be close to work and have massive bandwidth all the way to my dev box. Without that the experience is pretty poor, and I used to make it work by dropping bit depth to the minimum bearable level.
Remote development is tricky indeed. Many people run the IDE locally and use its remote development functionality. I run it remotely over VNC.