Current GPUs have a comparable number of transistors (92.2 billion in the current NVidia Blackwell according to https://chipsandcheese.com/p/blackwell-nvidias-massive-gpu) to the number of neurons in human brains (about 90 billion according to Wikipedia). Brains consume less energy and do more, though transistors beat them on density. This suggests there are alternative pathways to performing computation that will scale better.
Let me restate. The article is musing about medium terms difficulties on the current pathway for producing computation. I'm musing that perhaps we are on the wrong pathway for producing computation.
That's the best we have, and not for lack of trying. There are attempts at analog inference (like Mythic) but it doesn't look as they are going anywhere yet.
