I think it's something related to the curse of dimensionality [1] [2], basically just a property of high dimensional spaces (perhaps only certain kinds of spaces though).
Even though almost every all pairs of points are almost a full diameter away from each other, they are also almost all almost orthogonal (i.e. the angle they make with the center of the sphere is very close to 90 degrees).
The intrinsic dimensionality of a dataset is also relevant here.
The M-Tree is one of my favorite indexes. It works with data that's embedded in infinite dimensional spaces (sometimes; it's bumping up against an impossibility result that's sketched in a sibling comment).
[1] https://en.wikipedia.org/wiki/Curse_of_dimensionality
[2] http://kops.uni-konstanz.de/bitstream/handle/123456789/5715/...