The problem is that Bitcoin is a distributed network, and therefore the whole blockchain needs to be replicated across all the computers across all the network.
Right now, the blockchain is around 9GB, most of it because of "dust" - that is, if you want to set up your own Bitcoin node, you need to download 9GB of data. And this is growing quite fast.
The practice of microtransactions was abused a lot - for example, SatoshiDice uses 1 satoshi (the minimum amount) transactions to confirm that a player lost. Or some other guys used the blockchain to embed WikiLeaks documents using the same hacks - right now everyone needs to download all this data.
tl;dr; it's not about caching and speed, it's about the amount of storage required to keep all the dust forever and ever.
Also, from what I gather from the commit - this is supposed to be just a temporary solution, and there is something better in order.
How can Bitcoin network maintain integrity in a imaginary future where it has gained significant traction and there are millions or even more transactions happening every single second?
Actually, it was recently stated by a developer on #bitcoin-dev that it is an issue of the total set of unspent transaction outputs (i.e., what all nodes must maintain in memory), not total blockchain size (which only nodes that need to do complete validation actually need).
Of course, it may be that all the developers are not of one mind on this.
Right now, the blockchain is around 9GB, most of it because of "dust" - that is, if you want to set up your own Bitcoin node, you need to download 9GB of data. And this is growing quite fast.
The practice of microtransactions was abused a lot - for example, SatoshiDice uses 1 satoshi (the minimum amount) transactions to confirm that a player lost. Or some other guys used the blockchain to embed WikiLeaks documents using the same hacks - right now everyone needs to download all this data.
tl;dr; it's not about caching and speed, it's about the amount of storage required to keep all the dust forever and ever.
Also, from what I gather from the commit - this is supposed to be just a temporary solution, and there is something better in order.