If the standards were more strict, some of these issues would not exist. I see this as exploiting a lot of slop in protocols. It should not be possible to interpret a URL as anything but a URL, yet here it's being reflected back and interpreted as something else entirely.
It has nothing to do with the standards being strict; the standards can be as strict as they want. If the standards are strict and useless, no one will follow them, instead implementing something less strict and more useful.
For example, when downloading a file from a website, what default name should you use for it? There is a header to tell you, but not ever page supplies such a header; so the browser needs to do something. It chooses to pick the last component of the URL as that filename. However, URLs are somewhat more complex than you might expect, so this becomes more complicated and can lead to attacker controlled ways to manipulate this filename.
Now, you could make a more strict spec, for example by forbidding downloading files unless the filename is properly specified, or forbidding using any kind of default filename and making the user choose it themselves, or something of the sort. But if any browser vendor implemented this more strict spec, they would instantly annoy a lot of users who would find things breaking that used to work, and they would be likely to switch to another more permissive browser.
Security, compatibility, and robustness are hard factors to balance. Just blaming this on "slop in protocols" is a vast over simplification.
>> For example, when downloading a file from a website, what default name should you use for it? There is a header to tell you, but not ever page supplies such a header; so the browser needs to do something. It chooses to pick the last component of the URL as that filename.
Yeah, and that's slop in the protocol. If the header was required everything would still work, web sites would just have to fill in the header. What's easier to do, comply with a protocol where your site brakes if you don't, or to have swiss cheese and then make site developers learn a bunch of security best practices and hope they get it right?
Also in there is the good old "this site wants to blah blah" and ask the user to decide. If you have to ask, the answer is "No! fix your site so it's not on the user to decide". Broken certificates? Not my problem, browser should just say "sorry site security is busted" and leave it at that. It's an old debate, but AFIAC there is no debate, only lazyness.
