To be fair, iOS offers no capability to list the browsers a user has installed on their phone. So they have to assume Safari, which is a fair assumption for 90% of users, I imagine.
It's kind of a double joke. The machine is dumb because it doesn't remember what language you chose. The second part is that if his language suddenly changes, it's probably not him. Removing the withdrawal limit allows the person with his stolen credentials to get more money.
If you had a Chase card they could use some proprietary way of saving the language, but for other banks there's no standard way of saving the language (on the card for example), so the only solution is for the ATMs to maintain an online (since you're unlikely to return to the same ATM) mapping of card numbers to languages, and that would bring a huge liability.
Gmail somehow knows. Every time I click a link, it asks to open in Safari or for me to install Chrome. I check the never ask prompt, but it always asks.