One key consideration is “provable fairness”. It’s my understanding that exchanges use techniques like long, same length fiber optic cables to all racks within the exchange datacenter to convince customers that everyone is on a fair playing field.
This is a lot harder to do when a server is virtualized somewhere on some rack on EC2. Exactly as mentioned, people will try to optimize by spinning up/down instances as close to the exchange server as possible. Customers will be unhappy because they can’t prove that it’s fair, even if they have the closest server.
I think the lines between software and hardware-based are a little blurred these days with accelerator cards and whatnot. It's just a lot harder to come with the same level of guarantees when you're basically running a hypervisor on top of it.
At some point, someone has the shortest route connecting to the exchange's bare metal EC2 instance, and that organisation has a significant advantage in high frequency trading.
This is a lot harder to do when a server is virtualized somewhere on some rack on EC2. Exactly as mentioned, people will try to optimize by spinning up/down instances as close to the exchange server as possible. Customers will be unhappy because they can’t prove that it’s fair, even if they have the closest server.
Overall great, thought provoking writing btw