At this point, some companies depend on having that daily downtime. Their whole development is based around the fact that they will have guaranteed downtime. It's built right into their software stack.
Trying to fiddle with this expected downtime would throw (parts of) the industry into turmoil.
It's just a historical quirk, but it's probably here to stay.
All the people who work in finance I've spoken to have imputed to me that the industry is on the cutting edge, that they will and are able to go to any lengths to execute trades faster, and that they earn their bumper salaries by being the most talented technologists out there. They tell me stories of FPGAs and how they certainly couldn't use garbage collected languages and about people cutting holes in walls to shave valuable feet off cables.
Now, I'm sure that in the absence of any need to they haven't done the work to achieve 24-hour operation, but I didn't get the impression it would be beyond the abilities of the entire industry.
Do you think the people I've talked to misrepresented the industry, and it's not as advanced and competent as they made out?
That's definitely not representative. Perhaps you've spoken only to guys doing HFT. A lot of trading systems are written in Java, and something like 20ms between seeing a trigger and sending an order to the market is fast enough for most needs.
I think for 24/24 operation though, the problem isn't really a programming one but simply the sheer number of people who'd have to work in shifts. Traders, engineers, middle office, maybe even compliance or quants... all these people are really expensive. You can't just leave a system running without people to monitor risk, check for reporting breaks, approve transactions, etc. The end of day reconciliations and reports are also much easier done with the exchange down.
You don't understand. Not everything is written in C. There's a backend system for reporting OATS that starts at 5 PM. If you need to make a non-trivial change you better not deploy it during market hours. Most of these systems are written in Java by guys who aren't at the firm anymore and didn't care when they were. You say cutting edge (which I would strenuously debate). But it's not magic -- in fact it's actually quite mundane when you get down to the nuts and bolts.
Surely there are ways to get rid of it without the transition being so traumatic. For example, the change could be announced a few years prior, and the downtime could go down by an hour per year.
To put things into perspective, the company I worked for responded to bug reports by hiring a team of people whose role was to manually alter the database any time a customer reported a data problem. Dozens of times per day. In other words, instead of fixing the software, it somehow became a reasonable idea to dedicate employees to fixing the symptoms by hand. It made sense in hindsight: they were making so much money that they took the path of least resistance. Paying employees to fix the problem immediately was quicker than trying to hunt down a competent programmer to try to fix the problem without causing more problems.
The change would be traumatic no matter how long they are given to plan for it.
Oh, here's another reason I forgot to mention: You can capture huge profits by exploiting the opening and closing few seconds of the market. A huge portion of all daily trades happen within the first few and last few seconds of the day. So there's a financial incentive to leave things as they are.
From a standpoint of market efficiency, I believe the weekend is a similar "test". Although, i agree that their are diminishing returns to a-synchronous opening hours.
It's true! A certain company has a trading platform that, when run, will wait until the markets open, do some setup stuff around the open, trade for one day, then do nothing forever. Some script comes by to kill -9 everything later in the night so it can be born anew.
At this point, some companies depend on having that daily downtime. Their whole development is based around the fact that they will have guaranteed downtime. It's built right into their software stack.
Trying to fiddle with this expected downtime would throw (parts of) the industry into turmoil.
It's just a historical quirk, but it's probably here to stay.