Nice article. Wondering though why trading is not done in discrete batches, e.g. 5 second intervals? Trades in the same interval get filled equally or stochastically? Info about trades with that same 5 second batch delay? Is there some (theoretical) market efficiency thing at play? All this HFT feels wasteful and bad for 'regular' human investors.
> All this HFT feels wasteful and bad for 'regular' human investors.
Quite the opposite, thanks to the tough competition the market makers are setting the bid/asks spreads as minimal as possible. Which leads to less costs for human investors, pension funds, insurance companies etc.
I used to be a market maker in the 90's before HFT took off. The margins we kept sometimes felt like a rip off but customers had no other choice but to accept them.
People who ask for transaction fees, forced delays in executing or whatever, tend to forget that these force market makers to increase their spreads, which means customers eventually pay the price.
>Quite the opposite, thanks to the tough competition the market makers are setting the bid/asks spreads as minimal as possible. Which leads to less costs for human investors, pension funds, insurance companies etc.
It's not automatically the case that the disappeared margins & thinning of bid/asks have been shared equitably between the trading firms and customers.
Take two exaggerated markets for example:
1) No HFTs: The customer wants 100 shares in Company A. The shares are available on two exchanges, one at $100, and another at $105. A market maker charges the customer $5 to access the 100 shares at $1 each. The customer pays $105. The market maker earns $5.
2) With HFTs: The customer wants 100 shares in Company A. The shares are available on two exchanges, one at $100, and another at $105. The customer clicks "buy" on their trading platform, the HFT races to the $100 shares, and purchases them, then fulfills the order at $105. The customer pays $105. The HFT firm earns $5.
For the end-customer, all that's happened is the margin goes to another firm. The consumer still has no other choice but to accept these transaction fees. There was arguably a need for HFTs to reduce the market-makers exorbitant fees in the 2000's, but that requirement has been served, and the technology now exists to remove both from the market entirely.
HFTs are a rent-seeking entity interjecting in a market which, at least in theory, exists to most efficiently allocate capital to the productive benefit of all.
In the United States at least both scenarios you mentioned are illegal. Market makers are not just sitting in the middle of orders. They buy without a seller lined up and then fill orders from their own inventory (or route orders to an exchange in the case where they can't fill a buy order from their own inventory). In cases where they route to an exchange they are required by law to fill the order at the lowest price available. Typically they fill orders at better prices than what you can get on an exchange. So you, as a retail investor, are actually getting better prices than you would if your broker just filled orders on an exchange.
How the price improvement gets allocated is complicated. Some of the price improvement goes to the broker (in the form a payment-for-order-flow) and some goes to the actual investor (you). But in either case the retail investors are strictly better off.
Latency Arbitrage still exists in a world with NBBO regulations. Research consistently finds that not only does the strategy work in theory, but that it is consistently put into practice by HFT firms to the detriment of other market participants. If a firm can calculate the NBBO ahead of other market participants and the market regulator, it can still legally front-run the market, and risklessly extract rents from end-customers. The NBBO formula is not computationally expensive, and its underlying data is necessarily publicly available to all trading firms. This occurs in the real world, in the order of $billions annually.
The UK's Financial Conduct Authority:
>We use stock exchange message data to quantify the negative aspect of high-frequency trading, known as “latency arbitrage.” The key difference between message data and widely-familiar limit order book data is that message data contain attempts to trade or cancel that fail. This allows the researcher to observe both winners and losers in a race, whereas in limit order book data you cannot see the losers, so you cannot directly see the races. We find that latency-arbitrage races are very frequent (one per minute for FTSE 100 stocks), extremely fast (the modal race lasts 5-10 millionths of a second), and account for a large portion of overall trading volume (about 20%). Race participation is concentrated, with the top-3 firms accounting for over half of all race wins and losses. Our main estimates suggest that eliminating latency arbitrage would reduce the cost of trading by 17% and that the total sums at stake are on the order of $5 billion annually in global equity markets
The University of Michigan's Economics department:
>We illustrate this process and the potential for latency arbitrage in Figure 1. Given order information from exchanges, the SIP takes some finite time, say δ milliseconds, to compute and disseminate the NBBO. A computationally advantaged trader who can process the order stream in less than δ milliseconds can simply out-compute the SIP to derive NBBO,a projection of the future NBBO that will be seen by the public. By anticipating future NBBO, an HFT algorithm can capitalize on cross-market disparities before they are reflected in the public price quote, in effect jumping ahead of incoming orders to pocket a small but sure profit. Naturally this precipitates an arms race, as an even faster trader can calculate an NBBO* to see the future of NBBO, and so on.
>Conservative estimates suggest that at least 4% of dark trading occurs at stale reference prices. High-frequency trading firms (HFTs) almost always benefit from such stale prices, being on the profitable side of the trades between 96 and 99% of the time. Furthermore, stale trading does not happen at random but is driven by the behaviour of HFTs. HFTs as a group almost never provide marketable liquidity in the dark and rather behave strategically to exploit their speed advantage by submitting marketable orders to execute against stale quotes.
I described the canonical front-running example in my first comment, as it gives most non-finance readers a quick overview of how HFTs work, without needing to describe regulations, strategies, NBBO, SIP, etc.
The specific strategy currently employed by HFTs is somewhat immaterial in the broader context of a discussion about front-running. For as long as a firm can legally front-run the market with any strategy, it can undermine the market and risklessly extract profits.
This is entirely false and ignores Reg NMS. Everyone must execute at the NBBO. As well customer orders are often given better and tighter prices than other market participants. HFT firms will often offer them better than NBBO prices. As well, none of these prices you quote would not exist without market makers, its just now the fact that to be a market maker you must be an HFT firm as well due to the scale that is now required.
This is how the Taiwan exchange used to do matching, and I still think it's the best system I've seen.
I don't think the reason has anything to do with price discovery, it's just because exchanges want to maximise their trading fees. Continuous order book trading leads to more trades and hence more profit for the exchange.
They also charge differently (extortionately, some might say) for different speeds of data feed, although I'm not sure if they have tiers just for HFTs.
The HFT is wasteful but isn't bad for human traders, they tend to get better prices. It's bad (sometimes) for huge investors (VHNW individuals, hedge funds, pension funds which I guess represent regular people) that want to make large trades without moving the market but there are also winners here - e.g. if Johnny the day trader buys a stock that Texas Teachers Fund is selling huge batches of, he's better off if HFTs are causing price changes to propagate more quickly.
Those batched trades are called “auctions” and they are a part of many exchanges.
I think it’s pretty uncommon to do them every N seconds.
A common pattern is to collect quotes before the market open, do an “opening auction” to set the opening price, and then switch to continuous trading for the rest of the day. If trading in a stock ever pauses (which can happen for a variety of reasons) then another auction occurs when trading is restarted.
IIRC they have done trials of this on some exchanges. It didn't make a big difference either way. The opening and closing auctions are already sort of done the way you describe.
If you think about it you can never eliminate the advantage of being faster. If you do 5 seconds batches it just means the edges of the batches become the time-sensitive points.
If you want to kill HFT you can do it directly via very very small transaction fees. But guess how popular that is...
> If you think about it you can never eliminate the advantage of being faster. If you do 5 seconds batches it just means the edges of the batches become the time-sensitive points.
You mean you can never completely eliminate the advantage? But mostly eliminating it might still be useful?
Suppose the rule is that if you get your request in by 01:23:45 then it gets handled in the following 5-second period and the response is sent out at 01:23:50. Does someone (A) who finalises their request at 01:23:44.9999 and gets the result back at 01:23:50.0001 have an advantage over someone else (B) who has to finalise their request by 01:23:44.8 and gets their result back at 01:23:50.2? Yes, certainly, but it doesn't seem to be much of an advantage ... So person A can take account of exciting news that arrives at 01:23:44.9, while person B can't, true, but when it comes to reacting to other trades, person A has 4.9998 seconds to think about the news, while person B has 4.6 seconds to think about it, which doesn't seem like a huge difference. Compared to how things work today.
It's funny that you say that, with the crypto transaction fees still a big problem. Feels like HFT and crypto are on a convergence towards that concern.
Exchanges already extract per order commissions. You do not pay per message (so add, cancels and amends are free, you only pay when you get traded [1]).
A per-message would probably significantly affect existing strategies and greatly increase spreads, but I don't think it would prevent all forms of ULL trading.
[1] But even there exchanges offer rebates, if not outright incentives, for market makers to provide liquidity.
You could match what you can distributed equally and leave the rest unsettled.
You could let people decide whether to roll-over the partial bid into a new bid on the next clock or to cancel unsettled.
You could clock to something both very fast on a human scale (50ms), quick enough it'd still feel instant but slow enough that it could reduce HFT silliness and need for extreme low latencies.
> You could match what you can distributed equally and leave the rest unsettled.
Equally per market participant? Do large participant like banks trade same amount as retail investor one trade at a time?
Per quantity? HFT will time the end of the interval and decide to place a large order or not.
It would be weighted by bid size. If there's $10m of bids one side and $5m of offers on the other, you match up the $5m on that side and every bid gets 50% settled.
I'm not sure I understand the problem with "waiting" for the end of the clock. The pool wouldn't be public so you couldn't get knowledge inspecting the pool. All bids and offers would be published on the clock and settled by weighing all the bids and offers against each other and matching by volume.
The trickier issue is what happens in this scenario (assuming limit orders):
Person A bids for 500 units @62
Person B offers 100 units @61
Person C offers 400 units @60
Clearly there needs to be full settlement, we have a bidder who wants to buy 500 units at a price which sellers are happy to sell at.
Correct me if I'm wrong, but in a traditional market it would depend on the order they came in.
Here we would need a formula to work out the correct settlement price. Intuitively this ought to be somewhere just above 61. ( If it were just two people, a bid at 62 and an offer at 60, you could intuit a fair settlement would be 61. )
The general rule is HFT will have the elite level mathematicians to figure out what is the most optimal strategy is, and the most exotic hardware to implement it to the extreme. Other party will fall further behind given the more complex and unconventional exchange rule.
You trade latency arbitrage with statistical arbitrage were participants try to estimate the market and overbid to try to capture as much of the market as possible. That seems dangerous and unstable.
This is all gameable. Like he said, you just time it with an over-large order and let the remainder expire. Everyone gets 10% of their order but my order is 10x what I actually want so I actually get 100%.
Per rata matching is already a thing in some financial markets. They tend to be _more_ latency sensitive as size gets inflated to game the matching algo, thus risk being inflated and thus the value of timely cancels.
They already do this for the opening and closing auctions. You can have a market-on-close order or limit-on-close order for example. The market on close orders are guaranteed to fill. The limit orders are filled using price-time priority, so best prices submitted earliest fill first, after the market price orders.
I guess it is possible that there are remaining marketable orders that never fill because of an imbalance one way or the other, but I doubt that ever happens in practice.
