![Surowiecki_1-071014.jpg](http://www.nybooks.com/wp-content/uploads/2014/06/Surowiecki_1-071014.jpg)
Rich Miller/Data Center Knowledge
Equinix’s 340,000-square-foot NY4 data center in Secaucus, New Jersey. The modern equivalent of the old trading floor, it houses the servers and other facilities of major stock exchanges and of myriad high-frequency trading firms looking for the ability to trade at lightning speed. The yellow trays house fiber-optic cables, through which millions of trades travel each day.
With his new book, Flash Boys, Michael Lewis has made a story that very few people in America had known, or cared, anything about—the rise of high-frequency trading on Wall Street—into the object of national outrage. More than one reviewer wrote that Lewis’s book would make readers’ “blood boil.” Democrats in Congress cited it in renewing their call for a tax on financial transactions. The FBI publicly disclosed, the day Flash Boys was published, that it was investigating high-speed traders. Virtu Financial, a high-frequency trading firm that had been scheduled to go public in late April, delayed its IPO because of the negative publicity generated by the book. And so influential was Lewis’s claim that the US stock market was now being manipulated by high-frequency traders that SEC Chair Mary Jo White felt obliged to declare, in testimony before Congress: “The markets are not rigged.”
Flash Boys has also been the subject of a strong backlash from Wall Street in general, and high-frequency traders in particular, with Lewis’s critics dismissing the book as a simple-minded and ill-informed (if well-told) fairy tale that completely ignores the actual benefits that high-frequency trading brings to investors. As one critic put it simply, the “analysis is wrong. What he is missing could fill another book.”
That Wall Street responded in force to Lewis’s attacks was unsurprising. Though high-frequency trading is dominated by firms that most Americans have never heard of, like Getco and Citadel, it plays an integral part in the way today’s stock market works. As their name suggests, high-frequency traders buy and sell in large volumes and at an extraordinarily fast pace, trading thousands of times a second. Their trading is entirely automated—humans may come up with the algorithms that the computers use, but machines do all the trading. In 2009, a report from the Tabb Group estimated that high-frequency trading accounted for an estimated 70 percent of all the trading in US stocks. And while these firms are less active currently (in part because markets are calmer), they still account for roughly half the trading volume in the US. Indeed, much of what happens in the stock market these days consists simply of computers trying to outsmart each other.
The idea that every day, investing decisions worth many billions of dollars are being made entirely by machines is, even to some on Wall Street, disconcerting. But Lewis is arguing in Flash Boys that high-frequency trading is more than just risky. Instead, it’s a form of legalized theft. The market, Lewis contends, is now effectively designed to allow high-frequency traders to skim profits from other investors. And we’re all paying a price as a result.
To make this case, Lewis uses a formula that’s served him well in the past, focusing his narrative on an intrepid band of outsiders challenging an entrenched establishment. In Moneyball, Lewis’s hero was Billy Beane, the general manager of the small-market Oakland A’s, who was able to use keen statistical analysis to make his low-payroll team a winner. In The Big Short, Lewis’s heroes were short sellers who realized before anyone else that the housing “boom” was nothing more than a giant bubble waiting to pop. In Flash Boys, Lewis focuses on a group at the Royal Bank of Canada, led by a trader named Brad Katsuyama, who come to believe that the stock market has been rigged against them, and who then set about finding a way to fix it.
Structurally, Flash Boys is set up as a mystery. Katsuyama’s job was to buy and sell large chunks of stock, often on behalf of big investors like mutual fund and hedge fund managers. But beginning in 2007, that job became much more difficult. Katsuyama found that if, say, there were a hundred thousand shares offered for sale at a given price, it was next to impossible for him to actually buy the shares. As soon as he tried to, “the offers vanished,” meaning that suddenly the price would jump higher. So instead of being able to buy a hundred thousand shares, he might be able to buy ten thousand, while having to pay a higher price for the rest. Mysteriously, the mere fact of trying to take people up on their putative offers to sell stock was causing the price to rise.
So what was happening? To understand that, you have to know that in the US there isn’t one stock exchange. Instead, there are thirteen public stock exchanges—including the ones everyone’s heard of, like the NYSE and the Nasdaq, but also exchanges with names like BATS and EDGX. (There are also more than forty private exchanges, called “dark pools,” where shares are traded as well.) When Katsuyama went into the market, the shares he was trying to buy weren’t all for sale at one exchange—instead, some were being offered at BATS, some at Nasdaq, and so on. So when he hit the enter button on his computer, he was in effect transmitting a buy order to all of these different exchanges.
Advertisement
To the human eye, it would look like Katsuyama’s order was being sent to every exchange instantaneously. But the reality is that depending on where the exchanges were located, his order might reach one exchange a few milliseconds before it reached others. And in today’s stock market, those milliseconds are an eternity, because they give high-frequency traders the chance to beat you to the punch. What Katsuyama realized was that between the time he hit the buy button and the time that order arrived at an exchange where the shares were theoretically for sale, someone else was zipping in, buying up the shares he wanted, and then offering them back to him at a higher price. As Lewis puts it, “Someone out there was using the fact that stock market orders arrived at different times at different exchanges to front-run orders from one market to another.” It’s as if you went into a pizza place that had one slice left for sale, and between the time you said “Give me the slice” and the counter guy heard you, someone else jumped in, bought the slice, and then resold it to you for an extra nickel.
High-frequency traders were able to pull off this trick because they were faster than Katsuyama. Their trading machines were “co-located,” which means they were in the same room as the exchanges’ servers. And they had also developed sophisticated computer models that allowed them to predict not only who was placing big orders, but where and how much they’d be buying. These models weren’t perfect, but they gave high-frequency traders a reasonable picture of what was going to happen a few milliseconds down the road. And that was all they needed, as Lewis puts it, “to skew the odds systematically in their favor,” allowing them to capitalize on Katsuyama’s intention to buy.
Lewis calls this “front-running.” Defenders of high-frequency trading call it “anticipatory trading.” But whatever you call it, the consequence was the same: Katsuyama ended up paying more for the stock he bought (and getting less for the stock he sold) than he otherwise would have. So he and his crew decided to do something about it. First, they came up with a computer program called Thor that was designed to make their orders arrive at every exchange at exactly the same time. Thor worked reasonably well, but by that point Katsuyama and co. had become evangelists for a better market. So they left the Royal Bank of Canada and set up their own trading exchange, which they call IEX, and which went live last fall.
IEX doesn’t ban high-frequency traders, but it’s designed to make their lives difficult. There’s no co-location. An “electronic speed bump” slows down every order, so that front-running is impossible. And while other exchanges often have a plethora of order types (featuring names like “Hide Not Slide”), which high-frequency traders use cleverly to their advantage—allowing them, for instance, “to withdraw 50 percent of its order the instant someone tried to act on it”—IEX has only four order types. IEX is still just a small player in the stock market—it accounts for less than one percent of US stock volume. But the hope is that over time, investors will prefer to trade on an exchange where they don’t have to worry about being front-run.
Katsuyama’s descent into the high-frequency realm has a certain drama to it, but it’s not, on the surface, the stuff of popular nonfiction, let alone a number-one best seller. What Lewis does so well with high-frequency trading in Flash Boys, much as he did with the world of baseball statistics in Moneyball and the world of collateralized debt obligations and credit default swaps in The Big Short, is to take an esoteric and opaque subject and make it vivid, accessible, and even entertaining. Flash Boys is not, in the end, as exciting as those earlier books, largely because Lewis’s characters here are not as memorable. (Though he does his best to amp up the personal drama—there are a few too many mentions of September 11—ultimately Brad Katsuyama seems like a really nice, successful Canadian fellow, while his team of tech and finance guys seem a lot like other tech and finance guys.)
Advertisement
But Lewis makes the intellectual mystery they’re trying to solve entirely engaging, and in the process illuminates a part of Wall Street that has generally done business in the shadows. He does so without, for the most part, oversimplifying. In Lewis’s short-form journalism, he sometimes takes shortcuts. But in his books, he takes the time to walk through topics in depth and with considerable sophistication. Flash Boys does not provide as rigorous a look at high-frequency trading’s origins and its mechanics as Scott Patterson’s excellent 2012 book Dark Pools does. But readers will come away from Flash Boys with a good picture of how high-frequency trading works.
They’ll also, in all likelihood, come away from it convinced that high-frequency trading is a curse on the markets. Flash Boys may be sophisticated, but it’s not subtle. While Lewis does make one or two nods to the idea that not all high-speed traders are con artists, his book is ultimately a polemic, intended to convince you that high-frequency trading is nothing more than a scam. As he writes:
By the summer of 2013, the world’s financial markets were designed to maximize the number of collisions between ordinary investors and high-frequency traders—at the expense of ordinary investors, and for the benefit of high-frequency traders, exchanges, Wall Street banks, and online brokerage firms.
Which raises the obvious question: Is he right?
When you talk to people on Wall Street about why high-frequency traders are good for the market, the answer is always the same: they provide liquidity. When a market is liquid, it means that it has lots of buyers and sellers, which means that if you want to buy or sell shares, you can find someone to take the other side of the trade, and you can make trades without moving the price too much. When markets are illiquid, by contrast, it’s more difficult to find someone to trade with, and because it’s harder, the price you have to pay to trade rises. Liquid markets are good, in a social sense, because if markets are more liquid, people will be more likely to invest. (If you’re not sure that you’ll be able to sell your stocks at a fair price when you want to sell them, you’re going to be more cautious about investing in the first place.)
It’s undoubtedly true that high-frequency traders add to the liquidity in the market in normal times, since they’re always in there buying and selling, making bids and offers, putting in potential orders, and so on. Over the last decade, as the market has come to be dominated by electronic trading, trading volumes have risen, and the spreads on stocks—the gap between the bid and ask prices, or the most a buyer will pay and the least a seller will accept—have fallen. As a result, and contrary to what Lewis implies in Flash Boys, for ordinary investors (the kind of people who might use their Ameritrade account to buy some shares of Apple), it’s never been cheaper or easier to trade. Markets are also more efficient, in the sense that they react more quickly to new information than ever before.
That all sounds good. But there’s a catch. First of all, even though high-frequency trading has added to market liquidity, we don’t know how real that liquidity is. High-frequency traders are constantly throwing out buy-sell offers, for instance. But 90 percent of those offers are canceled without ever being filled—most are just designed to sniff out the plans of people like Brad Katsuyama.
More important, many observers are concerned that the liquidity high-frequency traders provide could turn out to be illusory, in the sense that it will disappear if stock prices take an unexpected plunge. As Andrew Haldane of the Bank of England has put it, the fear about this liquidity is that “in wartime, it disappears.” That’s what seems to have happened during the “flash crash” that hit on May 6, 2010, when the Dow plunged more than six hundred points in a matter of minutes. As stock prices plummeted, many high-frequency firms just stopped trading, which accelerated the decline and made it harder for prices to return to normal.
Even so, it’s fair to say that, much of the time, high-frequency traders do make the stock market more liquid and more efficient. And while it’s easy to dismiss them as scammers, it’s also true that they are, in their way, amazingly skilled at what they do. Deciphering market microstructure and figuring out ways to earn a profit while making only a few pennies a trade is not an easy task. The best high-frequency traders, in that sense, are as good at trading as, say, Warren Buffett is at investing. So why, really, should we begrudge them their profits and make it harder for them to make a living?
Lewis’s fundamental answer to that question in Flash Boys is that it’s a bad thing for the market to be “rigged” and for investors to feel like they’re being ripped off. There’s obviously something to this, and focusing on the idea of the rigged market was, from a polemical point of view, a smart move, since it helps outrage readers. But by overemphasizing the idea that the market is rigged, Lewis may, paradoxically, end up understating the real problems with high-frequency trading. We should really worry about high-frequency trading because it reduces the amount of real and valuable information in the stock market. It makes the system as a whole less stable and more risky. And it devotes an enormous amount of resources to an arms race that is of dubious social value.
Start with the way high-frequency trading makes the stock market less economically useful. Although we don’t always think of it this way, the stock market is, in part, a tool to allocate resources to companies. To do that well, stock prices need to be constantly incorporating new information about the real world (so companies with good prospects will find it easier to raise capital, and companies with poor prospects will find it harder). This can only happen when investors spend a tremendous amount of time and energy researching companies, trying to figure out what the future holds. It’s not that they’re always right, or that their information and predictions are always accurate. But the basic assumption is that if you set lots of motivated, intelligent investors to digging up new information about companies, their collective judgment (which is what the price of a stock reflects) will make the allocation of resources better.
The key to this process, though, is that there needs to be a way to reward investors for uncovering good information. And in the stock market, that reward is the profits big investors can make by being right (and early) about a company’s future. What high-frequency trading does, by front-running these orders, is eat into those profits. In other words, it reduces the rewards for uncovering real information. High-frequency traders are, in effect, free-riding on the work that real investors do. By doing so, they make real investing less valuable—one recent study of trades in 120 randomly selected Nasdaq stocks found that increased high-frequency trading cost the average institutional investor as much as $10,000 a day. Making investing less valuable will make people less willing to invest. And as Nobel Prize–winning economist Joseph Stiglitz put it in a recent speech, “If the returns to investing in information are reduced, the market will become less informative.” In cruder terms, high-frequency trading may make the stock market more efficient. But it also makes it dumber.
Most high-frequency traders aren’t interested in the real economy at all. They aren’t concerned about a company’s earnings, or its future prospects. The only information that they’re interested in, and that they devote so much energy and money to uncovering, is information about what other investors are going to do. That’s why Katsuyama, explaining why his attempt to buy stock moves the market, says, “I am the news.” Obviously, markets have always had middlemen whose only concern was what the market was doing. But high-frequency trading has made markets ever more self-enclosed and hermetic. One recent study of commodity markets found that 60–70 percent of all price movements in those markets had nothing to do with anything that happened in the outside world. They were solely the result of traders (or, more often, computers) reacting to internal action in the market.
![surowieckii-graph-20140710.jpg](http://www.nybooks.com/wp-content/uploads/2014/06/surowieckii-graph-20140710.jpg)
This not only makes markets dumber. It also makes the system as a whole less stable. In part, that’s because when trading is automated, there’s ultimately no way for humans to adequately supervise it. A market in which most traders are reacting to what other traders are doing is also unusually subject to feedback loops—selling begets selling—and hot-potato effects (stocks get bought and sold over and over, falling in price each time, since no one wants to hold on to them). That helps explain why the “flash crash” was so severe. Nothing that happened in the real world on May 6, 2010, could explain why stock prices fell six hundred points in a matter of minutes (or why, once the selling stopped, they almost immediately rebounded). It was a plunge driven entirely by the internal dynamics of the market. And while high-frequency traders didn’t cause the crash, a 2011 study found that “the trading behavior of HFTs appears to have exacerbated the downward move in prices.”
If high-frequency traders actually add liquidity to markets, as their defenders claim, then they should have stabilized the market on May 6 by stepping in and buying the falling stocks. They didn’t. And while the flash crash was an extremely unusual event, individual stocks in the market now see what are sometimes called “mini–flash crashes” on a daily basis, with their value plummeting sharply and then rebounding. Turmoil in the markets is nothing new, of course. But the speed with which everything happens today, and the way automation makes trading harder to control, raise systemic risk.
On top of all this, high-frequency trading is expensive. In recent years, enormous sums of money and enormous amounts of brainpower have gone into figuring out ways to trade a tiny bit faster. As one character in Flash Boys says, some of the traders “would sell their grandmothers for a microsecond.” Yet all that investment hasn’t made the market any smarter, and it hasn’t even made the market that much more efficient.
While high-frequency trading has been around for more than a decade, it really flowered beginning in 2007. In the years since, it’s become ever faster and ever more sophisticated, thanks to those huge investments in both hardware and software. Flash Boys opens, for instance, with the construction of an enormous data pipeline from Chicago to New Jersey, specifically for high-speed trading. So if high-frequency trading were actually an enormous boon to the market, you’d expect that since 2007, price spreads and trading costs would have plummeted. Instead, they’re roughly what they were in 2006. In other words, all the billions of dollars and endless hours of brain power that high-frequency firms have poured into their business over the last seven years seem to have had only a trivial impact on ordinary investors.
In Dark Pools, Scott Patterson quotes a man named Josh Levine, who was in some ways the father of electronic trading. Levine had hoped that electronic trading would make middlemen irrelevant, and markets more robust. But he became disillusioned by the way the quest to save a few milliseconds had trumped all. He said:
What an expensive and needless mess. You could probably find a cure for cancer in a year if you just reassigned all the smart people who are now working on this artificially created and otherwise useless problem.
Of course, as a society we spend lots of money on lots of completely useless things, and deciding what constitutes behavior that’s socially useful may seem like a mug’s game. But at this point, high-frequency trading really does look like an arms race, one that’s consuming enormous resources without actually generating any real social value, and doing damage to the market in the process. In that sense, Flash Boys should get us to ask questions that we normally ignore, like “What kind of behavior do we want to reward?” and “What do we want people to invest their time and money in?” The unstated assumption behind the rise of high-frequency trading is simple: You can never go fast enough. But the real lesson may be even simpler: Speed kills.