.jpg)
High frequency trading has tested the limits of servers, networks and software as prop desks and hedge funds constantly push for an edge. Sometimes they push beyond what their systems can do. Unless they have the right monitoring tools they might not know it.
Endace research shows that software- based monitoring systems can be relied on to capture 100% of packets up to approximately 2 Gb/s before they start to drop packets. As exchanges move to higher capacity networks, with 10 Gb/s becoming the new standard, traditional software monitoring systems will be overwhelmed. The technology to capture data at that speed and store it is very challenging. Only a few firms in the world can do it, and we are one.
Endace uses high performance hardware based on processors known as FPGAs to reliably capture 100% of the network data up to speeds of 40 Gb/s.
As soon as trade packet hits a firm’s firewall in either direction, Endace can capture it and time stamp it to within 100 nano-seconds (less than a millionth of a second) and store it in a massive database which can hold a week’s worth of trade data. (After that, a firm will either overwrite the data or transfer it to less expensive storage.)
A hedge fund in Chicago uses Endace to check on its order management systems (OMS). They had missed a trade because the OMS didn’t send it to the exchange, probably caused by a software bug. But with the data captured by Endace they had a record of every trade sent to the exchange so they were quickly able to learn if this was a rare event or something that had been happening more often.
Our main clientele in finance are high frequency traders and prop shops at the big investment banks. They use Endace hardware to collect the trade information and use it for data mining, algorithmic development and trade surveillance.Looking ahead, they also expect regulators will require the extremely accurate time-stamped records we provide.
Already this year, market volatility has proven the value of a hardware approach to trade data monitoring. The Endace hardware has no trouble keeping up with the peaks of market data, including the microbursts which are becoming more frequent. A major New York investment bank said they were seeing volumes double the average on some days. Because traffic peaks often coincide with significant price swings they are the worst possible time for your system to drop packets either for trading or for post-trade analysis. In volatile periods it’s vital to maintain complete visibility into your trading.
Firms can place the Endace at multiple points on the trade process without inducing latency to spot delays in routers, networks, or at the exchange.
Eventually regulators will want to take advantage of highly accurate time-stamped data, but they aren’t there yet. We have worked with regulators in the US, Europe and Australia as they develop their proposals for improved record keeping and audit trails. One of our suggestions, for example, was to mandate the time stamping of all trades globally to the GPS master clock to ensure a global uniform time.
As high frequency trading continues to spread, firms will need to develop more accurate information to manage their operations and to look for arbitrage opportunities. Both firms and regulators are going to need a single version of the truth, which you get only when you capture each packet in a hardware device as it passes between systems and out to execution venues.
Kevin Formby joined Endace in 2010 as Vice President, Business Development having previously been part of the original management team that created Radianz which became the standard means of interconnecting clients for electronic trading.
