How fast is fast? Faster than you think!

One of the excellent presentations delivered at the trading technologies conference I chaired this week was given by Steve Rubinow, EVP and CIO at NYSE Euronext.

Steve talked about how most latency lies in the infrastructure – the “trading ecosystem” as he referred to it – rather than in the applications and the algorithms. This is why there is such a major focus on connectivity, colocation, proximity and ensuring that the latency hand-offs between all the interconnected parts of the ecosystem are minimised.

It is also why I have heard that the exchanges are now charges for rackspace metres. The nearer in the rackspace you want to be to the exchange server, the more the cost. That’s a prohibitive game as it always means the most High Frequency Trading (HFT) firms get their orders filled first if they pay the price. That’s another story that I’ll come back to in the future.

Steve also referenced the regulatory overhead on storage these days, as you have to store every quote, no matter how important or unimportant that quote may be, because it is all part of the trading ecosystem. By way of example, I was chatting with Thomson Reuters recently and heard that they were tracking about 5,000 market movements per second in 2000, a decade ago; today, it’s more like 550,000 trade movements – quotes, indications of interest, executions – per second and by the end of the year, they expect this to exceed a million.

Steve corrected me on these numbers by referencing the US Options Exchange, who claims to track four million messages per second across the US trading markets, peaking at ten million per second.

Ten million trade movements per second in the USA ... that’s 60 million trade messages per minute or almost three billion per day.

And they all need to be stored, indexed, referenced and time stamped.

That got us into a discussion about multithreaded massively parallel processing, MPP (are you losing the will to live yet?).

As I asked Steve about the recent case of Barclays Capital getting fined by the FSA over their time stamping on trade flows being out of synch, he made it clear that this is happening a lot because firms built their trading systems in single thread environments – you process everything in sequence – whereas today it’s all about processing in multi-thread – you process everything in parallel. That’s a core challenge in the systems and applications of the City’s trading firms, as it pretty much means rearchitecting all systems involved in the pre-trade order management and execution management world built before the mid-2000’s. And now we’re in a cloud computing world, with trade execution applications moving into widget-based iPhone apps, that’s proving a stretch for everyone.

And remember, there’s over anything up to ten million messages per second ... it’s all about order flow, efficiency, algorithms and speed.

Steve reckons that, right now, the fastest processing in the world is 250 microseconds.

250 microseconds.

How fast is that?

250 microseconds is 0.00025 of a second.

0.00025 of a second.

By the time your brain engaged in reading that last sentence, about three seconds, this would have allowed 12,000 trade movements to have flowed through the systems, or 4,000 per second.

That’s today – 4,000 trade movements per second.

By the end of the year, Steve reckons that will increase to 10,000 per second (100 microsecond processing) and soon we will get into nanoseconds and picoseconds and femtoseconds:

1.0 second (s) can be broken into:

• 0.001 (a thousandth) millisecond (ms)
• 0.000 001 (a millionth) microsecond (µs)
• 0.000 000 001 (a billionth) nanosecond (ns)
• 0.000 000 000 001 (a trillionth) picosecond (ps)
• 0.000 000 000 000 001 femtosecond (fs)
• 0.000 000 000 000 000 001 attosecond (as)
• 0.000 000 000 000 000 000 001 zeptosecond (zs)
• 0.000 000 000 000 000 000 000 001 yoctosecond (ys)

Jeez, this stuff blows my mind.

Finally, Steve threw in another comment about fraud and security that piqued my interest.

He said that “stealing credit card data is child’s play; it’s where criminals cut their teeth before they go for the big deals”, and that NYSE Euronext gets “thousand of potential attack vectors” every day.

So they’ve build a secure system to avoid denial of service, phish, malware and other attacks.

How effective is that system?

Very.

Steve presented this anecdote and it’s a great story.

Last year, there was a massive attack on key US servers through the internet that emanated somewhere in Asia (don’t blame the Chinese whatever you do, Steve).

Most of the menace was trying to crack into US defence and government systems and the only servers that fell outside that loop were NASDAQ’s and NYSE Euronext’s.

NYSE Euronext weren’t aware of the attack until the Department of Homeland Security rang Steve and said: “do you know you’re systems are being cracked?”

That sounded a bit tardy on the CIOs part so, at that point, I thought Steve was about to announce his resignation from the Exchange ... but no, it was because their systems were so secure that the attack had not registered.

Steve rang his back office security guys and asked if they were aware of a breakthrough in the system, and they said ‘no’ because there wasn’t one.

Their systems designs were so good that they had deflected it.

So he rang the Department of Defence back and said there had been no breakthrough and they were a bit stunned, as most of their systems had been infiltrated through the attack.

They asked: “we spend so much on our systems and defences – far more than you – so how come we were compromised and you haven’t been.”

I think Steve said something like: “that’s for me to know and for you to find out ... or pay me to tell you”, and put the phone down.

Lovely.