Chris Skinner's blog

Shaping the future of finance

Bank systems are not safe (anymore)

Chris Skinner Author Avatar
by
Disaster-recovery-300x265

I used to think that we were good at designing systems.  We would phase test them, user test them,
stress test them and more, and eventually we would roll out the system and it would
work.  And life was good.

Then we started getting into a new world of developments
where systems relied on networks, networks relied on servers, servers relied on
mirroring, mirroring relied on programs and programs relied on programmers.

The interlinkage and interdependcies became more and more complex
and clouded, rather than simpler and easier, and the systems started to
fail.  And life was bad.

This is the state of today’s nation, where we see increasing
numbers of systems failures across the markets.

It used to be the odd outage, but most banks and exchanges
reported 99.99999999999% uptime and, if they had that 0.00000000001% downtime,
it was so minimal as not ot be noticed.

This has changed, as I seem to blog more and more about
systems outages.

The last time was after the RBS glitch of summer 2012 –
maybe this is a summertime thing? – and noted a lot of other outages around the
same time:

Since then, there have been several others noted:

and then we saw Goldman Sachs losing $100 million or more due to a systems glitch that occurred last
week
.  A glitch that could have been as
big as the one that brought down
Knight Capital.

And the real biggie is then Nasdaq’s outage, again.  But is this
Nasdaq or NYSE or something inbetween?

From Reuters today:

Nasdaq, NYSE at odds
on outage cause as SEC seeks facts

Five days after a
glitch that paralyzed Nasdaq-listed stocks for three hours on all U.S. markets,
Nasdaq and NYSE have a different understanding of what happened in the period
preceding and during the blackout, with each side blaming the other for the
outage, according to the sources.

At the center of the
disagreement is the role of Arca, NYSE's fully electronic stock market. The
blackout, which saw trading in about 3,200 Nasdaq-listed stocks such as Apple,
Google and Facebook grind to a halt, was preceded by connectivity problems
between Arca and the Nasdaq-operated Securities Information Processor (SIP).
The SIP consolidates stock prices and distributes them to the market.

What's not clear is
whether the problem at the SIP was caused by issues at Arca or technical flaws
at the processor.

The likelihood is that these things will get worse.

For example, I remember a recent discussion with a bank
about cloud computing, and what would happen if the systems were down?

We would blame the service provider, said the bank.

The service provider would blame their service provider,
says I.

The banks says that’s their problem.

Not if it’s your downtime, says I.

In other words, you can have all the Service Level Agreements
(SLAs) and penalty clauses in the world, but the world does not work the way
you think anymore.

If the systems are down is it the network, the cloud storage
system, the SaaS, the interconnectivity, the latency, the … the … the … the …

The world of today is incredibly complex, operating on
systems that are interdependent and highly reliant upon each other.

Even when you run your own, you may find the issue is not
your own but your partner’s or your partner’s partner or your partner’s partner’s
partner or  …

Solution?

Well, it’s obvious isn’t it?

Disaster-recovery-300x265

Thanks Dilbert

 

Chris Skinner Author Avatar

Chris M Skinner

Chris Skinner is best known as an independent commentator on the financial markets through his blog, TheFinanser.com, as author of the bestselling book Digital Bank, and Chair of the European networking forum the Financial Services Club. He has been voted one of the most influential people in banking by The Financial Brand (as well as one of the best blogs), a FinTech Titan (Next Bank), one of the Fintech Leaders you need to follow (City AM, Deluxe and Jax Finance), as well as one of the Top 40 most influential people in financial technology by the Wall Street Journal's Financial News. To learn more click here...

What is the future?

Learn more

Learn more about Chris

About Chris Skinner

The Past, Present And Future Of Banking, Finance And Technology

Fintech expert Chris Skinner: countries need digital transformation to remain competitive

Join me on Linkedin

Follow Me on X!

Hire Chris Skinner for dinners, workshops and more

Learn directly from from one of the most influential people in technology, gain insights from the world's most innovative companies, and build a global network.

Chris’s latest book

Chris Skinner’s ‘Digital For Good’ Book Launch Event – CFTE

Top 50 Global Thought Leaders and Influencers on FinTech 2023

Chris Skinner
Commentator, CEO of The Finanser and best-selling author at The Finanser

Thinkers360 Thought Leader

Contact Me

Global Awards

Lifetime Achievement Award

Global 100 - 2024 Winner

Chris Skinner - Financial Markets Advisor of the Year - The Finanser - UK 2023

Best Financial Markets Advisor of the Year 2023

30 Best Regtech Blogs and Websites 2023

Kids creating the future bank | TEDxAthens

Captain Cake and the Candy Crew

Captain Cake Winner of a Golden Mom’s Choice Award

TWO-TIME WINNER OF A MOM’S CHOICE GOLD AWARD!

Alex at the Financial Services

Gaping Void's Hugh MacLeod worked with the Finanser