How bad is deep fake fraud?

I’m listening to a fascinating podcast from the BBC about Whisky Bandits. The scam is that you buy a cask of whisky for thousands of dollars, but the cask you bought does not exist. In other words, you just got a piece of paper saying that you own a cask or, in simpler terms, you just bought a piece of paper.

The way the scam worked is that everything appeared above board and bona fide. A company with awards and endorsements and a team who could be trusted. Unfortunately, I got caught up in a similar scheme for property investing way back in the day. There are fraudsters everywhere.

https://www.youtube.com/watch?v=ZiJa9diJOMk

The thing is that AI is making this ten times worse. By way of example, I spotted this one the other day where you could take a picture of your car and ask AI generators to add bumps and scratches.

The result is that you can then send a claim to your insurance company and expect a decent payout. The thing is, for those scammers, insurance is the ‘common pool’, so the payout comes from the premiums of everyone else in the scheme. So they’re all losers and you are a winner because you conned them.

Another good scam is to generate fake receipts.

You had a client dinner or were travelling on expenses, and can then send lots of pictures of the fake bills to reclaim on expenses. Nice!

Thing is that we talk about agentic AI and these are two good examples where it can work, because the fraud detection bot analyses the fake bot and highlights when it is a fake receipt or insurance claim image. How? Pretty easy tbh. The fraud bot just looks for the image enhancement changes and tracks them down.

For example, imagine you got a Zoom call from your boss, but your boss is a deep fake boss and you wouldn’t be able to tell …

https://www.youtube.com/watch?v=SY1JiACTDSs

… we talk about Authorised Push Payment (APP) fraud today which is massive – around £400 million lost to APP in just the UK last year – then think about how much fraud there will be in the next decade thanks to deep fake fraud.

A good example is an employee at a Hong Kong-based firm who sent US$25 million to fraudsters after being instructed to do so by her chief financial officer on a video call. It turned out, however, that the CFO wasn’t on a call. Fraudsters was a deepfake designed to trick her into sending the money ... and it worked.

Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’

Deloitte predicts that gen AI could enable fraud losses to reach US$40 billion in the United States alone by 2027, from US$12.3 billion in 2023.

Deepfake banking and AI fraud risk | Deloitte Insights

In fact a recent Wall Street Journal article found that deepfake incidents increased 700% in fintech in 2023!

But there are possible preventions and solutions. As mentioned, bank bots will look for anomalies and highlight potential fraudulent deepfake actions; equally, corporate and personal bots will flag when you are in dialogue with a false image or voice; fraud bots will scan documents and photographs to recognise when things look fishy and highlight them; and so on and so forth.

If anything, the battle is the same as it has always been. The battle between keeping ahead of the criminal and fraud community versus being safe, stable and protecting the customer. The problem is that this battle has been turbo-charged in favour of the underworld through technology. Are you keeping up?

 

Postscript:

By law, banks have to retrieve fraudulent transactions and funds. According to Bell Gully, courts considered the duties that banks owe to victims of authorised push payment fraud in the wake of the UK Supreme Court’s decision in Philipp v Barclays Bank.¹ Most recently, in Santander UK PLC v CCP Graduate School Ltd,² the High Court rejected the suggestion that a recipient bank owes a duty to a third party victim of fraud, with whom it has no contractual relationship, to retrieve funds that have been fraudulently transferred.