How can you oversee an algorithm you can’t explain?

After yesterday’s blog How can you oversee a system you don’t see?, Ricardo replied on twitter:

So, I investigated how our social media platforms’ algorithms fail, and there are lots of examples. Let’s start with Facebook.

Facebook’s algorithm is blamed for the spread of misinformation and divisive content, radicalising users and failing to protect them from some of the most graphic content on the site. Even now, March 2022, a group of Facebook engineers identified a “massive ranking failure” that exposed as much as half of all News Feed views to potential “integrity risks” over the past six months, according to an internal report. Even worse is another internal report leaked yesterday makes clear that they have no control over their data or, more importantly, user's data.

“We’ve built systems with open borders. The result of these open systems and open culture is well described with an analogy: Imagine you hold a bottle of ink in your hand. This bottle of ink is a mixture of all kinds of user data. You pour that ink into a lake of water (our open data systems; our open culture) … and it flows … everywhere. How do you put that ink back in the bottle?”

To me, that says their business model is broke if, under GDPR and other regulations, they cannot track and block third party access to user data that has not been permissioned.

Oh dear.

YouTube.

According to research, YouTube’s recommender AI provides a huge amount of bottom-feeding, low-grade, divisive and disinforming content. The aim is to grab eyeballs by triggering people’s sense of outrage, sewing division and polarisation or spreading baseless and harmful disinformation.

In other words, provide more clickbait and the more the bait gets clicks and so the more clickbait is posted as recommended for you.

TikTok.

TikTok knows the issues and recognises its’ blind spots. In a 2020 blog, they stated:

“One of the inherent challenges with recommendation engines is that they can inadvertently limit your experience — what is sometimes referred to as a ‘filter bubble’. “By optimizing for personalization and relevance, there is a risk of presenting an increasingly homogenous stream of videos. This is a concern we take seriously as we maintain our recommendation system.”

That’s why they are opening up about how their algorithms work, and is pushing for more transparency.

I’m guessing this is why Elon Musk acquired Twitter.

Twitter.

When you tweet a photo on Twitter, it is cropped on feeds. You have to click the photo to see it in its totality. The claim is that algorithm prefers democrats to republicans, white to blacks and men to women.

Just before the acquisition, Elon posted this poll:

Expecting big things of that.*

The thing that strikes me, as I post this, is that if social media goes down rabbit holes based on clicks, what happens in the hyper-connected investment markets of flash trading?

Bearing in mind that the financial collapse of 2008 was due to incredibly complex connected contracts where collateralised debt obligations (CDOs) could be packaged into mortgage backed securities created a global meltdown … how well do we understand financial algorithms today.

If social media firms don’t understand their social algorithms, do financial institutions understand their financial algorithms? And if we oversee markets that we cannot see, how do we oversee algorithms we do not understand?

“We avoided many complex products, like structured derivatives, because I would ask traders to explain them. If I couldn’t understand what they said, then I would ask them to explain again. If I still couldn’t understand what they were talking about, then we wouldn’t do it.” Former Bank CEO

* Postnote:

I have a suspicion that Elon, the co-founder of PayPal, also wants to integrate commerce and payments into Twitter. Watch that space!