Goldman Sachs: “At the end of the day, we are a purely digital business”

I enjoyed listening to the Odd Lots podcast from Bloomberg this week, as they were interviewing Marco Argenti, the Chief Information Officer at Goldman Sachs about AI. They even transcribed the whole thing. As AI in banking is my big 2024 subject, I listened to the podcast, read the transcript, got the t-shirt and wear the badge. Here’s a few highlights and, if you have an hour free, you might want to listen to it.

Key insights from the pod:

• What the Chief Information Officer does at Goldman Sachs — 04:05
• How Goldman Sachs approaches Generative AI — 08:40
• What Goldman Sach’s AI model was designed to do — 13:45
• How regulation and information security affects Goldman’s AI usage — 20:12
• The impact of physical construction constraints on banking AI — 24:46
• Will GPUs forever be the base of Ai software? — 29:48
• How Wall Street’s attitudes have shifted on open source software usage — 33:57
• How Goldman Sachs attracts AI developers — 38:18
• Is AI a net hiring positive or negative for Goldman's employees overall — 40:35
• How AI has changed other job responsibilities at Goldman — 43:19
• What makes a good AI chat prompt — 45:39

My favourite parts are the quotes, and here are a few I will use in future presentations (they are all from Marco):

I was visiting my mother the other day “and so she said ‘But what do you do at Goldman?’ And I said — you know, I just try to simplify — I say ‘I make sure that the printers don't run out of paper.’”

“The role of a CIO has actually changed quite a bit. And now it's about really asking the question ‘How do we implement technology in order to achieve our strategic objectives and actually to be differentiated?’ And it's really sitting at the strategic table of the firm.”

“A lot of the things that we want to do [are] determined by how good you are at technology.”

He then talks about the fact that meetings are not based on presentations but documents, a trick he learned at AWS (he last worked at Amazon), and that documents overcome forceful personalities and language difficulties.

“Everybody starts reading. You start reading for like sometimes 30 minutes or 45 minutes. And if you are the author of the document, you're just sitting there basically and you're just trying to look at people's faces and understand what they think about your document.”

That’s an effective way to change cultures with engineers, in his view. The interview then moves on to Generative AI and Large Language Models (LLMs) [9:24]

“Even if for companies like us that have been working on machine learning and traditional AI for literally decades, this felt like a very different thing.”

“At the end of the day, we are a purely digital business … [and] it's all about how we service our clients. It's all about how smart we are. It's all about how we can process [an] incredible amount of information.”

Then discussed the risks of AI [11:27]

“Even a 0.1 percent inaccuracy is completely unacceptable ...

“… and so we built this GS AI platform, which essentially takes a variety of models that we select, puts them in a condition of being completely segregated and completely secluded and completely safe from an information security standpoint …

“… and so imagine this, we got a great engine and we decided to build a great car around that.”

“We have an AI committee that looks at the business case, should we do this? And then we have an AI control and risk committee that looks at, okay, how are we going to do that? And then the two of them need to actually come together before we can release a use case.”

What is that used for? [14:23]

“One of the first things that we did was use the platform and the models to extract information from publicly available documents … and put our bankers in a condition to be able to ask very, very sophisticated multi-dimensional questions around what was reported, cross ref[erence] it with previous reports, cross ref[erence] it with any announcement, any earnings call, transcripts, all things that are out there but just are difficult to bring together.”

“We're rolling it out right now as an assistant to our bankers so that they can service their client or answer client questions or even their own questions in a time that is a fraction of what it used to take.”

“We always have as a rule, like when you drive a car that has some autonomous capability, that you always keep the hands on the wheel. Our rule is that there always needs to be a human in the loop, okay?”

What about the risk of data leakage? [22:52]

“There are controls that guarantee that nobody has access to the data that we put into the model. That the data leaves no side effects, so it's not saved anywhere. It only stays in memory. The model is completely stateless, meaning that the state of the model doesn't change after the data comes through. So there is no training, there is nothing done on that data. And also that operator access, meaning who can actually access the memory of those machines, is restricted and controlled and needs to be agreed with us.”

It then moves on to the technology behind AI and specifically NVIDIA and GPUs and CUDA * [30:29]

“What NVIDIA’s been doing a great job at is actually to make [GPUs] work in unison with a virtualisation software called CUDA … because the performance premium that you have on those GPUs when you're trying to train those incredibly large (language) models is something that you really, really want.”

It gets kind of technical here, as Marco not only refers to GPUs and CUDA, but also Meta’s Llama 3.1 and Amazon Bedrock. There’s obviously a lot of competition in this space. That’s why [34:27]

“My guidance to my team is, don't build anything unless you have to. Don't think that just because you're a smart person, you can build software better than anybody else … focus on building things that are actually differentiating for us.”

“Open source software … heavily reduces the vendor lock-in.”

“Every developer in Goldman Sachs is equipped with generative AI coding tools and, you know, we have 12,000 of them” with an average 20% increase in productivity as a result.

And then a discussion of how technology integrates with business [39:15]

“Working at a technology company is absolutely fantastic, but you're always like one step removed from the business or from the application … developers, especially when AI are starting to do all those magical things that we are talking about, can see the impact on the business right away. And that is attracting a lot of people.”

“We're going to be less low level and more ‘Hey, I need to really understand the business problem; Hey, I really need to think outcome driven; Hey, I need to have a crisp mental model, and I need to be able to describe it in words.’ So the profession is going to change.”

“The focus shifts from ‘the how’ to ‘the what’ and to ‘the why.’”

In other words, it’s now cool to be a developer and code for a bank because you’re a digital banker! Oh, and the last comment makes me think of one of my slides from Deutsche Bank who said in their strategy five years ago that banking is what they do and technology is how they do it.

The bottom-line is that things are changing fast. From NVIDIA’s GPUs and CODA to Amazon’s Bedrock and Meta’s Llama 3.1, I was intrigued at the nuts and bolts of change to achieve AI. It’s incredibly technical and transformative. I was just a bit surprised at no mention of Quantum stuff, but hey, there you go.

The key learning lessons are around how technology is now totally integrated with the banking business as, as Marco said, “at the end of the day, we are a purely digital business”.

 

Listen to Odd Lots on Apple Podcasts
Listen to Odd Lots on Spotify

 

* GPUs are Graphics Processing Units. These are specialised processing cores that are used to speed computational processes. It is what has made NVIDIA a trillion dollar company. CUDA stands for Compute Unified Device Architecture. It is a parallel computing platform and application programming interface (API) model created by NVIDIA.