Sophie Bushwick: Today we're talking large language models: what they are, how they do what they do and what ghosts may lie within the machine. I’m Sophie Bushwick, tech editor at Scientific American.

George Musser: I’m George Musser, contributing editor. 

Bushwick: And you’re listening to Tech, Quickly, the AI-obsessed sister of Scientific American’s Science, Quickly podcast. 

[CLIP: Intro music]

Bushwick: My thoughts about large language models, which are those artificial intelligence programs that analyze and generate text, are mixed. After all, ChatGPT can perform incredible feats, like writing sonnets about physics in mere seconds, but it also displays embarrassing incompetence. It failed to solve multiple math brain teasers even after lots of help from the human quizzing it. So when you play around with these programs, you're often amazed and frustrated in equal measure. But there's one thing that LLMs have that consistently impresses me, and that's these emergent abilities. George, can you talk to us a little bit about these emergent abilities? 

Musser: So the word emergence has different meanings in this context. Sometimes, these language models develop some kind of new ability just because they're so ginormous, but I'm using the word emergent abilities here to mean that they're doing something they weren't really trained to do, they're going beyond their explicit instructions that they've been given.

Bushwick: So let's back up a little and talk about how these models actually work and what they are trained to do.

Musser: So these large language models work sort of like an autocorrect on your phone keyboard. They're trained on what are likely completions of what you're typing. Now they're obviously a lot more sophisticated than that keyboard example. And they use different computational architectural techniques. The leading one is called a transformer. It's designed to transform cues that we developed from context. So we know what a word is because of the words that are around it.

Bushwick: And transformer, that's the ‘T’ of GPT. Right? It's a generative pre-trained transformer.

Musser: Exactly. So that’s one component is the so-called transformer architecture. It goes beyond the old or older; it's not that old neural network architecture that's models on our brains. So another component that they've added is the training regimen. They're basically trained on like a peekaboo system where they're shown part of a scene, well, if they're trained on visual data, but part of texts if they're trained on text, and then they're trained to try to fill in the blanks on that. And that's a very, very stringent training procedure. If you had to go through that, if you were given half a sentence had to fill in the rest of the sentence, you would have to learn grammar. If you had known grammar, you'd have to learn knowledge of the world, if you hadn't known that knowledge of the world. It's almost like Mad Libs or fill-in-the blank training. So that is a hugely demanding training procedure that gives it these emergent capabilities. And then, on top of all that, it has a fine-tuning so-called procedure where not only will it autocomplete what you've typed in, but it'll actually try to construct a dialogue with you, and it will come back and speak to you as if it were another human. And you know, it's acting, it's responding to your queries in a dialogue format. And that's pretty amazing, as well, that it can do that. And if these are features that people didn't really expect AI systems to have for another decade or so.

Bushwick: And what's an example of something that it does that goes beyond just filling in part of a sentence or even engaging in dialogue with people? One of these abilities that are being called emergent abilities.

Musser: This is really cool because every AI researcher you talk to on this has his or her or their own example of the aha! moment of something it was not meant to do, and yet it did. So one researcher told me about how it drew a unicorn. He asked it, draw me a unicorn. Now it doesn't have a drawing capacity, doesn't have, like, an easel and brushes. So it had to create the unicorn out of graphical programming language. So you have to consider the number of steps that are required. It had to extract a notion of a unicorn from Internet text. It had to abstract out from that notion kind of the essential features of a unicorn, sort of like a horse, it has a horn, etcetera. And then it had to learn separately a graphical programming language. So its ability to synthesize across vastly different domains of knowledge is just astounding, really.

Bushwick: So that sounds really impressive to me. But I've also read some critics saying that some of these abilities that seem so impressive happened because all this information was just in the training data for the large language model so it could have picked it up from that, and they've sort of criticized the idea of calling these emergent abilities in the first place. Are there any examples of LLMs doing something that you're like, wow, I have no idea how they got it from that training data.

Musser: There's always a line you can draw between its response and what was in its training data. It doesn't have any magical ability to understand the world—it is getting it from its training data. It's really the ability to synthesize, to pull things together in unusual ways. And I think a kind of middle ground is emerging among the scientists who discover this, who, they're not dismissive and saying, oh, it's just autocorrect. It's just parroting what it knew. And to the other extreme, oh, my God, these are Terminators in the making. So there's kind of a middle ground you can take and say, well, they really are doing something new and novel that's unexpected. It's not magical. It's not like achieving sentience, or anything like that. But it's going beyond what was expected. And, you know, as I said, every researcher has his or her their own example of, whoa, how the freak did it do that. And skeptics will say, I bet that it can't do this. Next day, it did that. So it's going way beyond what people thought.

Bushwick: And when scientists say "How does it do that?" can they look into the the sort of black box of the AI to figure out how it's doing these things?

Musser: I mean, that's really the main question here. It's very, very hard. These are extremely complicated systems, the number of neurons in them is on par on the neurons in a human are, certainly a mammal brain. But they're using, in fact, techniques that are inspired by the techniques of neuroscience. So the same kinds of ways that neuroscientists try to access what's in our heads, the AI researchers are doing to these systems as well. So in one case, they create basically artificial strokes, artificial lesions in the system. They zap out or they temporarily disable some of the neurons in the network and see how that affects the function. Does it lose some kind of functionality? And then you could say, ah, then I can understand where that functionality is coming from. It's coming from this area of the network. Another thing they can do, which is analogous to inserting an electrical probe into the brain, which has been done in many cases for humans and other animals, is to insert a probe network, a tiny little network that's much smaller than the main one into the big network, and see what it finds. And in one case I was very struck by, they trained a system on Othello, the board game, and inserted one of these probe networks into the main network. And they found that the network had a little representation of the game board built within it. So it wasn't just parroting back game moves, ‘I think you should put the black marker on, you know, this square,’ it was actually understanding the game of Othello and playing according to the rules.

Bushwick: So when you tell me things like that, like the the machine learning the rules of Othello building a model of the game board, or a representation of the game board within its system, that makes me think that, you know, as these models keep developing, as more advanced ones come out, that these abilities could get more and more impressive. And so this brings us back to something you mentioned, which is AGI, or artificial general intelligence, this idea of an AI with the flexibility and capability of a human. So do you think there's any way that that kind of technology could emerge from these?

Musser: I think absolutely. Some kind of AGI is definitely in the foreseeable future. I mean, I hesitate to put the number of years on it. One researcher said within five years, we'll see something that's like an AGI—maybe not a human level, but a dog level or at rat level, which would still be pretty impressive. The large language models themselves alone don't really qualify as AGI. They're general in the sense that they can discourse about almost any piece of information or human knowledge that’s on the on the Internet in text form. But they don't really have a stable identity, a sense of self that we associate with most certainly animal brains. They still hallucinate, confabulate, they could have a limited learning ability, but you can't put them through college. They don't have this ongoing learning capacity. That really is what's so remarkable about mammals and humans, absolutely. So I think the large language models are basically solved, as far as the AI researchers are concerned, the problem of language, they got the language part. So now they have to bolt on the other components of intelligence, such as symbolic reasoning, our ability to intuit what physics is that things should fall down or break, etcetera. And those can be kind of put on in a modular way. So you're seeing a modular approach that's now emerging to artificial intelligence.

Bushwick: We talk about modular AI that sounds like what I've heard about plugins, these programs that work with an LLM to give it extra abilities like a program that can help an LLM do math.

Musser: Yes. So the plugins that OpenAI has introduced with GPT, and that the other tech companies are introducing with their own versions of that are modular, in a sense that's thought to be roughly similar to what happens in animal brains. I think probably you'd have to go even further than that to get something that's truly an artificial general intelligence system. Still, plugins are still invoked by human user. If you give a query to ChatGPT, it's capable of looking at the answer on an Internet search. It can run a Python script, for example. It could call up a math engine. So it's getting at the modular nature of the human brain, which has multiple components also that we call on in different circumstances. And whether that particular architecture will be the way to AGI, it's certainly showing the way forward.

Bushwick: So are AI researchers really excited about the idea that AGI could be so close?

Musser: Yeah, they're tremendously excited. But they're also worried they're worried that they're the dog that's about to catch the fire hydrant, because it's just like, the AGI has been something they've wanted for so long. But as you begin to approach it, and begin to see what it's capable of, you also get very worried—and a lot of these researchers are saying, well, you know, maybe we need to slow down a little bit, or at least, slow down is maybe not the right word. Some actually do want to slow down, some do want to pause or moratorium, but there's definitely a need to enter a phase of understanding, of understanding what these systems can do. They have a number of latent abilities, in other words, abilities that are not explicitly programmed into them that which they exhibit when they're being used that haven't been fully catalogued. No one really still knows what ChatGPT, even in its current incarnation, can do. How it does is still an open scientific question. So I think before we you know, have the the Skynet scenarios, we've got more immediate a) intellectual questions about how these systems work and b) societal questions about what these things might do in terms of algorithmic bias or misinformation.

Bushwick: Tech, Quickly, the most technologically advanced member of the Science, Quickly podcast family, is produced by Jeff DelViscio, Tulika Bose, Kelso Harper and Carin Leong. Our show is edited by Elah Feder and Alexa Lim. Our theme music was composed by Dominic Smith.

Musser: Don’t forget to subscribe to Science, Quickly wherever you get your podcasts. For more in-depth science news and features, go to ScientificAmerican.com. And if you like the show, give us a rating or review!

Bushwick: For Scientific American’s Science, Quickly, I’m Sophie Bushwick. 

Musser: I’m George Musser. See you next time!