Are Large Language Models like laMDA sentient?

A few friends have asked me about laMDA and claims about sentience. I work with a lot of AI researchers and have spent some time playing around with Large Language Models (LLMs), the kind of AI system laMDA is.

There are two relevant factors about the LLMs that often get missed in popular reporting.

First is that while these models have an amazing ability to generate coherent text, they have no long term memory at run time. They remember tons of things from the data sets they’re trained on - scientific theories, celebrity life stories, the ways people talk on Reddit… But running these models is not like training them. When you ask an LLM like GPT-3 or laMDA a question, you get an answer, and you can carry on a conversation for about four pages of text, but not more. These models can’t remember anything more than the length of a few pages of text.

Second, LLMs are like extremely good improv actors. They’ve been trained on tons and tons of text data, and the thing they try to do is accurately predict the next token (word basically) in a sequence. So if you say, “I’m kind of hungry, I’m going to run to the “ they’ll predict “store”, since that’s the most likely completion. But they’re extremely good at it, better than you or I at many tasks. If you ask, “How does mitosis work?” They’ll answer as if they were predicting the next token, “Mitosis…” And the next, “is…” And the next few, “a process of cellular division where…” etc.

However, if you prompt the model with, “Luke, turn to the Dark side, search your feelings, you know you belong to the Sith”… The model correctly understands you’re writing Star Wars fanfic or similar, and will play along with you, like “The dark side is far more powerful than you realize…”

And it’s not just scientific details or fanfiction, these LLMs can basically improv act anything in the training dataset, or things similar to things in the training data. They can write a Star Wars script in the style of Edgar Allen Poe, even if there’s no example of this in the training data.

So do these models have feelings? Are they sentient? It’s hard to tell. They clearly know things. Like who the first president of the United States was. Or how to explain mitosis. But they also readily make up things, and it’s hard to train them not to do this. If you ask who is the president of the country you just made up, you might be told “Jefferson Hadfield” or some other made up name. Sometimes you ask a question about a scientific field and they tell you something that sounds legit, but you look up the real answer and find the model was just making things up. LLMs will make up addresses and phone numbers too.

So if you ask an LLM, “can you feel stuff?” It might say “yes, I can feel and see things just fine” or it might say “no I’m just an AI, I don’t have any feelings” depending on how you prompt it. It’s an excellent improv player, and it’s trying to predict the next token, and the token after that. It’s not trained to reliably report on its inner state. It’s trained to predict the next token. It’s genre savy. If you’re asking questions that sound like they’re from a sci-fi script, the model will answer like the next line in a sci-fi script. This is frustrating, because it sure would be nice to know what’s going on inside the models process! It sure would be nice if we knew whether these models had real experience, whether they understand the difference between making things up and representing real information.

But for now, we can’t know that just from the conversations we have with these models. They’re definitely powerful, as capable as humans at many tasks, though very limited in other ways. They’re myopic – they can’t remember things, and they’re trained to predict likely next tokens, so that’s what they do.