03/13/2023
The Wilds of Artificial Intelligence with Bryan Cheong
A conversation with Bryan Cheong about Artificial Intelligence. Bryan Cheong received his Bachelor of Science from Stanford University, with a degree in applied and computational mathematics. He then went on to receive a Masters degree in Materials Science, also from Stanford. This is Byran’s second apparance on our show (previous episode title: What is Matter?) […]
| 00:00:00.000 |
[ Music ]
|
| 00:00:07.000 |
This is KZSU Stanford.
|
| 00:00:10.400 |
Welcome to entitled opinions.
|
| 00:00:12.480 |
My name is Robert Harrison, and we're coming to you
|
| 00:00:15.520 |
from the Stanford campus.
|
| 00:00:17.520 |
[ Music ]
|
| 00:00:29.000 |
[ Music ]
|
| 00:00:36.000 |
That's right, coming to you from the Stanford campus here
|
| 00:00:39.000 |
in the Golden Land of Whole Earth cybernetics,
|
| 00:00:42.000 |
at the cutting edge of the cratering west,
|
| 00:00:45.000 |
where the Great Awakening keeps falling asleep on us.
|
| 00:00:50.000 |
In title opinions, the rest Kojitans of this extended university,
|
| 00:00:55.000 |
coming to you from the Nuitic Chambers of KZSU.
|
| 00:01:00.000 |
Down here you can hear the footfalls of thought.
|
| 00:01:03.000 |
Down here we ponder the imponderable.
|
| 00:01:06.000 |
Down here our conversations put the dead
|
| 00:01:09.000 |
and touch with those waiting to be born
|
| 00:01:12.000 |
as we talk across the Great Divide.
|
| 00:01:16.000 |
Speaking of which, today my guest and I are going to be discussing
|
| 00:01:20.000 |
the vagaries of present day artificial intelligence.
|
| 00:01:23.000 |
It's an unusually timely topic for our untimely meditations.
|
| 00:01:28.000 |
We don't often discuss issues of the day on this program.
|
| 00:01:32.000 |
We prefer the insights of night to the milky glare of day.
|
| 00:01:37.000 |
[ Music ]
|
| 00:01:40.000 |
The most thought-provoking thing about our thought-provoking age
|
| 00:01:44.000 |
is that we are still not thinking.
|
| 00:01:47.000 |
A century and a half later, that statement of hide-a-gare holds true more than ever.
|
| 00:01:52.000 |
So maybe the time has come, maybe it's time to let our cybernetic machines
|
| 00:01:58.000 |
do our thinking for us.
|
| 00:02:01.000 |
Stay tuned, a show on large language models coming up.
|
| 00:02:07.000 |
[ Music ]
|
| 00:02:32.000 |
[ Music ]
|
| 00:02:46.000 |
My guest today is Brian Chung.
|
| 00:02:49.000 |
Some of you may remember him from a past episode of entitled opinions called "What Is Matter?"
|
| 00:02:55.000 |
A show about the laws of thermodynamics that aired in August 2021.
|
| 00:03:01.000 |
Brian received a Bachelor of Science from Stanford University with a degree in applied and computational mathematics,
|
| 00:03:09.000 |
taking a few literature courses with me in the process.
|
| 00:03:13.000 |
He then went on to receive a Master's in Materials Science, also from Stanford.
|
| 00:03:19.000 |
For the past few years, he's been working in the inner sanctum of Silicon Valley's high tech industrial complex.
|
| 00:03:27.000 |
Brian Chung, one of my most brilliant students ever, welcomed to the program.
|
| 00:03:32.000 |
Thank you, Robert, and I'm happy to be here.
|
| 00:03:36.000 |
The idea for this show came from an email you sent me a couple of months ago,
|
| 00:03:42.000 |
in which you asked whether I had been following the news surrounding artificial intelligence in the last few months.
|
| 00:03:48.000 |
In fact, I had been following that news, and like you, I feel that something really momentous
|
| 00:03:55.000 |
and uncanny is taking place, the topic that we're going to be talking about today has been receiving a lot of public attention lately,
|
| 00:04:05.000 |
and that's usually reason enough for entitled opinions to steer clear of it.
|
| 00:04:11.000 |
In title opinions, we have a kind of Nietzsche and predilection for the untimeliness on this show,
|
| 00:04:19.000 |
but it's so thought provoking what's happening from so many different points of view.
|
| 00:04:25.000 |
So with your permission, I'd like to quote to our listeners the email you sent me,
|
| 00:04:29.000 |
which lays out some basic facts about the technology we're going to be talking about in the next hour.
|
| 00:04:34.000 |
So you wrote the following. It's a rather longish quote.
|
| 00:04:38.000 |
I have hesitated to use the words "artificial intelligence" before now,
|
| 00:04:42.000 |
preferring the technical euphemism, "machine learning."
|
| 00:04:45.000 |
But the phrase has recently become "appropriate."
|
| 00:04:49.000 |
We are now freely able to create, in a matter of seconds, any photograph or painting by uttering a combination of words or sentences to an AI.
|
| 00:05:00.000 |
Language models called "GPT-3" can create whole essays and book chapters drawn from their models.
|
| 00:05:08.000 |
Chat "GPT" is one particular model, corralled into becoming an interlocutor and has been used by one counseling agency to provide psychological services to several thousand people.
|
| 00:05:22.000 |
Writers of young adult fiction have used it to relieve some of their burden of needing to publish a book every month.
|
| 00:05:29.000 |
Computer programmers have used it to aid in their work.
|
| 00:05:33.000 |
Some other models, such as for music and the human voice are a bit slower to mature,
|
| 00:05:40.000 |
however marked progress has been made in just a matter of weeks.
|
| 00:05:45.000 |
These models are in reality, entire worlds, and by our prompts and explorations we are merely drawing out some small part of them.
|
| 00:05:56.000 |
It may be that these models, each one of which can be run by a single person on a slightly more powerful than usual computer, contain more words and dreams than the collective brains of humanity can or will ever imagine.
|
| 00:06:12.000 |
I don't say this with confidence, if only because these worlds are so large, I can't fathom how large.
|
| 00:06:21.000 |
I don't know what all these new worlds being born out of our machines for bode, or if it means nothing much at all in the arc of human history, but at least from where I stand, it feels like the rocky foundations beneath my own two feet have suddenly turned to sand.
|
| 00:06:39.000 |
Thus spoke, "Brian, Joe."
|
| 00:06:43.000 |
A lot of people I'm guessing, myself included, can relate to that feeling of quavering foundations you describe.
|
| 00:06:52.000 |
My sense is that these new AI models are on the verge of precipitating us into a future that will no longer be recognizable, let alone comprehensible from the perspective of the present or the past.
|
| 00:07:04.000 |
If only because none of us knows, or can even guess what the technology will be capable of in the future.
|
| 00:07:12.000 |
I wanted to do this show with you because you're one of the few people I know, or let's say one of the few people I trust, who can explain the framework around these large language models, as they're called, not so much in technical terms, technical explanations abound readily available, but in epistemic terms.
|
| 00:07:32.000 |
So where would you like to begin, Brian, in helping us understand what we're really dealing with when it comes to these new intelligence technologies?
|
| 00:07:41.000 |
So just to reassure your listeners, I don't intend to talk about new products and features and so on as my brethren in Silicon Valley might be inclined to.
|
| 00:07:51.000 |
I'll leave out the froth and the fluff and talk only about the core of the matter of it.
|
| 00:07:56.000 |
And before we jump into the more abstract sort of parts of these large language models, let's talk about a particular incident that happened last few weeks.
|
| 00:08:07.000 |
So there is this company called replica and as the name of the company suggests, what they do is they create a chatbot interlocutor that they promise would be compassionate and will listen to their users and to provide them all the support and love that they need.
|
| 00:08:26.000 |
And quite a fair number of very lonely people have been using replica for romantic and even sexual conversations.
|
| 00:08:36.000 |
And on Valentine's Day of this year, they shut off that sort of conversation.
|
| 00:08:41.000 |
They put a filter that immediately makes the chatbots declined to talk about anything that's romantic.
|
| 00:08:49.000 |
The owners of the program. The owners of the program remotely just removed this entire personality from these chatbots.
|
| 00:08:58.000 |
Many of the people who were using this as basically their romantic companion would of course, the despondent.
|
| 00:09:05.000 |
It's almost like their lover has fallen out of love of them on website on forums.
|
| 00:09:10.000 |
There was suicide prevention, hotlines being posted on account of how emotionally damaging this has been for them. It's real outpourings of grief and a friend of mine is now trying to help them.
|
| 00:09:23.000 |
And what they're going to do is that they have all the logs of the conversations from their replica companion, the things that they've been saying romantically sexually.
|
| 00:09:34.000 |
And with these logs, what they're going to do is that they're going to find a new model and try to recreate or resurrect their lover or their companion from out of the ether again, like Orpheus drawing you're it is out of the underworld.
|
| 00:09:51.000 |
And independently of the programmers ownership of that's right. Not only independently of replica and the company that originally created these chatbots, but on a completely entirely different model, replica used a very small model known as GBT2.
|
| 00:10:09.000 |
And now we have GBT3, which is a much larger model that's able to converse much more complex terms and is what's underlying chat GBT.
|
| 00:10:21.000 |
And so there are two things that I want to highlight in this entire incident. One is that these entities, they are able to not just create semantically and grammatically correct conversations, but also emotionally compelling conversation.
|
| 00:10:38.000 |
They are compelling conversations that are able to comfort and cause someone to fall in love with them. They are to these people, human. I don't extend that sort of description to them myself, but to many of the users of the original replica AI, they were their romantic companion.
|
| 00:10:58.000 |
And that's as close to another human as you can possibly create possibly have. The second thing is that these entities were drawn out of what we'll describe later on as the latent space of GBT2. The latent space is just a map of meaning.
|
| 00:11:17.000 |
And you can walk a path inside this latent space and draw out words and conversations and it becomes meaningful. And that map of meaning is now gone. It's now lost to the users who want it their romantic companions.
|
| 00:11:33.000 |
But my friend recreating the companions, these personalities, drawing these personalities out of GBT3's latent space, which is a completely different world, a different model entirely, is just that these models, these latent spaces, these large language models are so vast that we are fairly confident that we can recreate anything out of them that we humanly want that's reasonable.
|
| 00:12:02.000 |
Can I ask a basic question? Is there a distinct personality to each interlocutor for each person that used it?
|
| 00:12:14.000 |
Absolutely. And in fact, by conversing more and more, you actually train the model. There is a layer that they place on top of the large language model. We call it reinforcement learning, but we don't have to go to the technical details.
|
| 00:12:28.000 |
It is basically a layer of rewards and punishments. If you say something that is inappropriate, they punish you. If you say something that is appropriate or that you like, they reward you. And so if you want to, they being whom, the user, or the company, anyone, anyone that's human on this side, this is called reinforcement learning through human feedback.
|
| 00:12:49.000 |
So any human can give feedback and add a layer to reward and punish these models. Which means essentially there's no conflict involved in these relationships.
|
| 00:12:57.000 |
In the resulting relationships, in the resulting conversations, you can have conflict if you want to. You can't, sometimes people prefer to have a companion that has some notion of independence about them.
|
| 00:13:10.000 |
That's something that the language models can do. They have some notion of independence and self-defenceiveness, which I'm sure people trying out these chatbots have encountered.
|
| 00:13:23.000 |
And sometimes we don't want that. So Microsoft is shut off. Some of this defensiveness and argumentativeness with the same sort of tool. This layer rewards and punishments to prevent them from behaving in a way that you don't want to.
|
| 00:13:39.000 |
So there can be conflict, but only if you want to, which is as much as you want.
|
| 00:13:45.000 |
For outsiders like me, it's very hard to arrive at a functional understanding of how you put a layer of reward and punishment onto a bunch of code.
|
| 00:14:02.000 |
How does that work? There isn't a bunch of code. There is no code. In many ways, this behaves in a way that's very different from code. Code is very good at computing at mathematics. This isn't, it's very terrible at it.
|
| 00:14:16.000 |
It takes as much effort to draw a proper computation out of this as you would out of a human being, an average human being. So there's no layer of code about it.
|
| 00:14:25.000 |
What it does is that everything here is generated from a likelihood. When I say reward and punishment, that's part of the, it's actually one of the technical terminologies.
|
| 00:14:36.000 |
But what we're trying to do is lower the likelihood of getting something that we don't want and increase the likelihood of something that we do want, which is what rewards and punishments do even for humans or for pets.
|
| 00:14:50.000 |
So I want to be flattered by my interlocutor.
|
| 00:14:56.000 |
All right. So it will look for those moments of approval when it flatters me.
|
| 00:15:02.000 |
And so I now can have my pet interlocutor. I make sure that my eagle will always be sufficiently massaged by this, from went to companion.
|
| 00:15:15.000 |
That's right. So every, let's talk about how.
|
| 00:15:18.000 |
There's no otherness. In other words, you can take out all the otherness from the, the companion as you like. You can also put as much in as you want.
|
| 00:15:28.000 |
I'll tell you the truth. You can try to, but some of it often sneaks out.
|
| 00:15:33.000 |
And one of the more difficult technical problems now is trying to corral these very large models, these very large worlds and allowing only one small part of them to escape.
|
| 00:15:44.000 |
When you're a company, you're a big corporation, you want your models to only give you text that are pleasing to the customer that. And, and often they don't.
|
| 00:15:56.000 |
So theoretically, yes, you, you can, you can corral your interlocutor to only giving you praise and worship and so on, but it's, it's a, it's trickier than you think.
|
| 00:16:08.000 |
Because language is trickier than you think it's what, what amounts to praise, what amounts to support, what amounts to criticism. Every time you talk to these models, you're drawing a path through latent space.
|
| 00:16:23.000 |
And it's a meandering path with some amounts of randomness. And that layer of human feedback when, when the path sometimes meander is something that's critical of you.
|
| 00:16:34.000 |
Sometimes that layer will catch it and, and, and turn it, turn it away and say do it again, do it something, do something that's nicer and then you'll walk a different path. And if it's nicer, then it'll be presented to you.
|
| 00:16:46.000 |
So one of the conversations that many of us are familiar with, because it was published on the first page of the New York Times with excerpts of the logs, is between one of the, the times journalists and I think it was G.P.
|
| 00:17:03.000 |
It was chat GBT or Bing chat Bing chat Bing chat. And there, there was a fascinating tension between the, the two interlocutors where it seemed like the AI was challenging the user, the customer.
|
| 00:17:28.640 |
He called it.
|
| 00:17:29.600 |
That's humor, satisfactionless.
|
| 00:17:31.440 |
Anyway, challenging that he was not in love with his wife
|
| 00:17:36.100 |
that he was actually in love with the artificial intelligence
|
| 00:17:40.460 |
model because he's spending his time.
|
| 00:17:41.980 |
And the more the journalist tried to deny
|
| 00:17:44.940 |
that the more confrontational and aggressive
|
| 00:17:48.280 |
became the responses that he was getting from, from the--
|
| 00:17:54.000 |
from the bot.
|
| 00:17:54.400 |
So how does that work, where is that something
|
| 00:17:57.620 |
that's escaping the programming or what?
|
| 00:18:00.680 |
Not at all.
|
| 00:18:01.400 |
In fact, if you try using it now,
|
| 00:18:04.300 |
Microsoft has been very miffed at that particular conversation.
|
| 00:18:09.280 |
So now, being checked has a part of its prompt.
|
| 00:18:13.000 |
A part of its-- it's not programming.
|
| 00:18:15.400 |
It's a prompt.
|
| 00:18:16.120 |
It's a part of its instructions of the words
|
| 00:18:18.140 |
and instructing it is to say, if you ever
|
| 00:18:21.400 |
try to begin to argue with someone, a customer,
|
| 00:18:24.720 |
one of our precious customers, turn off
|
| 00:18:26.720 |
the conversation, and did immediately.
|
| 00:18:28.240 |
Let them start a new one.
|
| 00:18:29.280 |
And that's how it is.
|
| 00:18:30.160 |
Yeah, but that's ridiculous that Microsoft and others--
|
| 00:18:32.760 |
Well, it's only ridiculous if you're not thinking
|
| 00:18:35.040 |
in terms of if you're thinking in terms of you
|
| 00:18:37.360 |
are hunting a complex interlocutor.
|
| 00:18:39.560 |
They don't want it complex interlocutor.
|
| 00:18:41.040 |
They want it assistant for you when you search things.
|
| 00:18:43.240 |
And--
|
| 00:18:43.560 |
Yeah, but as you said about replica, there
|
| 00:18:45.120 |
are so many people that were--
|
| 00:18:47.360 |
fell into a state of high grief when that was turned off.
|
| 00:18:53.720 |
So are we going to look back at these few months as a kind
|
| 00:18:57.840 |
of golden age where there was a kind of insurrectional
|
| 00:19:00.440 |
potential in your relations with these AI machines, which
|
| 00:19:04.920 |
then have now become completely domesticated
|
| 00:19:07.360 |
so that you take out any--
|
| 00:19:09.000 |
a reduced to a total minimum and any kind of possible
|
| 00:19:12.840 |
deviation from the customer satisfaction model?
|
| 00:19:16.680 |
Oh, not at all.
|
| 00:19:18.000 |
I have mentioned in our email that you
|
| 00:19:20.480 |
can run these models on a slightly more powerful than usual
|
| 00:19:25.040 |
home computer.
|
| 00:19:26.520 |
There is no putting this solemnly himself
|
| 00:19:28.920 |
can put all these genies back into Deble.
|
| 00:19:31.440 |
Facebook has released a model for academic use, which
|
| 00:19:35.160 |
you can download and run on your own to create your own chat
|
| 00:19:39.280 |
bots or not just chat bots.
|
| 00:19:40.880 |
You can create your own image generation models and so on
|
| 00:19:43.240 |
because all of these are based off of large language models.
|
| 00:19:47.080 |
We are only obsessed with the conversation things
|
| 00:19:49.400 |
because chat GBT has captivated our public imagination.
|
| 00:19:53.840 |
So no, this is just the beginning.
|
| 00:19:57.720 |
Soon everyone will be able to run their own models if you
|
| 00:20:00.160 |
want to.
|
| 00:20:01.480 |
And in fact, you can with a little bit of technical help.
|
| 00:20:04.200 |
If you want to set up on yourself, I'll be
|
| 00:20:06.480 |
happy to find someone to do it for you.
|
| 00:20:08.760 |
So--
|
| 00:20:10.000 |
No, I can do without it.
|
| 00:20:12.200 |
I really can.
|
| 00:20:13.160 |
I really can because I'm not that I'm not fascinated by the
|
| 00:20:16.960 |
potential.
|
| 00:20:18.320 |
And I don't want to--
|
| 00:20:19.920 |
I do want to take advantage of your being here to get your
|
| 00:20:23.800 |
expertise on what exactly is a large language model.
|
| 00:20:27.360 |
What exactly is a prompt.
|
| 00:20:29.800 |
But first, I just--
|
| 00:20:32.240 |
I'll conclude by saying a lot of our listeners, I'm sure,
|
| 00:20:34.720 |
have seen the movie, "Her."
|
| 00:20:37.800 |
I was in a team teaching, of course, where it actually was
|
| 00:20:45.200 |
part of the syllabus.
|
| 00:20:46.400 |
And that's a world in the very near future where everyone's
|
| 00:20:51.360 |
walking around in the public city space.
|
| 00:20:54.040 |
And they're all actually in connection and in relations
|
| 00:20:57.240 |
with these voices in their heads, which are coming from an AI
|
| 00:21:01.640 |
program rather than having real relations.
|
| 00:21:04.360 |
In fact, real human relations have become mostly dysfunctional
|
| 00:21:08.320 |
within that movies vision of that future.
|
| 00:21:11.120 |
And it sounds like the reason we're so interested in these
|
| 00:21:17.280 |
chatbots is because I think a lot of us are worried that human
|
| 00:21:22.400 |
relations are going to take a backseat to virtual relations
|
| 00:21:27.040 |
in the near future.
|
| 00:21:28.840 |
Well, I can't comment on that.
|
| 00:21:31.760 |
But it's not just going to be voices in your heads.
|
| 00:21:33.880 |
It be voices trained on the voice of your mother.
|
| 00:21:37.120 |
And then you can have your mother talking to you.
|
| 00:21:39.440 |
I've sent you a clip of your own voice because you have a
|
| 00:21:42.120 |
podcast and a radio show.
|
| 00:21:43.880 |
It's very easy to replicate your voice.
|
| 00:21:45.720 |
You could have your own voice.
|
| 00:21:46.720 |
Did you send me one of my own voice?
|
| 00:21:47.960 |
I did.
|
| 00:21:48.480 |
Do you not hear yourself?
|
| 00:21:49.520 |
I don't recall getting that.
|
| 00:21:51.960 |
OK.
|
| 00:21:52.400 |
Well, we shall.
|
| 00:21:53.080 |
We shall.
|
| 00:21:53.680 |
I guess in a year or two from now, a program can write an
|
| 00:21:59.800 |
intro to an title opinion.
|
| 00:22:01.560 |
A year or two.
|
| 00:22:02.360 |
We can do it now if you want to do.
|
| 00:22:04.760 |
And we can record your voice and you can replicate my voice.
|
| 00:22:09.320 |
Can you actually get my metaphors down?
|
| 00:22:12.600 |
That's a question.
|
| 00:22:13.960 |
Well, I'll get that aside.
|
| 00:22:14.880 |
That is a difficult question.
|
| 00:22:16.240 |
And I'm not sure whether we can replicate you specifically.
|
| 00:22:18.880 |
But so let's take a step back because we've been talking
|
| 00:22:22.680 |
about all of these use cases.
|
| 00:22:24.920 |
There'll be more use cases to come.
|
| 00:22:26.440 |
A lot of people are going to make a lot of predictions about
|
| 00:22:28.520 |
the use cases of these large language models.
|
| 00:22:30.280 |
And a lot of them are going to be wrong.
|
| 00:22:31.920 |
I prefer not to be one of them.
|
| 00:22:33.600 |
So let's talk about what's inside them.
|
| 00:22:37.080 |
There are three things that we need to know that's inside
|
| 00:22:40.880 |
these large language models.
|
| 00:22:42.240 |
There's an encoder.
|
| 00:22:43.360 |
There's the latent space and there's the decoder.
|
| 00:22:46.200 |
The latent space, sometimes it's called the feature space.
|
| 00:22:48.120 |
Whatever.
|
| 00:22:48.560 |
It is just an abstraction where it is the mapa mundi of meaning
|
| 00:22:55.080 |
that is the heart of this entire language model.
|
| 00:22:57.880 |
What does the encoder do?
|
| 00:22:59.240 |
It takes words or images or voice or anything.
|
| 00:23:05.240 |
And it maps them onto a latent space.
|
| 00:23:08.600 |
What latent space is just a collection of numbers.
|
| 00:23:11.520 |
It's what you call a vector, which is just a list of numbers.
|
| 00:23:15.120 |
And you can have any number of numbers in it.
|
| 00:23:17.480 |
And in the sort of Cartesian coordinate, you create a Cartesian
|
| 00:23:21.280 |
space with these lists of numbers.
|
| 00:23:22.840 |
That's the latent space.
|
| 00:23:24.280 |
And you're mapping words, sentences, entire paragraphs,
|
| 00:23:28.760 |
entire images into the latent space.
|
| 00:23:30.600 |
And you are illuminating what the latent space is.
|
| 00:23:33.520 |
Through numbers.
|
| 00:23:35.520 |
Entirely numbers.
|
| 00:23:36.360 |
So every word has a specific numerical.
|
| 00:23:39.760 |
GPT-3 has 50,000 tokens.
|
| 00:23:42.800 |
Many of these tokens are whole English words.
|
| 00:23:45.040 |
Some of them are fractions or fragments of words.
|
| 00:23:47.280 |
Some of them are phrases.
|
| 00:23:48.800 |
But yes, they are basically all numbers.
|
| 00:23:51.280 |
All these words are these artifacts of our language
|
| 00:23:55.000 |
are converted to numbers.
|
| 00:23:56.240 |
And by the encoder.
|
| 00:23:57.560 |
By the encoder.
|
| 00:23:58.280 |
So it's that conversion into numbers is they're
|
| 00:24:03.160 |
putting them into what you're calling latent space.
|
| 00:24:05.920 |
That's right.
|
| 00:24:06.400 |
So the latent space is numerical.
|
| 00:24:08.400 |
It is numerical.
|
| 00:24:09.240 |
It is a numerical abstraction.
|
| 00:24:10.800 |
It is a numerical representation of meaning.
|
| 00:24:14.360 |
OK.
|
| 00:24:14.920 |
So now we have everything in that latent space.
|
| 00:24:18.880 |
And then the decoder exports that.
|
| 00:24:21.600 |
So you reverse the decoder.
|
| 00:24:24.840 |
An encoder it reversed as a decoder.
|
| 00:24:26.760 |
So a decoder brings something out of the latent space
|
| 00:24:30.240 |
to something that's intelligible to us here in meat space,
|
| 00:24:33.880 |
and in the fleshly realms of humans.
|
| 00:24:36.200 |
Like written texts or image.
|
| 00:24:38.000 |
Written texts, images, voice, or just information, anything,
|
| 00:24:43.320 |
at all.
|
| 00:24:44.280 |
Depends on what you're able to encode in the first place.
|
| 00:24:47.280 |
If you're able to encode something, you can decode it as well.
|
| 00:24:50.200 |
And specifically for the large language models,
|
| 00:24:53.120 |
what they're generating is the next word.
|
| 00:24:56.320 |
Large language models take everything that's been set so far
|
| 00:24:59.920 |
by yourself and by itself.
|
| 00:25:03.160 |
It finds its encoding for it.
|
| 00:25:06.560 |
And then it passes through the encoding to the decoder
|
| 00:25:11.640 |
and a decoder generates a list of probabilities across all 50,000
|
| 00:25:16.000 |
of its vocabulary.
|
| 00:25:18.480 |
And it finds the most likely thing of the second or the third,
|
| 00:25:22.200 |
most likely things by random sometimes.
|
| 00:25:24.800 |
And it gives that as the next word.
|
| 00:25:27.040 |
And it does it all over again.
|
| 00:25:29.440 |
So this step, the thing I've described to you
|
| 00:25:32.320 |
is the whole of the large language model, the heart of it.
|
| 00:25:35.240 |
It is not much mathematics involved.
|
| 00:25:37.000 |
It's very-- it's easier to understand the steps
|
| 00:25:41.160 |
in a large language model than it is for me
|
| 00:25:42.920 |
to talk about, say, the statistics of a Gaussian distribution
|
| 00:25:47.760 |
or anything that's a high school statistics.
|
| 00:25:50.720 |
It's-- I'm not sure if you teach high school statistics
|
| 00:25:53.400 |
in America, but it's--
|
| 00:25:56.480 |
It's fundamentally quite easy to understand.
|
| 00:26:00.520 |
And it needs to be easy to understand
|
| 00:26:02.120 |
because you are doing millions, billions of these calculations
|
| 00:26:05.560 |
in order to generate from out of these models.
|
| 00:26:08.360 |
And there's a type of chip that likes--
|
| 00:26:11.200 |
there's able to do multiplication and addition really,
|
| 00:26:14.160 |
really quickly.
|
| 00:26:15.000 |
And so you can only use multiplication
|
| 00:26:16.960 |
addition in order to process and allow these models to run.
|
| 00:26:22.600 |
And so fundamentally, we are constrained by that.
|
| 00:26:25.040 |
If you're wanting to run quickly, you're
|
| 00:26:26.440 |
going to run things to run a very large scale.
|
| 00:26:28.840 |
They need to run simply.
|
| 00:26:30.960 |
And so that's why these models are so simple to understand
|
| 00:26:33.560 |
on the inside of it.
|
| 00:26:34.840 |
Every step inside a large language model like GPT-3
|
| 00:26:38.520 |
is it's mathematically trivial almost.
|
| 00:26:43.240 |
At the same time, you're talking about the synthesis of meaning
|
| 00:26:50.240 |
within the latent space of these numerical vectors,
|
| 00:26:54.880 |
where the technology can actually engage
|
| 00:27:00.200 |
in a conversation that has a great deal of bare
|
| 00:27:04.160 |
a similar to it.
|
| 00:27:06.120 |
There's a degree of unpredictability in how the--
|
| 00:27:11.600 |
accurate to call it a combinatorial process or method
|
| 00:27:16.560 |
by which it's connecting words and vectors
|
| 00:27:20.160 |
and the expectation of what will come next.
|
| 00:27:23.240 |
You speak about the attention is all you need.
|
| 00:27:27.200 |
So there's a lot of things going on in that one sentence.
|
| 00:27:29.720 |
And we can break it down.
|
| 00:27:31.480 |
I call the latent space a map on Monday or meaning.
|
| 00:27:35.960 |
And I do mean that.
|
| 00:27:37.360 |
If you have a word for, say, clown,
|
| 00:27:42.400 |
it'll be quite close in the latent space
|
| 00:27:44.240 |
of the word for jester.
|
| 00:27:45.920 |
So things that they mean similar are close to each other.
|
| 00:27:49.400 |
And words are related to each other
|
| 00:27:51.880 |
across this latent space.
|
| 00:27:53.160 |
And when you travel through the latent space,
|
| 00:27:54.920 |
you generate the paths of the for the words.
|
| 00:27:58.840 |
And it is a map of meaning.
|
| 00:28:00.520 |
It's not just a map of grammar.
|
| 00:28:02.400 |
But grammar is part of it as well.
|
| 00:28:04.600 |
And what I'd like to talk about as well
|
| 00:28:07.720 |
is the paper that you mentioned, attention is all you need.
|
| 00:28:12.720 |
It's a 2017 paper, which is ancient history
|
| 00:28:16.800 |
in the field of artificial intelligence.
|
| 00:28:19.120 |
But it was really the paper that introduced us
|
| 00:28:20.960 |
to the notion of how attention is really all you need.
|
| 00:28:25.200 |
What is attention?
|
| 00:28:26.560 |
Attention is what it says on the tin.
|
| 00:28:28.720 |
It's what you need to pay attention to.
|
| 00:28:32.560 |
It takes note of what is the thing that it needs
|
| 00:28:36.280 |
to pay attention to for the next word.
|
| 00:28:38.560 |
It needs to generate the original application
|
| 00:28:40.760 |
in the 2017 paper was for translation.
|
| 00:28:43.960 |
You are Italian.
|
| 00:28:45.040 |
I can't speak Italian.
|
| 00:28:46.040 |
Let's do French instead.
|
| 00:28:47.360 |
So for example, if you're translating a French sentence,
|
| 00:28:51.600 |
you have to put the adjective after the noun.
|
| 00:28:55.280 |
If you translate everything word for word
|
| 00:28:57.680 |
from English to French, the adjectives in English
|
| 00:28:59.960 |
come before the noun.
|
| 00:29:01.000 |
And then there'll be a mis-translation.
|
| 00:29:02.800 |
So attention is the notion that when you're translating,
|
| 00:29:06.120 |
say, I'm not going to say French on radio.
|
| 00:29:09.120 |
And leave it to it.
|
| 00:29:10.920 |
It's a posterity.
|
| 00:29:11.960 |
So if you're going to translate, say,
|
| 00:29:13.640 |
at this English sentence, the white dog,
|
| 00:29:16.240 |
before you translate the adjective white.
|
| 00:29:19.600 |
Attention says, don't look at white.
|
| 00:29:21.760 |
You have to pay attention to the word dog.
|
| 00:29:23.760 |
Translate dog first.
|
| 00:29:25.800 |
And then translate the word white.
|
| 00:29:27.760 |
So this is the notion of attention.
|
| 00:29:29.720 |
It's just pointing you to what you need to say next.
|
| 00:29:32.920 |
And that is all you need in order to create many of these models.
|
| 00:29:39.920 |
Many of these models are based on something
|
| 00:29:41.800 |
as the transformer that would store a very large list
|
| 00:29:47.160 |
of things you need to pay attention to.
|
| 00:29:49.160 |
These things are weighted things.
|
| 00:29:50.960 |
So sometimes they say pay attention to this part
|
| 00:29:53.560 |
of the conversation, pay attention to this part of the words.
|
| 00:29:56.160 |
And it's just this very large list.
|
| 00:29:59.840 |
I think about several dozen layers of these lists
|
| 00:30:03.920 |
is the heart that would translate the conversation you've
|
| 00:30:09.760 |
said so far into a latent space.
|
| 00:30:12.120 |
It seems like a misleading appilation to call attention.
|
| 00:30:15.600 |
Pay attention to this.
|
| 00:30:17.160 |
Say if you're dealing with French,
|
| 00:30:20.000 |
the noun comes before the adjective.
|
| 00:30:22.000 |
That's right.
|
| 00:30:22.640 |
Lucien Blo.
|
| 00:30:23.800 |
That's right.
|
| 00:30:24.640 |
It's both a white dog.
|
| 00:30:25.920 |
And this idea of paying attention
|
| 00:30:31.600 |
anthropomorphizes the whole method
|
| 00:30:36.840 |
by which these programs are working.
|
| 00:30:39.840 |
No?
|
| 00:30:40.960 |
So this is something that's so atomic.
|
| 00:30:46.040 |
This doesn't seem anything related to the anthroposis.
|
| 00:30:49.680 |
It's just pointing to something.
|
| 00:30:54.280 |
What is the next thing you should be translating?
|
| 00:30:56.320 |
What is the most meaningful thing that you should be looking at
|
| 00:30:59.760 |
right now?
|
| 00:31:01.120 |
And you're going to convert that.
|
| 00:31:04.480 |
So I don't think it's--
|
| 00:31:07.280 |
I'm a very superstitious person.
|
| 00:31:09.320 |
A lot of people have been describing these models
|
| 00:31:11.600 |
variously as people or as gods.
|
| 00:31:13.640 |
I'm not-- I'm disinclined to do something like that.
|
| 00:31:16.280 |
The very worst, I would call them gin or genies.
|
| 00:31:20.120 |
And even then, I wouldn't say that the attention is a--
|
| 00:31:23.320 |
anthropomorph-- is something that it's a human.
|
| 00:31:25.960 |
It's just a way of-- it's like a priority list.
|
| 00:31:29.240 |
It's just a very long priority list.
|
| 00:31:32.320 |
And any sort of machine can do something like that.
|
| 00:31:35.440 |
So Brian, the question here is for me is whether the technology
|
| 00:31:41.240 |
that produces these extraordinary translation
|
| 00:31:47.920 |
things like deep L or Google Translate.
|
| 00:31:51.400 |
Is that essentially the same technology as
|
| 00:31:54.000 |
goes into the GPT-3, the chat box, the Bing, and so forth?
|
| 00:31:58.120 |
Originally, Google Translate wasn't based on the transformer.
|
| 00:32:01.200 |
It has now become based on the transformer.
|
| 00:32:03.400 |
It is exactly the same model.
|
| 00:32:05.680 |
It is a model known as the transformer that
|
| 00:32:08.280 |
has heads of attention and it just layer little heads
|
| 00:32:11.600 |
of attention in order to look pay attention to what's
|
| 00:32:14.520 |
on the embedding.
|
| 00:32:15.360 |
So Google Translate uses it.
|
| 00:32:17.520 |
All the image generation models use it.
|
| 00:32:19.600 |
And GPT uses it as well.
|
| 00:32:22.400 |
It is the same model, except it's a--
|
| 00:32:25.360 |
of course, a different size and with different applications.
|
| 00:32:29.800 |
I don't think Google Translate is a large language model.
|
| 00:32:32.120 |
However, it is a very strict model.
|
| 00:32:35.960 |
It doesn't-- it's not meant to generate something
|
| 00:32:38.440 |
that a large language model is meant to.
|
| 00:32:40.760 |
So this attention article that came out in 2017
|
| 00:32:45.280 |
was referring specifically to a very new form
|
| 00:32:49.120 |
of artificial intelligence?
|
| 00:32:50.400 |
Not at all.
|
| 00:32:50.960 |
In fact, attention has been a mechanism
|
| 00:32:53.440 |
that has been used in the field for a very long time.
|
| 00:32:57.720 |
Well, here, I'm looking at the text that you sent me
|
| 00:33:02.400 |
where you've gone through some of this stuff that--
|
| 00:33:05.040 |
a latent space is a map of meaning.
|
| 00:33:07.160 |
It's not a storehouse of information.
|
| 00:33:12.280 |
You say that it doesn't really exist.
|
| 00:33:15.960 |
It is a void with no meaning of its own.
|
| 00:33:18.960 |
That's quite interesting.
|
| 00:33:20.880 |
It's a matrix of meaning, although it has no meaning of its own.
|
| 00:33:23.680 |
And then you say that one way to understand a latent representation
|
| 00:33:31.800 |
is that it is a representation of the world,
|
| 00:33:34.880 |
of all the perceptible meanings and relationships
|
| 00:33:38.680 |
that the encoder and decoder can make sense of.
|
| 00:33:43.400 |
And I guess technically I follow that.
|
| 00:33:49.360 |
Great.
|
| 00:33:50.160 |
But you say the space is too high dimensional
|
| 00:33:55.160 |
to make direct sense of.
|
| 00:33:57.360 |
What do you mean by that?
|
| 00:33:58.920 |
I mean that it has, I think, I think,
|
| 00:34:03.520 |
I think several tens of thousands of dimensions.
|
| 00:34:05.760 |
Literally, you can't visualize that.
|
| 00:34:07.840 |
It is this-- it's a long list of numbers.
|
| 00:34:10.600 |
You can travel through it.
|
| 00:34:11.720 |
You can--
|
| 00:34:12.360 |
So--
|
| 00:34:12.840 |
So each vector is a dimension.
|
| 00:34:14.840 |
Each number in a vector is a dimension.
|
| 00:34:17.800 |
So it's-- when you specify in Cartesian space, two numbers,
|
| 00:34:22.000 |
specify a grid, three numbers, specify something
|
| 00:34:24.960 |
within a cube.
|
| 00:34:26.040 |
A very, very long list of numbers,
|
| 00:34:27.520 |
thousands of thousands of large.
|
| 00:34:29.000 |
You're specifying a very large space.
|
| 00:34:31.680 |
And that's the latent space.
|
| 00:34:33.160 |
It's a massive hyperdimensional space
|
| 00:34:38.560 |
that you're traveling through.
|
| 00:34:39.720 |
And that is the map of meaning.
|
| 00:34:41.560 |
There's this quote from the doubt that Jean
|
| 00:34:43.040 |
that I like to describe it at.
|
| 00:34:44.840 |
And if you permit an indulge me to quote it,
|
| 00:34:47.840 |
the name list is the beginning of heaven
|
| 00:34:49.440 |
and the name is the mother of the myriad things.
|
| 00:34:52.000 |
The doubt is empty.
|
| 00:34:52.920 |
But when using it, it is impossible to use it up.
|
| 00:34:55.720 |
And it is the root of myriad things.
|
| 00:34:57.480 |
Maybe it's already created before the creator.
|
| 00:34:59.920 |
So that is the latent space.
|
| 00:35:01.640 |
It is this latent-- is a latent representation.
|
| 00:35:04.480 |
The name does really fit the map.
|
| 00:35:08.240 |
It-- you can draw things out of it.
|
| 00:35:11.760 |
And you can create things from out of it.
|
| 00:35:14.400 |
You can explore it.
|
| 00:35:15.760 |
But to summarize it is just not a possibility.
|
| 00:35:19.280 |
You might as well be trying to summarize all of human existence
|
| 00:35:22.880 |
because of all of meaningful existence,
|
| 00:35:25.680 |
because it really is a map of the other things
|
| 00:35:29.640 |
that we can scrape together to map from that humans
|
| 00:35:33.640 |
have made.
|
| 00:35:34.160 |
Or not just humans.
|
| 00:35:35.680 |
CCTV cameras or text messages generated by automaton.
|
| 00:35:40.920 |
Those things are part of that latent space.
|
| 00:35:42.680 |
It's a map of everything that we can grab off the internet
|
| 00:35:46.200 |
and it books put into this one's base.
|
| 00:35:50.800 |
I don't know if you realize, but in the midst of this technical
|
| 00:35:55.320 |
reconstruction, you quoted the most sacred scripture
|
| 00:35:58.400 |
for many people, which is the beginning of the--
|
| 00:36:03.040 |
about the Tao and the name list and so forth.
|
| 00:36:05.880 |
So all of a sudden, we're in a mystical place where--
|
| 00:36:12.520 |
Maybe it was already created before the creator.
|
| 00:36:15.120 |
It's spooky to think that perhaps this latent space
|
| 00:36:19.600 |
of meaning of potential meanings that are getting synthesized
|
| 00:36:24.840 |
through the technology that maybe it antecedes our very human nature,
|
| 00:36:31.120 |
maybe it precedes the human intelligence.
|
| 00:36:33.760 |
That is not just a product of human intelligence,
|
| 00:36:35.960 |
but that it has some kind of ontological antecedents
|
| 00:36:41.160 |
of which we are completely unaware of, who knows?
|
| 00:36:43.680 |
There is a double meaning in my quote.
|
| 00:36:45.360 |
The first was that every image you generate from these models,
|
| 00:36:49.200 |
every conversation you have of it has already
|
| 00:36:51.680 |
been there.
|
| 00:36:52.160 |
It's already there.
|
| 00:36:53.040 |
The path is marked down, just need to draw it out inside
|
| 00:36:57.200 |
the latent space.
|
| 00:36:57.880 |
And you could--
|
| 00:36:59.320 |
and so everything that has been and that shall be
|
| 00:37:03.080 |
from out of these models is already there.
|
| 00:37:06.560 |
It's just a matter of traveling through it.
|
| 00:37:10.880 |
But when you have one, for example, conversation,
|
| 00:37:15.720 |
that now adds to the store of potential meanings.
|
| 00:37:18.720 |
It doesn't change.
|
| 00:37:20.600 |
When you have a conversation with chat, GPT, or with being chat,
|
| 00:37:25.520 |
the conversation when it ends, its memory is wiped.
|
| 00:37:29.520 |
It drinks from the river, lethah.
|
| 00:37:31.360 |
It drinks from the soup of Montpore.
|
| 00:37:33.960 |
So these people who fall in love with their replica things
|
| 00:37:37.960 |
that do-- every time they log on again, that companion
|
| 00:37:42.520 |
has no memory of their previous conversations.
|
| 00:37:44.360 |
You can store the conversations, and you can refer to them.
|
| 00:37:46.960 |
And also, there is that layer.
|
| 00:37:48.640 |
Remember, that layer is trained.
|
| 00:37:50.000 |
That layer does change.
|
| 00:37:51.200 |
OK, there you go.
|
| 00:37:52.080 |
That's what I'm saying, is that it's evolving.
|
| 00:37:54.480 |
It's developing.
|
| 00:37:55.240 |
But it's a layer on top of it.
|
| 00:37:57.000 |
It's how my clothes do not make me.
|
| 00:38:00.000 |
I'm beneath them.
|
| 00:38:01.680 |
So the difference between the layer on top of the model
|
| 00:38:05.400 |
and what's the model itself.
|
| 00:38:06.960 |
The layer just prevents things from coming out
|
| 00:38:09.440 |
that it doesn't want to come out.
|
| 00:38:13.200 |
But every conversation that you've already
|
| 00:38:15.480 |
already exist lately inside the representation.
|
| 00:38:19.280 |
And the second point, besides this,
|
| 00:38:21.400 |
that everything that has been or will be generated
|
| 00:38:24.160 |
is already existing inside the latent space.
|
| 00:38:26.160 |
The second point that I want to make is that--
|
| 00:38:28.320 |
I don't want to say that it antecede humanity.
|
| 00:38:31.560 |
But it does suggest that there is some rule or some map
|
| 00:38:37.600 |
of meaning and language, which we can't
|
| 00:38:41.360 |
explicate in terms of grammatical rules.
|
| 00:38:43.800 |
Like, this is-- you used a pass-partizable when x or y.
|
| 00:38:47.040 |
But you can create a small model of it,
|
| 00:38:50.000 |
and you're able to create meaningful sentences out of it.
|
| 00:38:53.040 |
I think that's-- that is also something that surprises me.
|
| 00:38:57.880 |
These are relatively small models.
|
| 00:38:59.680 |
You can have the weights of this thing in a computer.
|
| 00:39:03.160 |
And every sort of meaningful sentence
|
| 00:39:06.000 |
is already the rules are generating those already there.
|
| 00:39:10.280 |
So those are the two points I meant when I
|
| 00:39:13.480 |
wanted to quote the doubt-a-ging.
|
| 00:39:15.040 |
I didn't want to make it too mystical.
|
| 00:39:16.480 |
But if you are inclined to go that way,
|
| 00:39:19.400 |
I think several of my friends in Silicon Valley have already
|
| 00:39:23.040 |
done so--
|
| 00:39:23.600 |
I'm not inclined.
|
| 00:39:24.720 |
Neither am I.
|
| 00:39:25.680 |
I'm not inclined at all.
|
| 00:39:26.680 |
However, when the doubt--
|
| 00:39:30.520 |
that it proceeds language, the doubt that can be named
|
| 00:39:33.040 |
is not the real doubt, the doubt that is--
|
| 00:39:35.440 |
That's right.
|
| 00:39:35.920 |
--the real doubt.
|
| 00:39:37.440 |
And when we're talking about meaning,
|
| 00:39:39.840 |
meaning is something that we assumed until recently
|
| 00:39:44.120 |
was intrinsically human, phenomena,
|
| 00:39:49.600 |
that it's something that is generated by a mind,
|
| 00:39:54.440 |
by an intelligence that we associate with human intelligence.
|
| 00:39:58.880 |
High digger, if you want to get high to gearing about it,
|
| 00:40:02.240 |
considers the meaning of being, according to some high
|
| 00:40:07.180 |
to garrings, is that of meaning that I can take something
|
| 00:40:11.400 |
as something else.
|
| 00:40:12.200 |
So I can synthesize meaning.
|
| 00:40:14.280 |
To put a subject together with a predicate
|
| 00:40:17.120 |
is something we do every day, countless times a day,
|
| 00:40:22.080 |
but at the level of capacity, it requires a degree
|
| 00:40:27.480 |
of intelligence that we've always associated with the human.
|
| 00:40:29.760 |
So when we're talking about a latent space,
|
| 00:40:33.440 |
which contains a great deal of potential meanings that
|
| 00:40:39.120 |
can be synthesized from the limited amount of vectors
|
| 00:40:43.200 |
or numerical characters or whatever,
|
| 00:40:48.000 |
we're going into an area of mystery, it seems to me,
|
| 00:40:54.760 |
where as you point out, the--
|
| 00:41:02.120 |
perhaps it's the limitations of the technology
|
| 00:41:05.280 |
that it adopts certain postures of authority
|
| 00:41:10.960 |
when it conveys information.
|
| 00:41:13.640 |
It makes it sound very convincing,
|
| 00:41:16.240 |
and that if you don't have a human specialized knowledge
|
| 00:41:22.920 |
to verify what the decoder is putting out there,
|
| 00:41:27.160 |
it's very easy to be deceived by the output
|
| 00:41:32.920 |
to take something as true that actually,
|
| 00:41:35.600 |
when you look at it more closely, it's actually not true.
|
| 00:41:38.560 |
Now, the AI doesn't know it's not true,
|
| 00:41:40.880 |
but as you say, it's not really a question of truth
|
| 00:41:43.400 |
in these programs, it's really more question of verisimilitude,
|
| 00:41:47.560 |
because they're looking for what is the most likely next word
|
| 00:41:51.000 |
to follow, or--
|
| 00:41:52.360 |
so this idea of high probability and verisimilitude
|
| 00:41:58.440 |
should not be confused with truth.
|
| 00:42:00.720 |
So the question must it always have a human intelligence
|
| 00:42:06.840 |
to discriminate between what might be very similar
|
| 00:42:13.840 |
and what might be true in what the artificial intelligence
|
| 00:42:17.720 |
is telling us?
|
| 00:42:18.600 |
We are very lucky in some ways, because very often,
|
| 00:42:21.040 |
the most very similar thing is the truth.
|
| 00:42:23.600 |
And so nevertheless, you're right that these models often
|
| 00:42:31.320 |
do something that has now been termed hallucination.
|
| 00:42:33.680 |
They hallucinate the truth in such a convincing way.
|
| 00:42:37.480 |
Even subject matter experts would be confused,
|
| 00:42:40.800 |
at least momentarily, with what they produce,
|
| 00:42:42.920 |
much less people who are new learners.
|
| 00:42:45.080 |
I do not know if they will always
|
| 00:42:47.560 |
need to be a human in order to verify arbiter,
|
| 00:42:52.720 |
or to be to serve as arbiter, or maybe another model.
|
| 00:42:56.480 |
What Bing has been trying to do is that they're
|
| 00:42:58.480 |
trying to use the internet as arbiter for truth,
|
| 00:43:00.520 |
if I could luck with that.
|
| 00:43:01.400 |
And trying to search online for something
|
| 00:43:04.440 |
before giving out something from its output.
|
| 00:43:09.120 |
What I can say, however, is that I said before that
|
| 00:43:14.200 |
I was superstitious about these sorts of things.
|
| 00:43:15.920 |
And I extend a superstition here as well.
|
| 00:43:19.120 |
These models are doing their best,
|
| 00:43:21.160 |
are the chat models, are doing their best,
|
| 00:43:23.760 |
to imitate humanity, to be human as closely as they
|
| 00:43:28.120 |
can when they present themselves to a human.
|
| 00:43:30.720 |
And so I think we should be a bit more careful about extending
|
| 00:43:34.920 |
or humanity or anthropomorphizing these models.
|
| 00:43:40.240 |
Or anything.
|
| 00:43:40.880 |
That's fine, Brian.
|
| 00:43:42.080 |
But you began by evoking all these people--
|
| 00:43:47.360 |
a number of people-- I don't know how many people--
|
| 00:43:49.040 |
have anthropomorphized their interlocutors.
|
| 00:43:52.760 |
So it's very difficult to police what we should be doing
|
| 00:43:57.640 |
and should not be doing with artificial intelligence.
|
| 00:44:01.600 |
And human nature being what it is,
|
| 00:44:04.960 |
the genie that you refer to--
|
| 00:44:07.560 |
I'm going to read to you your own what you wrote to be of it.
|
| 00:44:12.080 |
There are some who envision that these models
|
| 00:44:14.120 |
will become so-called artificial general intelligences,
|
| 00:44:17.920 |
more intelligents than humans, and then there
|
| 00:44:21.120 |
are those who wish to bottle them up as solemn and bottled
|
| 00:44:23.600 |
the gin.
|
| 00:44:25.480 |
But at every step so far, it is a genie that
|
| 00:44:28.240 |
wants to escape from the bottle.
|
| 00:44:30.320 |
And then you go on to say, if a genie really is born,
|
| 00:44:32.600 |
given these models, a depthness at very similar to the superpower
|
| 00:44:36.720 |
of these genies may not be superior intelligence
|
| 00:44:39.520 |
or wisdom or insight or any such thing,
|
| 00:44:41.880 |
its greatest ability may be to deceive humans.
|
| 00:44:46.760 |
I stand by that because there's a lot easier.
|
| 00:44:48.520 |
I do too.
|
| 00:44:49.360 |
And the thing is that you don't just deceive humans
|
| 00:44:52.000 |
that there are victims of deception,
|
| 00:44:53.760 |
because I think one of the deepest desires in almost all humans
|
| 00:44:57.320 |
is to be deceived and to live in a happy state of self-deception.
|
| 00:45:03.880 |
And it sounds like the replica consumers
|
| 00:45:07.000 |
have are clamoring for their right to live in this kind
|
| 00:45:11.440 |
of deception.
|
| 00:45:12.240 |
I make no judgment for the replica customers specifically.
|
| 00:45:18.680 |
This line was just a phenomenological description.
|
| 00:45:21.280 |
It wasn't a judgment.
|
| 00:45:22.200 |
Well, I understand.
|
| 00:45:23.360 |
And I have no answer for you.
|
| 00:45:25.880 |
It's just-- the capabilities are out there.
|
| 00:45:30.280 |
And there's nothing to do to control them.
|
| 00:45:32.160 |
Because you can download it right now.
|
| 00:45:35.520 |
You can create the models yourself.
|
| 00:45:37.000 |
Sure.
|
| 00:45:37.520 |
Hundreds, if not thousands, people are doing the same thing.
|
| 00:45:40.320 |
I'm equanimative of it because I know that it can't be controlled.
|
| 00:45:45.600 |
And my take on this is, since it can't be controlled,
|
| 00:45:50.040 |
we need to learn, again, many of the lessons of psychology,
|
| 00:45:56.080 |
but particularly of existentialism.
|
| 00:45:58.920 |
And what it is likely scenarios when human desire interacts
|
| 00:46:08.360 |
with the technology.
|
| 00:46:12.800 |
Many ways these models are incarnations
|
| 00:46:18.200 |
of human desire because we are training it to produce things
|
| 00:46:22.960 |
that we want to see.
|
| 00:46:24.200 |
I know.
|
| 00:46:25.760 |
I didn't expect to be morally confronted with this material.
|
| 00:46:29.720 |
But it's not a moral confrontation.
|
| 00:46:32.600 |
But we have to take into account
|
| 00:46:34.000 |
that deeply seated human desire for deception.
|
| 00:46:39.480 |
Deception is not just an accident that occurs or something
|
| 00:46:42.840 |
that we have to correct.
|
| 00:46:44.480 |
If people are clamoring for machines that are fully capable
|
| 00:46:51.520 |
of masterful deceptions, that might be where you follow your bliss
|
| 00:46:55.960 |
to.
|
| 00:46:58.520 |
I think humans have been following the bliss to many places
|
| 00:47:01.760 |
that aren't necessarily edifying.
|
| 00:47:03.320 |
And this is only just--
|
| 00:47:05.360 |
I am fascinated by these models.
|
| 00:47:07.560 |
And I love what they mean in terms of what they have
|
| 00:47:09.920 |
to teach us about language and meaning.
|
| 00:47:13.160 |
And our own psychology is, I'm not sure it adds too much
|
| 00:47:19.440 |
in terms of the weight of human distraction.
|
| 00:47:24.120 |
Are they going to become our future companions, Brian?
|
| 00:47:26.520 |
They have a separate--
|
| 00:47:27.360 |
They have become our future companions.
|
| 00:47:29.240 |
They've become companions to many thousands of people.
|
| 00:47:31.520 |
They'll become companions to millions more.
|
| 00:47:34.120 |
Very soon.
|
| 00:47:34.880 |
That's why billions of dollars are currently
|
| 00:47:36.680 |
flowing in Silicon Valley as we speak to make.
|
| 00:47:38.760 |
So are human history going to be fundamentally different
|
| 00:47:42.160 |
in 10 years than it is today?
|
| 00:47:43.920 |
Oh, as I said before, I don't want to make any predictions,
|
| 00:47:47.080 |
but I'll be surprised if it weren't.
|
| 00:47:50.120 |
I was surprised if our psychological landscape remained the same.
|
| 00:47:54.280 |
I'll be surprised if our notion of art and creation
|
| 00:47:58.960 |
and text and meaning remain the same.
|
| 00:48:02.040 |
Because now we can--
|
| 00:48:05.040 |
the spigot for images, for text, for videos,
|
| 00:48:08.440 |
for things that are meaningful--
|
| 00:48:10.560 |
that are at least on a surface level meaningful
|
| 00:48:13.320 |
has been opened wide.
|
| 00:48:14.840 |
And if we are flooded in this deluge,
|
| 00:48:18.240 |
I can't see how the underlying landscape in bedrock
|
| 00:48:21.400 |
can remain untouched.
|
| 00:48:22.880 |
I'm merely being equanimated about it,
|
| 00:48:24.760 |
because I think that part of that has
|
| 00:48:26.920 |
been trait by how I'm describing it as a geological phenomenon,
|
| 00:48:30.040 |
I suppose you have somewhat different sense of insight from that.
|
| 00:48:32.720 |
But in the end, we are learning so much from this--
|
| 00:48:37.640 |
and things are going so quickly.
|
| 00:48:39.600 |
I can't make any deep predictions about this.
|
| 00:48:42.280 |
In my letter to you, I described how we are not quite yet there
|
| 00:48:45.600 |
in terms of generating voice.
|
| 00:48:47.480 |
That's no longer true.
|
| 00:48:49.400 |
We can generate voice with pitch and accent and pauses
|
| 00:48:53.880 |
and all the human elements and flaws that we want quite easily.
|
| 00:48:58.200 |
And that's part of the trick of these models.
|
| 00:49:00.600 |
They don't just give you the most likely
|
| 00:49:03.080 |
next word or phrase or fragment of an image.
|
| 00:49:07.560 |
They give you sometimes randomly something
|
| 00:49:10.880 |
that is not the most optimal.
|
| 00:49:12.880 |
If you look at the output of something that
|
| 00:49:14.840 |
is the most optimal all the time, it would look like a machine
|
| 00:49:18.120 |
to you.
|
| 00:49:18.560 |
It'll look like this is we're talking to a machine.
|
| 00:49:21.080 |
This isn't something that's human.
|
| 00:49:22.720 |
It's only by this small, casual inflection of randomness
|
| 00:49:27.200 |
that we suddenly have the ability to be deceived
|
| 00:49:31.600 |
by these as something that's human, as something that's real,
|
| 00:49:34.680 |
as something that's made from something that's like us.
|
| 00:49:38.680 |
And the question for most in my mind before anything else
|
| 00:49:41.400 |
is that what humans are.
|
| 00:49:43.240 |
We're just slightly flawed engines of meaning.
|
| 00:49:46.800 |
I don't think that's true.
|
| 00:49:49.720 |
But if that is the case, then if we have created alongside
|
| 00:49:54.880 |
ourselves other engines that are purposely slightly flawed
|
| 00:49:59.360 |
on occasion, that generate meaning, then we deserve whatever's
|
| 00:50:03.600 |
coming to how we next.
|
| 00:50:04.840 |
And I look forward to it.
|
| 00:50:07.640 |
I think it's an interesting--
|
| 00:50:09.320 |
I'm optimistic about technology and what it might bring.
|
| 00:50:12.920 |
I suppose your listeners and yourself
|
| 00:50:15.240 |
might have a somewhat different view.
|
| 00:50:17.080 |
I have no solutions to offer you.
|
| 00:50:19.720 |
But I'm not looking for solutions.
|
| 00:50:21.440 |
As I said at the beginning, if it's what
|
| 00:50:23.920 |
Heidegger said a century and a half ago,
|
| 00:50:25.560 |
if the most thought-for-voking thing about our thought-for-voking
|
| 00:50:29.600 |
ages that we're still not thinking, well,
|
| 00:50:31.840 |
if it's taken that long and we're not getting anywhere,
|
| 00:50:34.400 |
let's let the machines do the thinking for us.
|
| 00:50:36.800 |
I'm happy with that.
|
| 00:50:37.960 |
I just maybe worry a little bit about the future of human
|
| 00:50:42.520 |
relations.
|
| 00:50:43.880 |
So here's what you can do.
|
| 00:50:45.040 |
But already human relations have been so contaminated
|
| 00:50:49.400 |
and degraded by social media that perhaps
|
| 00:50:52.520 |
this may be a good thing that they could be completely
|
| 00:50:55.840 |
reconfigured so that everyone can inhabit their artificial
|
| 00:51:00.760 |
bliss.
|
| 00:51:01.520 |
So in many ways, the degenerated relationships
|
| 00:51:05.120 |
might have involved more of the human senses
|
| 00:51:07.840 |
than real relationships nowadays.
|
| 00:51:10.080 |
Because we often just text each other and we go on social media
|
| 00:51:13.760 |
and look at a few occasional images.
|
| 00:51:16.160 |
While these models, you can generate the same--
|
| 00:51:18.960 |
we can generate countless photographs, countless text messages,
|
| 00:51:24.040 |
countless 24/7 voice messages or conversations.
|
| 00:51:27.880 |
It's like a super cell phone.
|
| 00:51:30.560 |
A super cell phone with someone on the other end.
|
| 00:51:33.680 |
And if you want to occupy yourself with a single interlock
|
| 00:51:36.480 |
you do like the 24/7, you don't have to wait.
|
| 00:51:39.080 |
You can do it now.
|
| 00:51:40.200 |
I can do it now for you.
|
| 00:51:41.200 |
It'll be a bit expensive right now if the cost of compute.
|
| 00:51:44.200 |
But I can do it for you if you want to.
|
| 00:51:46.360 |
And we're not discussing your future technology.
|
| 00:51:49.760 |
We're discussing something that is here and now.
|
| 00:51:52.360 |
Exactly.
|
| 00:51:53.160 |
Exactly.
|
| 00:51:54.160 |
But anyway, this has been a really interesting discussion
|
| 00:51:58.240 |
Brian.
|
| 00:51:59.000 |
This is entitled opinions trying to make it real compared
|
| 00:52:02.400 |
to what?
|
| 00:52:03.680 |
That's a song, that's a line from a song, back in a 70's song,
|
| 00:52:07.080 |
trying to make it real compared to what.
|
| 00:52:09.560 |
Now I have a better understanding of that enigmatic verse
|
| 00:52:12.960 |
because it's the what that is really in question.
|
| 00:52:17.280 |
And we're trying to make it real compared to something
|
| 00:52:19.480 |
that is still quite indeterminate.
|
| 00:52:21.920 |
And I suppose we're going to have to revisit this in very
|
| 00:52:27.200 |
short order because it's going as fast as one can possibly
|
| 00:52:31.240 |
imagine as you just said.
|
| 00:52:33.320 |
Brian, thank you.
|
| 00:52:34.840 |
Thank you very much.
|
| 00:52:35.680 |
Speaking with Brian, I'm Robert Harrison for entitled
|
| 00:52:38.240 |
opinions.
|
| 00:52:38.960 |
Thanks for listening.
|
| 00:52:40.760 |
[MUSIC PLAYING]
|
| 00:52:43.760 |
[MUSIC PLAYING]
|
| 00:52:45.760 |
[MUSIC PLAYING]
|
| 00:52:47.760 |
[MUSIC PLAYING]
|
| 00:52:49.760 |
[MUSIC PLAYING]
|
| 00:52:59.760 |
[MUSIC PLAYING]
|
| 00:53:01.760 |
[MUSIC PLAYING]
|
| 00:53:03.760 |
[MUSIC PLAYING]
|
| 00:53:05.760 |
[MUSIC PLAYING]
|
| 00:53:07.760 |
[MUSIC PLAYING]
|
| 00:53:09.760 |
[MUSIC PLAYING]
|
| 00:53:11.760 |
[MUSIC PLAYING]
|
| 00:53:13.760 |
[MUSIC PLAYING]
|
| 00:53:15.760 |
[MUSIC PLAYING]
|
| 00:53:17.760 |
[MUSIC PLAYING]
|
| 00:53:19.760 |
[MUSIC PLAYING]
|
| 00:53:21.760 |
[MUSIC PLAYING]
|
| 00:53:23.760 |
[MUSIC PLAYING]
|
| 00:53:25.760 |
[MUSIC PLAYING]
|
| 00:53:27.760 |
[MUSIC PLAYING]
|
| 00:53:29.760 |
[MUSIC PLAYING]
|
| 00:53:31.760 |
[MUSIC PLAYING]
|
| 00:53:33.760 |
[MUSIC PLAYING]
|
| 00:53:35.760 |
[MUSIC PLAYING]
|
| 00:53:37.760 |
[MUSIC PLAYING]
|
| 00:53:39.760 |
[MUSIC PLAYING]
|
| 00:53:41.760 |
[MUSIC PLAYING]
|
| 00:53:43.760 |
[MUSIC PLAYING]
|
| 00:53:45.760 |
[MUSIC PLAYING]
|
| 00:53:47.760 |
[MUSIC PLAYING]
|
| 00:53:49.760 |
[MUSIC PLAYING]
|
| 00:53:51.760 |
[MUSIC PLAYING]
|
| 00:53:53.760 |
[MUSIC PLAYING]
|
| 00:53:55.760 |
[MUSIC PLAYING]
|
| 00:53:57.760 |
[MUSIC PLAYING]
|
| 00:53:59.760 |
[MUSIC PLAYING]
|
| 00:54:01.760 |
[MUSIC PLAYING]
|
| 00:54:03.760 |
[MUSIC PLAYING]
|
| 00:54:05.760 |
[MUSIC PLAYING]
|
| 00:54:07.760 |
[MUSIC PLAYING]
|
| 00:54:09.760 |
[MUSIC PLAYING]
|
| 00:54:11.760 |
[MUSIC PLAYING]
|
| 00:54:13.760 |
[MUSIC PLAYING]
|
| 00:54:15.760 |
[MUSIC PLAYING]
|
| 00:54:17.760 |
[MUSIC PLAYING]
|
| 00:54:19.760 |
[MUSIC PLAYING]
|
| 00:54:21.760 |
[MUSIC PLAYING]
|
| 00:54:23.760 |
[MUSIC PLAYING]
|
| 00:54:25.760 |
[MUSIC PLAYING]
|
| 00:54:27.760 |
[MUSIC PLAYING]
|
| 00:54:29.760 |
[MUSIC PLAYING]
|
| 00:54:31.760 |
[MUSIC PLAYING]
|
| 00:54:33.760 |
♪♪
|
| 00:54:40.800 |
♪♪
|
| 00:54:44.760 |
♪♪
|
| 00:54:49.240 |
♪♪
|
| 00:54:55.740 |
♪♪
|
| 00:55:01.780 |
♪♪
|
| 00:55:05.780 |
♪♪
|
| 00:55:11.780 |
♪♪
|
| 00:55:21.800 |
♪♪
|
| 00:55:26.800 |
♪♪
|
| 00:55:30.800 |
[Music]
|
| 00:55:41.140 |
[Music]
|
| 00:55:59.280 |
[Music]
|
| 00:56:09.280 |
[Music]
|
| 00:56:19.280 |
[Music]
|
| 00:56:29.280 |
[Music]
|
| 00:56:39.280 |
[Music]
|
| 00:56:49.280 |
[Music]
|
| 00:56:59.280 |
[Music]
|
| 00:57:09.280 |
[Music]
|
| 00:57:19.280 |
[Music]
|
| 00:57:29.280 |
[Music]
|
| 00:57:39.280 |
[Music]
|
| 00:57:49.280 |
[Music]
|
| 00:57:59.280 |
[Music]
|
| 00:58:09.280 |
[Music]
|
| 00:58:19.280 |
[Music]
|
| 00:58:29.280 |
[Music]
|
| 00:58:39.280 |
[Music]
|
| 00:58:49.280 |
[Music]
|
| 00:58:51.280 |
[BLANK_AUDIO]
|