I've been quizzing ChatGPT itself on how it works, how it stores
information, and why it's wrong so much.
I think the actually amazing thing is that any correct information comes
out of it at all.
ChatGPT and systems like it don't know anything. At all. They are super
complicated "word bagatelles" where they just start at a word, and then
figure out what the next word is "most likely" based on that word, and then
the word after that and the word after that, etc.
Now, if you just analyzed English as a whole, taking a word such as "The"
and working out what the next plausible word is would be a total shot in
the dark. You'd think it could be anything, but actually the next word
probably is an adjective or a noun, at the very least, and probably isn't a
word like "and". But still, there's a big-ass list of words that could
follow "the".
So, ChatGPT narrows it down by rating every word within a bazillion
semantic spaces. So, if you're talking about deer, then if you looked at
all the literature about deer, the next word should probably be somewhere
in there. And if you're talking about roads, then there's a different
probability distribution for road-related words. And if you're talking
about night, then there's a different night-related distribution, and so
on. And then there are a variety of built in distributions, such as
grammar, and syntax, and style, and etc.
You can think of all these topic distributions as Venn Diagram circles,
where this circle contains all the deer-related words and how probable they
are in the deer literature, and this circle is the road circle with all the
word probabilities associated with road literature, and so on. The
intersection of all these Venn circles give it the proper probabilities for
determining the next word. And it just picks the top "most likely" word
that comes out.
It doesn't look back at what it's already said, and it doesn't look forward
to where it's going. It only ever cares about "the next word", constrained
by all the semantic spaces it worked out that are in play. That's it.
That's why it gets stuff wrong. No knowledge or experience really ever
enters it, and it isn't drawing upon anything except probability
distributions. It just randomly generates the next word in a
super-complicated way that results in good English writing, but not
necessarily containing factual material. Every sentence was just generated
by probability matrices.
In some ways, that makes systems like these similar in concept to tarot
cards.
You can ask tarot cards for facts, or you can ask tarot cards for ideas.
No matter what you ask, you will receive an answer, for sure. It will be
completely random, of course, but: there's the cards, holding your answer.
Now, if you are asking for facts (*"Does Julie like Mark better than me?*"),
good luck with that. Maybe you'll get them, maybe you won't, that's pretty
much a luck of the draw, and I wouldn't rely on any of it. However, if
you're just asking for ideas ("*What should my next short story be about?*")
the process might be very helpful.
ChatGPT is very much like that right now. You want facts, you might need
to do a little validation; "facts" aren't what this is good at right now.
You need ideas, however, you will find an endless cornucopia of them.
I expect multi-pass validations will eventually be added. Right now
ChatGPT, I think, is only part of what full AI will be like. We're seeing
one layer in what should be a multi-layer system, but even what that one
layer is doing is actually astonishing. Like I said, I'm surprised that *any
*correct info comes through at all, let alone how much actually does.
Scott
On Sun, Feb 5, 2023 at 6:53 AM Darius Dunlap <darius@xxxxxxxxxxx> wrote:
I agree with Scott, mostly.
The technology of ChatGPT can be what finally supplants the “search page
with Ads” model of a search engine. But I think it may only make Google
stronger.
Google has its own answer generators, a simple version of which you can
see on many web searches. It also has its own ChatGPT-like generative text
capability, which reportedly has not been released because Google felt it
wasn’t good enough. It wasn’t good enough in pretty much the same way
ChatGPT is trash – it gives very confident answers that are wrong. But
Google is pressing hard on their technology, largely driven by those
factors Scott points to, especially the increasing siloing of content out
of the reach of their crawlers.
If you think about what Cory Doctorow says about “Enshittification”, and
project onto the future of these technologies, it doesn’t look good. Google
is already obscuring the sources of information it presents and is also
already making it less and less clear what is and is not an Ad (or “paid
placement”). So how bad does that get when all we get is a very confident
sounding “Answer” to our query?
It comes down to a simple question. Whose needs are being served? The
human asking the question? The Advertisers paying the bills? Or the giant
corporation turning the screws on them both and consolidating its control?
Darius Dunlap
darius@xxxxxxxxxxx
On Feb 2, 2023, at 6:44 AM, Mr. Ransom <mr.ransom@xxxxxxxxx> wrote:
ChatGPT could certainly take Google down a stack of pegs. Google is
enormous, so I doubt it's all that easily *destroyed*, but... "search"
isn't going to be what it used to be.
Google is was all about finding pages and sites. ChatGPT is about
finding information. They used to be mostly the same thing, but they
aren't any longer. Google will need to adapt to that, but, as so many
large companies have found in the past, trying to pivot away from a core
business is harder than it seems.
Google indexes the world. ChatGPT synthesizes it. The generative text
model appears to be the more compelling model for a whole variety of things
that indexing just isn't sufficient for, because the new model can create
things that aren't there to be indexed. Google is reliant on what already
exists out there (which is increasingly being siloed inside systems like
Facebook, reddit, instagram, and so on. ChatGPT is not reliant on any of
that, and can create new material on its own, which is certainly not
something we generally expect from search engines.
Scott
On Wed, Feb 1, 2023 at 2:34 AM Steve Crane <steve.crane@xxxxxxxxx> wrote:
https://bgr.com/tech/gmail-creator-says-chatgpt-might-destroy-google-within-2-years/
Maybe time to start moving our mail lives to ProtonMail or some other
paid service. If Google dies they won't be giving us free email any
longer. 🤔
Steve Crane
about.me/stevecrane
