"We’re Gonna Need a Bigger Moat" by Steve Yegge
https://steve-yegge.medium.com/were-gonna-need-a-bigger-moat-478a8df6a0d2

“In related news, these “billion dollar”-class LLMs can now be cloned on
macbooks and copied directly onto Boston Dynamics robots via their
Raspberry Pi adapter, at which point…”

“Oh yeah, it was training costs. Remember when it was roughly $1B to
train an LLM like GPT-4?"

“According to the leaked Google memo, world-class competitive LLM
training costs just dropped from a billion dollars to… that’s right, you
guessed it…”

"A hundred dollars.”

“Before last week, there were, oh, maybe five LLMs in GPT’s class. In
the whole world. It was like back in the 1950s when there were like five
computers in the world, and IBM owned three of them.”

“Which sounds to me like a very safe and defensible moat. That is, until
you realize LLMs can f*****’ copy each other. So their so-called “data
advantage” was really only going to be safe for as long as all the big
players kept the AIs locked up.”

“I swear this is a damn Jerry Bruckheimer movie, unfolding before our eyes.”

Uh, it did not go well for the human race in the Robopocalypse book that
I just read and reviewed. 99.9% dead in three years.
https://www.amazon.com/Robopocalypse-Contemporaries-Daniel-H-Wilson/dp/0307740803/

Lynn

On Thursday, May 18, 2023 at 3:25:18 AM UTC+1, Lynn McGuire wrote:
> "We’re Gonna Need a Bigger Moat" by Steve Yegge
>
> https://steve-yegge.medium.com/were-gonna-need-a-bigger-moat-478a8df6a0d2
>
> “In related news, these “billion dollar”-class LLMs can now be cloned on
> macbooks and copied directly onto Boston Dynamics robots via their
> Raspberry Pi adapter, at which point…”
>
> “Oh yeah, it was training costs. Remember when it was roughly $1B to
> train an LLM like GPT-4?"
>
> “According to the leaked Google memo, world-class competitive LLM
> training costs just dropped from a billion dollars to… that’s right, you
> guessed it…”
>
> "A hundred dollars.”
>
> “Before last week, there were, oh, maybe five LLMs in GPT’s class. In
> the whole world. It was like back in the 1950s when there were like five
> computers in the world, and IBM owned three of them.”
>
> “Which sounds to me like a very safe and defensible moat. That is, until
> you realize LLMs can f*****’ copy each other. So their so-called “data
> advantage” was really only going to be safe for as long as all the big
> players kept the AIs locked up.”
>
> “I swear this is a damn Jerry Bruckheimer movie, unfolding before our eyes.”
>
> Uh, it did not go well for the human race in the Robopocalypse book that
> I just read and reviewed. 99.9% dead in three years.
>
> https://www.amazon.com/Robopocalypse-Contemporaries-Daniel-H-Wilson/dp/0307740803/
>
> Lynn
I am in the skeptical minority. I think that a Large Language Model is just that - a means of predicting the probabilities of the different possible extensions of a stretch of text. If it models human intelligence, it models the intelligence of a human who is articulate and plausible, but doesn't actually know anything or have any skills. There is in fact a precedent for this - from https://en.wikipedia.org/wiki/Williams_syndrome "Despite their physical and cognitive deficits, people with Williams syndrome exhibit impressive social and verbal abilities. Williams patients can be highly verbal relative to their IQ. When children with Williams syndrome are asked to name an array of animals, they may well list a wild assortment of creatures such as a koala, saber-toothed cat, vulture, unicorn, sea lion, yak, ibex and Brontosaurus, a far greater verbal array than would be expected of children with IQs in the 60s."

Perhaps an LLM would be truly useful as part of a larger system which had some way of applying quality control to its outputs, such as turning the LLM speculations into a valid mathematical argument if possible. But I have seen no sign that this has happened (yet).

On Wednesday, May 17, 2023 at 10:25:18 PM UTC-4, Lynn McGuire wrote:
> "We’re Gonna Need a Bigger Moat" by Steve Yegge
>
> https://steve-yegge.medium.com/were-gonna-need-a-bigger-moat-478a8df6a0d2
>
> “In related news, these “billion dollar”-class LLMs can now be cloned on
> macbooks and copied directly onto Boston Dynamics robots via their
> Raspberry Pi adapter, at which point…”
>
> “Oh yeah, it was training costs. Remember when it was roughly $1B to
> train an LLM like GPT-4?"
>
> “According to the leaked Google memo, world-class competitive LLM
> training costs just dropped from a billion dollars to… that’s right, you
> guessed it…”
>
> "A hundred dollars.”
>
> “Before last week, there were, oh, maybe five LLMs in GPT’s class. In
> the whole world. It was like back in the 1950s when there were like five
> computers in the world, and IBM owned three of them.”
>
> “Which sounds to me like a very safe and defensible moat. That is, until
> you realize LLMs can f*****’ copy each other. So their so-called “data
> advantage” was really only going to be safe for as long as all the big
> players kept the AIs locked up.”
>
> “I swear this is a damn Jerry Bruckheimer movie, unfolding before our eyes.”
>
> Uh, it did not go well for the human race in the Robopocalypse book that
> I just read and reviewed. 99.9% dead in three years.
>
> https://www.amazon.com/Robopocalypse-Contemporaries-Daniel-H-Wilson/dp/0307740803/
>
> Lynn

I've got to say, this is starting to sound like the precursors of a 'Technological Singularity'
novel.

pt

Andrew McDowell <mcdowell_ag@sky.com> writes:
>On Thursday, May 18, 2023 at 3:25:18=E2=80=AFAM UTC+1, Lynn McGuire wrote:
>> "We=E2=80=99re Gonna Need a Bigger Moat" by Steve Yegge=20
>>=20
>> https://steve-yegge.medium.com/were-gonna-need-a-bigger-moat-478a8df6a0d2=
>=20
>>=20
>> =E2=80=9CIn related news, these =E2=80=9Cbillion dollar=E2=80=9D-class LL=
>Ms can now be cloned on=20
>> macbooks and copied directly onto Boston Dynamics robots via their=20
>> Raspberry Pi adapter, at which point=E2=80=A6=E2=80=9D=20
>>=20
>> =E2=80=9COh yeah, it was training costs. Remember when it was roughly $1B=
> to=20
>> train an LLM like GPT-4?"=20
>>=20
>> =E2=80=9CAccording to the leaked Google memo, world-class competitive LLM=
>=20
>> training costs just dropped from a billion dollars to=E2=80=A6 that=E2=80=
>=99s right, you=20
>> guessed it=E2=80=A6=E2=80=9D=20
>>=20
>> "A hundred dollars.=E2=80=9D=20
>>=20
>> =E2=80=9CBefore last week, there were, oh, maybe five LLMs in GPT=E2=80=
>=99s class. In=20
>> the whole world. It was like back in the 1950s when there were like five=
>=20
>> computers in the world, and IBM owned three of them.=E2=80=9D=20
>>=20
>> =E2=80=9CWhich sounds to me like a very safe and defensible moat. That is=
>, until=20
>> you realize LLMs can f*****=E2=80=99 copy each other. So their so-called =
>=E2=80=9Cdata=20
>> advantage=E2=80=9D was really only going to be safe for as long as all th=
>e big=20
>> players kept the AIs locked up.=E2=80=9D=20
>>=20
>> =E2=80=9CI swear this is a damn Jerry Bruckheimer movie, unfolding before=
> our eyes.=E2=80=9D=20
>>=20
>> Uh, it did not go well for the human race in the Robopocalypse book that=
>=20
>> I just read and reviewed. 99.9% dead in three years.=20
>>=20
>> https://www.amazon.com/Robopocalypse-Contemporaries-Daniel-H-Wilson/dp/03=
>07740803/=20
>>=20
>> Lynn
>I am in the skeptical minority. I think that a Large Language Model is just=
> that - a means of predicting the probabilities of the different possible e=
>xtensions of a stretch of text.

Indeed.

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

It is _NOT_ AI. Nor is it sentient.

Scott Lurndal <scott@slp53.sl.home> wrote:
> Andrew McDowell <mcdowell_ag@sky.com> writes:
>>On Thursday, May 18, 2023 at 3:25:18=E2=80=AFAM UTC+1, Lynn McGuire wrote:
>>> "We=E2=80=99re Gonna Need a Bigger Moat" by Steve Yegge=20
>>I am in the skeptical minority. I think that a Large Language Model is just=
>> that - a means of predicting the probabilities of the different possible e=
>>xtensions of a stretch of text.
>
> Indeed.
>
> https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
>
> It is _NOT_ AI. Nor is it sentient.

It's just Babel-17, We could all learn to do it ;)

Actually, one earlier model (Word2vec) of the chemistry literature
successfully predicted properties of new compounds tested after the
year cutoff of the papers the program was trained on. Just by
calculating "a dot product projection) of normalized word embeddings".

Minsky's 1968 definition of AI: "the science of making machines do the
things that would require intelligence if done by men."