GPT-3, which Hinton considered the "answer to the universe," is "ignorant of the world" in LeCun's view.

Xiao Xiao from Aofei Temple
Qubit Report | Public Account QbitAI

“Some people have unrealistic fantasies about what large language models like GPT-3 can do… GPT-3 has no idea how the world works.”

Recently, LeCun suddenly expressed his views on GPT-3 on Facebook.

LeCun said his friends have conducted research that defies expectations about how large language models can be used in healthcare .

He believes that GPT-3 is "entertaining in nature" and may play a certain role in creative work.

But beyond that, LeCun pointed out: "Expecting to build intelligent machines by making large language models is like trying to build a high-altitude aircraft and expecting it to land on the moon."

Netizens were furious about this.

How to understand LeCun's evaluation

LeCun believes that GPT-3 has some role to play, but it does not work in a way that “understands” text.

It collects a lot of text, but when asked to predict what text will come next, it actually does so based on "probabilities of words appearing" rather than "the way the world works."

Acquiring knowledge vs. understanding language

Some netizens' discussions focused on whether GPT-3 actually understands the world.

Some of them believe that "acquiring knowledge" and "understanding language" are two different abilities.

Netizen leftyted believes that whether GPT-3 has the ability to understand is actually a philosophical question. After all, knowledge and language are two different things.

He believes that if there were no various metaphors in language, GPT-3 would have the same ability as humans in mastering knowledge.

Understanding itself is difficult

However, netizen babesh believes that if language and knowledge are discussed separately, then GPT-3 will not be able to obtain all the knowledge of the world. This is because many things can be perceived by the senses, but without imagination, this sensory "knowledge" cannot exist at all.

Does GPT-3 really work?

Other netizens believe that LeCun’s statement that “GPT-3 is not very good at either dialogue or question-answering” is actually not correct.

In fact, many netizens who have used GPT-3 said that its capabilities are beyond imagination.

Netizen neuro_thing believes that LeCun’s ability to draw such a conclusion shows that his team must have done relevant research.

But from his experience using GPT-3, it is very convenient and works well. Not only that, the netizen also said that as far as he knows, GPT-3 has replaced the work of two machine learning engineers.

Of course, some people refute this view.

Netizen confuseshrink believes that even if you think GPT-3 is useful, as long as you don’t trust its output and even need to manually check the results, it is indeed not as useful as imagined.

“Don’t expect the upper limit of GPT-3”

Some netizens also believe that what LeCun really wants to express is not whether GPT-3 is effective, but how high its upper limit is.

Netizen moultano believes that LeCun’s original intention was not to say that “GPT-3 is nothing special”, but that he believes that continuing to stack large language models has no future.

△ “No matter how high the house is, it can’t reach space”

But in any case, the experts have already expressed their opinions on how far AGI (artificial general intelligence) like GPT-3 can go.

GPT-3: Hinton praises it, but Bengio and LeCun are not optimistic

Three Turing Award winners, Geoffrey Hinton, Yoshua Bengio, and Yann LeCun, have given different views on general artificial intelligence such as GPT-3.

Hinton said that from the outstanding performance of GPT-3, it can be inferred that the answer to life, the universe and everything is "no more than 4.398 trillion parameters."

However, compared to Hinton's views, LeCun is not optimistic about models such as AGI.

LeCun believes that neither self-supervised learning nor neurobiology is sufficient to achieve general artificial intelligence (AGI) , or even any intelligent machine with the ability to "understand".

"AGI doesn't exist—there's no such thing as general intelligence," LeCun said. "You could say a machine has mouse-level intelligence, cat-level intelligence, dog-level intelligence, human-level intelligence, but none of those are AGI."

In contrast, Bengio believes that if we don't talk about general artificial intelligence from the perspective of "understanding", it may also have its own role.

Bengio believes that machines will eventually be able to acquire and use all knowledge about the world, but not in a way that "understands" language, but in a way similar to "oral narration."

However, Bengio still expects machines to have the ability to "understand".

"I think culture is a big part of why humans are so smart and able to solve problems... If we really want AI to be useful, machines need to be able to not only translate but also understand language."

Yann LeCun Facebook page:
https://www.facebook.com/yann.lecun

Reference links:
https://twitter.com/geoffreyhinton
https://news.ycombinator.com/item?id=24907318&fbclid=IwAR2mN_6HoLeEumQzYmvDGJrE7SldBFhB98PHp0oaWAWugBx6xmYhBTCGKbA
https://venturebeat.com/2020/05/02/yann-lecun-and-yoshua-bengio-self-supervised-learning-is-the-key-to-human-level-intelligence/
https://venturebeat.com/2020/06/01/ai-machine-learning-openai-gpt-3-size-isnt-everything/