Unlike the treatment scripted dialogue found in most books and movies, the language of everyday interactions tends to be messy and incomplete, full of false starts, interruptions, and people talking to each other.
From casual conversations between friends to quarrels between siblings to formal discussions in a boardroom, authentic conversation is chaotic. It seems miraculous that anyone can learn a Language at all, given the random nature of the linguistic experience.
For this reason, many language scientists — including Noam Chomsky, a founder of modern linguistics — believe that language learners need a kind of glue to master the unruly nature of everyday language. And that cement is grammar: a system of rules for generating grammatical sentences.
Children must have a grammar model hardwired into their brain to help them overcome the limits of their language experience – or so it seems.
This template, for example, may contain a “super-rule” that dictates how new elements are added to existing sentences. Children then only have to learn if their mother tongue is one, such as English, where the verb precedes the object (as in “I eat Sushi“), or one like Japanese, where the verb goes after the object (in Japanese, the same sentence is structured as “I eat sushi”).
But new insights into language learning are coming from an unlikely source: artificial intelligence. A new breed of great AI language models can write newspaper articles, poetryand computer code and answer questions honestly after being exposed to large amounts of language input. And even more amazing, they all do it without the help of grammar.
language without grammar
Even if their choice of words is sometimes strange, absurd or contains racist, sexist and other harmful prejudices, one thing is very clear: the overwhelming majority of the output from these AI language models is grammatically correct. And yet, there are no patterns or grammar rules hard-wired into them – they rely solely on linguistic experience, however messy.
GPT-3, arguably the best known of these models, is a gigantic deep learning neural network with 175 billion parameters. It was trained to predict the next word in a sentence given what happened before on hundreds of billions of words from the internet, books and Wikipedia. When it made a wrong prediction, its parameters were adjusted using machine learning algorithm.
Remarkably, GPT-3 can generate believable text reacting to prompts such as “A summary of the latest ‘Fast and Furious’ movie is…” or “Write a poem in the style of Emily Dickinson”. In addition, GPT-3 can respond to SAT-level analogies and reading comprehension questions and even solve simple arithmetic problems – all while learning to predict the next word.
Comparison of AI models and human brains
The similarity to human language does not end there, however. Research published in Natural neuroscience demonstrated that these artificial deep learning networks seem to use the same calculation principles as the human brain.
The research group, led by neuroscientist Uri Hassonfirst compared how GPT-2 — a “little brother” to GPT-3 — and humans could predict the next word in a story from the “This American Life” podcast: People and AI predicted the exact same word almost 50% of the time.
The researchers recorded the brain activity of the volunteers while listening to the story. The best explanation for the activation patterns they observed was that people’s brains – like GPT-2 – were not just using the previous word or two to make predictions, but were relying on the accumulated context so far. to 100 previous words.
Altogether, the authors conclude: “Our finding of spontaneous predictive neural signals when participants listen to natural speech suggests that active prediction may underlie lifelong language learning in humans.”
One possible concern is that these new AI language models are powered by lots of input: GPT-3 was trained on a linguistic experience equivalent to 20,000 human years. But a preliminary study which has yet to be peer-reviewed found that GPT-2 can still model human next-word predictions and brain activations, even when trained on just 100 million words. This is well below the amount of language input an average child could hearing during the first ten years of life.
We are not suggesting that GPT-3 or GPT-2 learn language exactly like children. Indeed, these AI models do not seem to understand much, if anything, of what they are saying, while understanding is fundamental to the use of human language. Yet what these models prove is that a learner – albeit a silicon – can learn language well enough from mere exposure to produce perfectly good grammatical sentences and do so in a way that resembles the processing of the human brain.
Rethinking language learning
For years, many linguists believed it was impossible to learn a language without a built-in grammar model. New AI models prove otherwise. They demonstrate that the ability to produce grammatical language can be learned from linguistic experience alone. Likewise, we suggest that children do not need innate grammar to learn the language.
“Children should be seen, not heard” goes the old adage, but the latest AI language models suggest nothing could be further from the truth. Instead, children should be engaged in back and forth conversation as much as possible to help them develop their language skills. Linguistic experience – not grammar – is essential to becoming a proficient language user.