LaMDA is a language model designed by Google to power its chatbot, Bard. This is the technical foundation to compete with OpenAI’s ChatGPT.
What is LaMDA?
TheMDA is an English acronym meaning Language Model for Dialogue Applications, or language model for dialogue applications, in French. It is a system developed by Google within the framework of automatic language processing, in order to simulate a natural conversation, as if one were exchanging with another person.
The idea of a language model is to determine the arrangement of words in a sentence, the way they fit together, the sequences of terms, and so on. It is therefore based on a statistical model, and therefore mathematics, to model this distribution. This is what then allows it to determine what should be the word after other terms already displayed.
There are various language models: BERT (“Bidirectional Encoder Representations from Transformers”) is another from Google, used in its search engine. GPT (“Generative Pre-trained Transformer”) too, except that it was designed by OpenAI, a company now very close to Microsoft. GPT-3.5 powers the famous ChatGPT.
According to GoogleLaMDA is a “ next generation conversational artificial intelligence model “. The reference to AI is somewhat excessive. What should be remembered is that it is in short the technical base useful to a chatbot, that is to say a conversational agent, to exchange with a person, while seeming to be smart.
In this case, LaMDA mobilizes processes typical of work in artificial intelligence, in particular an artificial neural network (the one Google uses for LaMDA is called Transform and which was finalized in 2017). It has also been pre-trained on huge amounts of data, with a particular focus on its ability to dialogue.
When was LaMDA introduced?
Google has been working around LaMDA for several years. Proof of this is with its artificial neural network, Transformer, designed in 2017, then on other works of 2020 supposed to show, according to the American company, ” that dialogue-trained, Transformer-based language models could learn to speak about virtually anything. »
However, the public discovered LaMDA in May 2021, during the annual Google I/O conference. The sequence, at that time, had been amazing to follow, because the American company had the idea of staging a discussion with a paper airplane on the one hand and with the dwarf planet Pluto on the other. . All under the eyes of Sundar Pichai, the boss of Google.
At the time, Google had to demonstrate its advances in automatic language processing, in the ability to hold the thread of an exchange and in the quality of the answers provided, as if we were conversing with a another person, in a fluid and natural way. Encapsulated in a chatbot, LaMDA could do wonders, according to the Mountain View firm.
But since this presentation, LaMDA has remained very discreet. Google had said that he saw himself integrating it into Google Assistant or Google Maps – for example to ask to find roads “with a nice view”. But for more than one and a half, no integration accessible to the general public has been observed. But that was before. Now there is Bard.
What is LaMDA used in?
Today, the first effective use of LaMDA is announced in Bard. It is an “experimental conversational AI service”, the existence of which was formalized in February 2023 – the press had heard of the project a few days earlier. The tool is not accessible to the general public, but this project makes it possible to give meaning and an application to LaMDA.
In fact, Bard works with a scaled-down version — stripped down, according to Google — of LaMDA. A kind of junior LaMDA, to put it another way. ” This model, which is smaller and requires less computing power, allows us to deploy to a greater number of users and process a larger volume of feedback. », justifies the American company.
In the short and medium term, the tool will gradually be made available to the public. For Google, it is a question of moving forward cautiously to avoid releasing a tool that could, in its answers, pose a problem: by generating racist texts, by proposing false answers, by producing texts that are meaningless, and so right now. For now, Google is sticking to tests.
Special feature of Bard compared to ChatGPT as we know it: Bard is able to work on fresh data, where ChatGPT is based on a corpus of information stopping at 2021 (this could however change very soon with GPT-4 or with the integration in Bing). This is obviously very practical in case of connection with the Google search engine.
It is also because Bard will have access to recent information that Google intends to provide an intermediate test phase with people supposed to note any problems upstream so that they can be corrected in LaMDA. That said, an old corpus is not a guarantee of security either: ChatGPT may not have access to fresh data, but it has already made a mistake.
What is LaMDA based on?
LaMDA mobilizes Transformer, an artificial neural network, and artificial intelligence algorithms. ” This architecture produces a model that can be trained to read many words (a sentence or a paragraph, for example), pay attention to how those words relate to each other, and then predict the words it thinks be next “, according to Google.
The focus was on the ability to dialogue. Early 2022, VentureBeat reported that LaMDA absorbed more than 1.56 trillion words from nearly three billion documents, more than one billion conversations, and more than 13 billion dialog utterances (i.e. spoken dialog transcripts). According to Google, this allows it to capture millions of topics.
LaMDA’s training took nearly two months to culminate in its most mature version (at the time). Google has notably mobilized its chips cut to accelerate the training of artificial intelligence (Tensor Processing Unit) — It is with these components that DeepMind, a subsidiary of Google specialized in AI, has trained AlphaStar, a super-AI for gaming StarCraft 2.
Google’s goal was also to make LaMDA understand the existence of nuances in open conversations, other exchanges, including “sensitivity”; that is, if the response given to an exchange makes sense. Today, LaMDA is based on 137 billion parameters. LaMDA has been in version 2 since May 2022 (before it was LaMDA v1).
A real AI? The LaMDA controversy?
LaMDA was at the heart of a curious case in 2022, when a Google engineer spoke up and claimed that the chatbot had “evolved” to the point of becoming an artificial intelligence, in the truest sense of the word, with a awareness. Some considered that the engineer was struck by anthropomorphism and that he “projected” fantasies onto this chatbot.
The case was somewhat embarrassing for Google, which ended up firing the engineer. The person concerned believed that this AI was as conscious as his 14-year-old child. He had also hired a lawyer to represent LaMDA—or rather, he had said that it was LaMDA who had hired a lawyer, to defend his rights, and that he had only been the messenger.
The employee has since been widely contradicted by his peers and other artificial intelligence specialists. “These types of systems are not powerful enough to achieve true intelligence,” noted, for example, Yann LeCun, head of computer science research at Meta (Facebook) and key figure in the rise of neural networks, recalled the New York Times.
If you liked this article, you will like the following ones: do not miss them by subscribing to Numerama on Google News.
