The First Open Source Very Large Language Model
In AI, large language models are all the hype. OpenAI is responsible for creating the most famous of these model, called GPT-3. This model has made a lasting impression with its ability to generate articles, finish sentences, and even help programmers write computer code!
Although impressive, GPT-3 has faced much criticism.
The most important criticism of the model has been that its source code was never released. OpenAI has defended this approach with claims about the “dangers” of their models, and the importance of monitoring responsible use.
But this week another company, Meta (you might remember them as Facebook) has decided to give the people what they want. They have just released the first completely open-source large language model!
Better For Science And The Environment
What makes GPT-3 so powerful is its size. It has a whopping 175 billion parameters. A very large language model indeed! It’s so big, that it bases its understanding on how humans write, exchange, and understand language on (almost) the entire internet!
The newly released model by Meta is just as powerful as GPT-3. But it differs on two critical aspects.
First, of course, it’s completely open-source. This is important because this allows scientists and engineers to better understand the way the model works. Such understanding is essential for enabling future progress.
Secondly, Meta’s model turns out to be much better for the environment than GPT-3. It has only a fraction of the carbon footprint of the famous Open-AI model. This is important because the training of AI models consumes a lot of energy, and we need to find ways to reduce its impact on the environment.
But what about “responsible use”, you may wonder? Won’t we see an explosion of nefarious applications now that this code is out in the open?
These are valid questions, to which no one has a definitive answer.
But one can only applaud Meta’s effort in communicating the limitations, biases, and risks of their model.
In order for science and AI to move forward, we need to share our knowledge with others. Open-source large language models are an essential part of that approach. Let’s hope that many other companies will follow Meta’s example!