Meta's open source GPT-3 competitor

The First Open Source Very Large Language Model

In AI, large language models are all the hype. OpenAI is responsible for creating the most famous of these model, called GPT-3. This model has made a lasting impression with its ability to generate articles, finish sentences, and even help programmers write computer code! 

Although impressive, GPT-3 has faced much criticism. 

The most important criticism of the model has been that its source code was never released. OpenAI has defended this approach with claims about the “dangers” of their models, and the importance of monitoring responsible use. 

But this week another company, Meta (you might remember them as Facebook) has decided to give the people what they want. They have just released the first completely open-source large language model! 

Better For Science And The Environment

What makes GPT-3 so powerful is its size. It has a whopping 175 billion parameters. A very large language model indeed! It’s so big, that it bases its understanding on how humans write, exchange, and understand language on (almost) the entire internet! 

Meta released the first fully open-source large language model

The newly released model by Meta is just as powerful as GPT-3. But it differs on two critical aspects.  

First, of course, it’s completely open-source. This is important because this allows scientists and engineers to better understand the way the model works. Such understanding is essential for enabling future progress. 

Secondly, Meta’s model turns out to be much better for the environment than GPT-3. It has only a fraction of the carbon footprint of the famous Open-AI model. This is important because the training of AI models consumes a lot of energy, and we need to find ways to reduce its impact on the environment.  

Responsible Use

But what about “responsible use”, you may wonder? Won’t we see an explosion of nefarious applications now that this code is out in the open? 

These are valid questions, to which no one has a definitive answer.  

But one can only applaud Meta’s effort in communicating the limitations, biases, and risks of their model.  

In order for science and AI to move forward, we need to share our knowledge with others. Open-source large language models are an essential part of that approach. Let’s hope that many other companies will follow Meta’s example!

Learning more

To learn more about Meta's new model, and why they decided to make it open-source, you could read the recent Blogpost from the Meta AI team.




Tous Les Articles

Vous y êtes presque...

Nous venons de vous envoyer un e-mail. Veuillez cliquer sur le lien contenu dans l'e-mail pour confirmer votre abonnement !