Blog

GPT-3: artificial intelligence technology for the future

With all the technological advances emerging today, it is very difficult to focus on just one: life on Mars, smart cities, 5G technology, artificial intelligence…. It seems that we want to get closer to the future that science fiction promised us faster.

Open AI for some time has been working on a language model that is capable of programming, designing and even talking about politics and the future; creating predictions just by giving it orders.

The way it works is predictive, since its operation serves through predicting which word follows, according to the study of previous data in the same context. Thus, it gives us as answers possibilities that strangely can be very close to reality.

All the answers are given according to all the information with which the model is trained (that is, machine learning), simulating the behavior of the brain (neural networks).

But the interesting thing about this is that technologies are being created (more specifically machines) that little by little could replace all the people in a work team.

The curious thing here is to question what level of technological singularity we can reach, creating things that from ignorance or knowledge become something really revolutionary for employment, industries and even humanity.

This type of technology will soon be the day-to-day of all industries: According to the MIT Technology Review, "GPT-3 is the most powerful language model ever created."

GPT-n

The concept of generative pre-training (GP) was first introduced in 2018 by San Francisco-based artificial intelligence laboratory OpenAI, which generated many improvements to alleviate the dependence of supervised learning on natural language processing (NLP), since this was the key to learning untagged plain text corpus. This included two phases: unsupervised pre-training on various raw text data followed by supervised fine-tuning for the specific task. The model architecture uses the Transformer, the primary deep learning model for NLP and hence the name GPT (Generative Pretrained Transformer). GPT-1 made use of around 7,000 unpublished unique books for its training and was able to outperform other specifically supervised NLP models in 9 of the 12 data sets they were compared to.

GPT-2 improved the initial model by adding more parameters and using a larger data set, 1.5 billion parameters compared to 117 million parameters in GPT-1 with text data from more than 8 million documents. GPT-2 excelled in areas such as reading comprehension, but was rudimentary in others such as text summarization.

GPT-3, introduced in May 2020, uses 175 billion parameters, which is the largest in the world and is 10 times larger compared to Microsoft's Turing NLG, the second largest model currently in existence. The quest consisted in eliminating the need for adjustments and using the less necessary demonstrations for the model to understand and perform the tasks. GPT-3 has been able to accomplish the ultimate in a variety of tasks such as translating, answering questions, reasoning on the fly, word decoding, and performing 3-digit arithmetic, to name a few. You are also able to write SQL queries and code using the natural language description of the task.

The capabilities of the GPT-3 are not limited to programming capabilities, far from it. The dataset used to train GPT-3 is, as mentioned above, of astronomical proportions. The entirety of English Wikipedia, which contains around 6 million articles, is said to account for just 0.6 percent of the training data. The coding manuals are just another set of data feeding into a gigantic network of neural networks, which would include everything from Reddit posts about cats to the most complex theories of quantum physics. In short, GPT-3 is the only NLP model to rule them all.

Can artificial intelligence create creativity?

In this unconscious race of humans against machines a big question to answer will be -- what technology cannot create.

Today many arts disciplines and subjects are already immersed in technological tools. Music, architecture, graphic design, marketing and advertising are increasingly influenced by technology, reaching a point where technology became a necessity to succeed in today's ecosystem.

Processes, song creation, social algorithms, translators and technical insights are many things that technology has facilitated in the creative field. Soon, development will focus on replicating "art creation" through machines and algorithms.

Today there is "computational creativity" that seeks to understand human creativity, designing programs that can understand and explain it.

Even though GPT3 is more on the programming side, some of the tasks it can already perform are:

- Program SQL code correctly

- Create articles by giving only the title

- Simulate conversations between various historical figures

- Predictions based on variables

Creativity is genius

GPT-3 is called smart and sarcastic, but will it be creative? It said that it can have unpredictable effects when solving problems; effects that undoubtedly assist and help human creativity, having the same impact as "Software as a service" immersed in all industries.

For now, creativity needs human talent. Of the ability to invent and create that only a few have and only a few more are truly capable of exploiting in all its splendor.

But are we witnessing the birth of AGI?

Artificial General Intelligence or AGI are programs or models that would possess all the complexities of the human mind. Think, learn and behave like a human being. Despite all its impressive abilities, GPT-3 isn't there yet, according to most observers. However, it could most likely be the first step towards such intelligence.

Conclusion

For all its baffling capabilities and silly flaws, GPT-3 is indeed a paradigm shift in the AI ​​world and will force us to rethink the way we go about our lives. In its current avatar, it may not steal all of our jobs, but that day may not be far off. At the same time, it also presents us with countless possibilities and how we shape it will determine our future.