GPT-3 is aNatural Language technology by OpenAI, and is considered one of the most powerful and sophisticated tools available. There is a huge level of hype about GPT-3 on social media, people are writing about how it can change revolutionize the world. They are sharing demos of things they have done with GPT-3.In this blog post we will investigate~ What it is? ~ How it works? ~ Why it is special? and What the use-case are and current thinking about GPT-3.
Before starting let’s understand some basic terminologies which will be used throughout the blog post.
Language Model: Is a model that makes predictions regarding the structure of a sentence or phrase, and generate the probability that they exist in real world situations, for example “Rain falls from the sky” has a higher probability of existence than “Rain falls from the earth”. It uses statistics and probability to test if a sequence of words is likely to exist or not.
Neural Network: It is a set of algorithms that establish the relationship between the data by mimicking the function of a human brain.
Transformer: One type of neural network architecture is transformer. It is used by some of the biggest AI based companies, such as DeepMind and OpenAI, for creating a language model.
The Transformer architecture was proposed in paper “Attention is All You Need” by google this architecture transform one sequence into another for example if we are using transformer for the purpose of machine translation from French to English it will take sequence of French and transforms it into English this architecture using the attention mechanism as it is clear from title of the paper. This architecture has encoder taking input decoder performing transformation and generating the output. The power that lies with this model is use of parallelization and its boosting speed with which model are trained.
What is GPT-3?
GPT-3 is the third generation of the Generative Pretrained Transformer created by OpenAI. It is used to perform tasks such as text completion, machine translation, and predicatively writing. Developer communities are sharing how they have used GPT-3 in their demos to illustrate the possible use cases, but demand has seen the creation of a waiting list for its use
How GPT-3 Works?
GPT-3 is a language model which is powered by a Neural Network architecture.The Neural Network architecture used in GPT-3 is based on transformer. This became popular some years back due it use in the predecessor GPT-3 and other powerful language models. GPT-3 Neural Networks ingest huge amounts of data in the attempt to figure out how humans communicate. GPT-3 has currently consumed approximately 45 TB of data from many different sources to learn how word sequences statistically depend on one another.The core predictive text generator assigns probabilities to text sequences and the probability of real word use or existence.
Why GPT-3 is so special?
One of the biggest features of GPT-3 is its size.It has 175 billion learning parameters which makes it the biggest language model available, beating Microsoft Corp’s Turing-NLG algorithm which had 17 billion learning parameters. This huge difference in scale that makes it very powerful as compared to all its predecessors. This edge makes GPT-3 very good in performing a particular task. After the release of GPT-3,access to private beta was given to the developer community to see what, and the range of applications could be created. It has been demonstrated that GPT-3 has the ability to write coherent text including, for example, programs or poetry. What makes GPT-3 special is its ability to generalize and perform tasks in different areas of interest.
What people are saying about GPT-3
The current hype and admiration of GPT-3 is setting it aside from other forms of processing.
For example, Arram Sabeti in his blog describe show he was able to write a screenplay, a press release, a song and a blog post with GPT-3, and how amazed he was by results. Some developers who have been provided with access to the private set have used GPT-3 for writing code,as well as creating interfaces, SQL queries and writing email in their style, as shown below in some popular tweets.
In the blog I asked GPT-3 to make presentation for me, the author demonstrated how he created a presentation with GPT-3.Despite of all the applications and praise,there are some fundamental problems with GPT-3 as mentioned by Facebook’s head of AI.In a broad spectrum of tweets using the hashtag “#gpt3”,pro’s and con’s can easily be identified. GPT-3 is surprising and creative, but it’s also unsafe due to harmful biases.
This saw the CEO of OpenAI, Sam Altman taking to twitter to downplay the hype by tweeting “The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!), but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.”
OpenAI has created a powerful language model using machine learning transformer architecture namely GPT-3.
There is huge excitement among pear members of the developer community about GPT-3. Its generalization seems to be very powerful around creating a model that is able to perform specific tasks where various developers have tried to perform in the past.
With the help of GPT-3 text generation we have seen the ability to write songs, poetry, press releases, and the auto creation of power point presentations,through auto-coder helpers that can help creators in their work.
It’s not perfect, there are some problems with GPT-3 despite of all this hype, but it appears this is just the beginning and there is a lot to come.
- @bemmu: I asked GPT-3 to make a presentation for me
- @arr:GPT-3: An AI that’s eerily good at writing almost anything
- @arXiv:Attention Is All You Need
- @technologyreview:OpenAI’s new language generator GPT-3 is shockingly good—and completely mindless
- @thenextweb:OpenAI’s new GPT-3 language explained in under 3 minutes
Happy Coding ❤