OpenAI is the world's most famous artificial intelligence research institution. It has released many famous artificial intelligence technologies and achievements, such as the large language model GPT series, text-generated picture pre-trained model DALL·E series, speech recognition model Whisper series, etc. Because these models have quite amazing performance in their respective fields, they have attracted widespread attention from all over the world. This blog has a long content because it covers important technical achievements released by OpenAI in the past 7 years! ( original details:
This is the Pioneer Building in San Francisco, and also the location of OpenAI's office
The founding history of OpenAI
OpenAI is made by Musker (Elon Musk) and others founded a non-profit artificial intelligence research company in San Francisco in 2015. The startup capital is 1 billion US dollars, which is considered a standard second-generation rich. It is precisely because of the power of "money" that OpenAI's goal is to promote digital intelligence to benefit mankind without being restricted by economic returns. The goal of OpenAI is to cooperate with other institutions to conduct related AI research, and open up research results to promote the development of AI technology. OpenAI is also considered to be DeepMind has a strong competitor. However, since the GPT-2 model, OpenAI believes that the model is too effective and may be used to do things that are not good, so it has begun to limit the "openness" of research results, which has been criticized by many people.
On March 1, 2019, OpenAI announced that it will be overprofit from "non-profit" to "capped" ('capped') For profit) "for-profit, the profit of is 100 times that of any investment (the OpenAI LP company was founded). It was also in this year that Microsoft invested $1 billion in the company and obtained the commercialization of OpenAI technology. Since then, some OpenAI technologies have appeared in Microsoft's products and businesses. However, the cooperation between OpenAI and Microsoft actually started in 2016. In 2016, Microsoft's cloud service Azure provided a platform for large-scale experiments for OpenAI. At that time, Azure had provided them with K80 with InfiniBand interconnection. At that time, it had provided them with K80 with InfiniBand interconnection. GPU computing resources to optimize deep learning training. On September 22, 2020, OpenAI began to authorize Microsoft to use their GPT-3 model, and is also the world's first company to enjoy GPT-3 capabilities.
On June 11, 2020, OpenAI released OpenAI API, this is also OpenAI's first commercial product. The official explained that they believe that developing commercial products is an effective means to ensure that OpenAI has enough funds to continue to invest in AI research. Since then, OpenAI has officially begun commercial operation. The official also explained that using APIs to provide models instead of open source models will also lower the threshold for model use. After all, for small and medium-sized enterprises, the cost of deploying a powerful AI model may be higher.
A brief history of technology released by OpenAI
OpenAI has released many artificial intelligence-related technologies, from tools to algorithms to papers to models. Here we will briefly introduce the relevant research results they released. Since OpenAI was established for a short time, we will explain the main technical achievements released by OpenAI based on the year.
2016
On April 27, 2016, OpenAI released their first project - OpenAI Gym Beta, a tool used to develop and compare different reinforcement learning algorithms. This tool was originally used by OpenAI researchers to accelerate their reinforcement learning research, and this tool is also the first open achievement of OpenAI.
2017
On May 24, 2017, OpenAI opened the source of a tool to reproduce reinforcement learning algorithms - OpenAI Baselines. Reinforcement learning is very complex and has many influencing factors, making it difficult to reproduce many experiments. Therefore, OpenAI opens the source tool with the goal of providing some best practices for the implementation of correct reinforcement learning algorithms to help everyone improve the research efficiency of reinforcement learning.The first baselined model in OpenAI Baselines is DQN (Deep Q-Network)
2018
June 11, 2018 OpenAI announced an algorithm that has achieved good results in many language processing tasks, namely the famous GPT, which is also the first version of this algorithm. GPT is the first to combine transformers with unsupervised pre-training techniques, which achieve better results than current known algorithms. This algorithm is a pioneer in the exploration of OpenAI's large language model, and has also led to the emergence of a more powerful GPT series.
Also in June 2018, OpenAI announced that their OpenAI Five has begun to defeat the amateur human team in the Dota2 game, and said it will fight against the world's top players in the next 2 months. OpenAI Five uses 256 P100 GPUs and 128,000 CPU cores to train models by playing 180-year-old games every day. OpenAI Five details continue to be announced in the following months. In the professional competition in August, OpenAI Five lost 2 games against top players, but within the first 25-30 minutes of the game, OpenAI Five's model performed very well. OpenAI Five continued to grow and announced on April 15, 2019 that it had defeated the then Dota2 world champion.
2019
2019 February 14, 2019, OpenAI officially announced the GPT-2 model in its blog "Better Language Models and Their Implications". It is also in this blog that the official said that because the model is too effective, they are worried that the model will be used maliciously, and they will not publish pre-training results without thinking about how to limit malicious applications. The GPT-2 model has 1.5 billion parameters and is trained based on 8 million web page data. GPT-2 is the scaled result of GPT, and the data with more than 10 times of the data are trained with more than 10 times of the parameters. When the release of GPT-2 in February, OpenAI only disclosed their pre-training results for 124 million versions. In May, it released the pre-training results for 355 million parameter versions, and in August six months later, it released the pre-training results for 774 million parameter versions of GPT-2. On November 5, 2019, the full version of GPT-2 pre-training results with 1.5 billion parameters were released.
On March 4 of the same year, OpenAI released a large-scale multi-agent gaming environment for reinforcement learning agents: Neural MMO. The platform supports the presence of a large number of variable agents in a persistent, open task. The addition of many agents and species leads to better exploration, divergence niche formation, and greater overall capabilities.
htmlOn April 25, OpenAI continued to announce their latest research results: MuseNet, a deep neural network that can generate 4-minute music works with 10 different instruments, and can combine the style from the country to the Mozart to the Beatles . This is the expansion of generative models from the field of natural language processing to other fields.
2020
2020 On April 14, 2020, OpenAI released Microscope, a visualization tool used to analyze the internal feature formation process of neural network , and it is also an effort made by OpenAI to understand the neural network model.
On May 28, 2020, OpenAI researchers directly submitted the paper "Language Models are Few-Shot Learners", officially releasing the research results related to GPT-3, which was also the world's largest pre-trained model at that time, with 175 billion parameters! GPT-3 demonstrates its powerful capabilities in the paper, but as in the previous version, the official did not publish the pre-training result file. However, in September of the same year, the commercialization of GPT-3 was licensed to Microsoft.
On June 17 of the same year, OpenAI released the Image GPT model, introducing the successful GPT into the field of computer vision . The researchers believe that transformers are domain-independent, and they are all modeled from sequences, so the computer vision field is still usable. Image GPT also achieved good results at that time!
2021
2021 January 5, OpenAI released CLIP, which can effectively learn visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark, simply providing the name of the visual category to be identified, similar to the "zero-shot" capabilities of GPT-2 and GPT-3.This model is a very representative work in the multimodal field this year.
On the same day, OpenAI released the DALL·E model, which is also a very influential model. DALL·E is a 12 billion parameter GPT-3 version, which is trained to use a dataset of text-image pairs to generate images from text descriptions. DALL·E can create anthropomorphic versions of animals and objects, combine irrelevant concepts, render text, and transform existing images in a reasonable way. DALL·E's release once again amazed the world.
On August 10, 2021, OpenAI released Codex. OpenAI Codex is a descendant of GPT-3; its training data contains both natural language and billions of lines of public source code, including code in the Github public repository. OpenAI Codex is the model behind Github Coplilot. Of course, Codex has not been announced, but rather an API for OpenAI.
2022
On January 27, 2022, OpenAI released InstructGPT. This is a better language model that follows user intentions than GPT-3, while also making them more realistic and less toxic, using technology developed through alignment research. These InstructGPT models are trained with human participation, an AI dialogue system and an OpenAI charging API.
On March 15, 2022, the new version of OpenAI's GPT-3 and Codex were released, adding the ability to edit and insert new content. In other words, in addition to the previous generation capabilities, new edits and modifications have been added.
On April 6 of the same year, DALL·E2 was released. Its effect is more realistic than the first version, with richer details and higher resolution. Since the DALL·E series can generate any image content, although the official has made a lot of efforts to prevent malicious results, it is still not released because of concern. Perhaps because of pressure from open source competing products such as Stable Diffusion, on July 20, 2022, OpenAI's API added DALL·E released a year ago (note that it is not the V2 version).
htmlOn June 23, OpenAI trained a neural network to play Minecraft on a large number of label-free video data sets of humans playing Minecraft through video pre-training (VPT), and only a small amount of label data was used. With fine-tuning, the model can learn to make diamond tools, a task that usually takes more than 20 minutes (24,000 moves) for a skilled human. It uses human native keys and mouse movement interfaces, making it quite versatile and represents a step towards general-purpose computer use agents.
htmlOn September 21, OpenAI released Whisper, a pre-trained model for speech recognition, which is close to human level and supports multiple languages. Most importantly, compared with other models with long and unopen source results, this is a completely open source model, but its parameters are only 1.55 billion.
On November 30, OpenAI released the ChatGPT system, an AI dialogue system, and its powerful capabilities have allowed everyone to see its powerful capabilities again. ChatGPT's nearly perfect performance on many issues has made it reach 1 million users in just 5 days. It can help us write code, blog, explain techniques, and can have multiple rounds of conversations, write short dramas, and so on.
Summary
OpenAI is a star company in the field of artificial intelligence. It has attracted a lot of attention since Musk and others founded it. At first, its research seemed to be mainly working towards reinforcement learning. However, with the rise of pre-trained models, their innovations in many fields have also allowed everyone to see the powerful strength of OpenAI. Many of the models and systems released by OpenAI have surprising effects. Although as its commercialization accelerates, free and open source technologies seem to have become rare. However, the technology they released has attracted numerous followers and competitors. Competitors including Meta AI and StabilityAI have released open source versions of brother models. Promotes the development of the AI field.
Original details: OpenAI introduction and its results introduction | Learning data (Datalearner)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-