recommended introduction: The combination of games and artificial intelligence will allow people to further update their understanding of games. It also means more possibilities for radiating reality. A small step for AI in the program may become a big step in reality.
No game competition final can be so quiet. The players didn't exchange a word, they just watched and recorded in silence. Two hours later, the organizer announced that the game was over and students from the Computer Science Department of Tsinghua University stood out from the 23 participating teams.
But to be precise, the contestants were not these students - they were not even doing anything in front of the screen. What actually operates the game characters is the AI trained millions of times by the students. In the 90 days before
, these young geeks from top domestic universities used the algorithms, computing power and other resources of the "Enlightenment" AI open research platform to train an AI that can operate the King of Glory, and competed for the final championship.
The precise name of the " King of Glory " AI testing proving ground
competition is "Enlightenment Multi-Agent Reinforcement Learning Competition", which is held for the second time. Both the organizer and the contestants value the process of the competition more than the results. Their purpose is to "use competition for research": compete and iterate to develop better and stronger AI agents.
This competition is based on the carrier of King of Glory. This game focuses on collaboration and is extremely complex. In a 5V5 game, the player's action state space is as high as 10 to the 20,000th power, which even exceeds the total number of atoms and in the entire universe (10 to the 80th power). It is a natural testing ground for training multi-agent algorithms.
multi-agent algorithm is the frontier of artificial intelligence. In the past, due to issues such as scarcity of research scenarios, difficulty in algorithm testing, and expensive computing power, the experimental fields for AI research in universities have been limited. In order to broaden its boundaries and promote the development of artificial intelligence research, this year, the Kaiwu Platform also cooperated with 19 universities to develop innovative courses, which are open to more AI researchers and professional students.
AI vs. AI
Chen Huayu, the captain of the championship team, has a completely different desire to win when it comes to training AI to play Honor of Kings and playing by himself. After class, he relaxed and played two games. He didn't care whether he lost or won. But when it comes to cultivating AI to achieve better results, he is one of the most ruthless players in " chicken baby ".
players compared AI to a child. At the beginning, it was born like a blank piece of paper and knew nothing. With continuous training by humans, it gradually evolved various abilities. In the preliminary competition, Chen Huayu and his teammates conducted more than 40,000 battles between their own AI and the baseline AI provided by the organizer, and updated hundreds of models. Their opponent in the finals, University of Electronic Science and Technology of China "Chicken Baby", was even more ruthless, having played more than 60,000 games.
students from the University of Electronic Science and Technology of China are writing AI code
Like human players, AI needs to learn through repeated training and battles. Humans train for technical and tactical proficiency, while AI needs to optimize its own strategies through behavioral feedback through massive trials. During the 190-day training of
html, Chen Huayu’s greatest satisfaction for her competitive spirit was the moment when the AI learned to “squat in the grass”. Even now it's not squatting completely enough, just staying a few seconds longer when passing by the grass. "Squatting in the grass" is a behavior that human players use to hide their traces in the game to lure enemies. It is very simple, but AI is not born with it. It can learn, and it seems to have a touch of "spirituality", which also shows that it has become smart enough - in tens of thousands of trainings, it has learned that squatting in the grass can give it advantages such as vision and initiative, thus increasing its winning rate. This small step forAI to learn from humans is regarded by them as a giant step for artificial intelligence. A simple squatting action in the grass may require a model. The macro strategic decision-making and micro-numerical calculation capabilities involved behind it often require a lot of energy to be verified and tested.
The fun of training AI lies in the unknown. "Machine reinforcement learning is a kind of training similar to a black box. As a researcher, it is difficult to know clearly where the algorithm's deficiencies caused problems." This makes the AI trained by students "both strong and weak." On the one hand, they make decisions quickly and accurately, and end a game faster than ordinary players; on the other hand, the game characters may still hit the wall stupidly.And when it trains for hundreds of hours to learn a certain ability, a certain ability will suddenly decline rapidly or even collapse.
Chen Huayu has never known how to solve this problem. It was not until after the game that he heard the sharing from the University of Electronic Science and Technology of China team and got a lot of inspiration. The other party shared a set of methods that can stabilize the learning process of the agent.
Chen Huayu admires this opponent very much. The final was very close, and they only narrowly won by a few points. The stalemate shows that the capabilities of the two AIs are already comparable, and winning is just a matter of probability. "We just have a strong foundation and can barely hold on." He recruited a group of powerful students, and their strengths and weaknesses complemented each other, making their algorithms easier for AI to learn from big data and using data more efficiently than other teams. high.
"The significance of this competition is not about the ranking." Xie Ning, the instructor of the University of Electronic Science and Technology of China team, believes that the most important value of the Enlightenment Competition is to let students know that AI can achieve such capabilities. Its underlying technology is reinforcement learning. With the same training resources, the better the algorithm model design, the more powerful the AI decision-making intelligence that can be trained. Students fully trained in reinforcement learning, neural network, algorithm and other aspects during the competition. research ability.
New courses in universities
When they learned that the "Enlightenment" project had cooperated with 19 universities to offer free and open technical research resources, many teachers responded immediately like Xie Ning. For them, an artificial intelligence research platform that effectively connects scenarios, computing power, and algorithms is a timely blessing.
algorithms, scenarios, and computing power are the core of AI research, among which algorithm research is the main focus of many universities. However, due to limited computing power, research around large-scale computing and business scenarios is often hampered. "'Enlightenment' is unique. No other company at home or abroad has similar open source and will share computing power and resources." Chen Huayu made a calculation. If he wants to train King AI in his own laboratory, He had to gather the computing power of the forty or fifty computers there.
Students use Enlightenment Platform to conduct AI research
In the past, students had to train multi-agent AI. If possible, they would download game resources from open source platforms and write their own programs to practice. However, the complexity of game scenarios that were willing to be open source was generally very low. When teachers assign homework, they can only assign the homework with the lowest computing power and the least difficulty. This is for teaching fairness. Students have different economic levels and have different computer hardware conditions.
The King AI elective course offered by Xie Ning for undergraduates will start in the next semester. During the course selection, 200 students flocked to it, and the enthusiasm was extraordinary. He could only assign a course design report question and select 36 students from 200 students.
Some contestants who have participated in the Enlightenment Competition have also consciously become the link for courses to enter universities. In the tutor's "Algorithms in Game AI" elective course, Peking University doctoral student Lu Yunlong is responsible for guiding the teaching assistant of the King's AI part. Assignments related to
King AI account for the largest proportion of points in this course, and these post-00 students also showed great interest. This popular game has grown up with this generation, and it is a fresh experience to be able to use the learned algorithms to train the AI for playing games. During the two months that
guided the students, Lu Yunlong taught them all the experience of building models and training AI learned in the competition. He was pleasantly surprised by the students' performance. In the submitted homework, he found that there was a level 4 confrontation between the student-trained AI and the Tencent baseline AI, and the probability of winning had reached 50:50. This means that if this student participated in that competition that year, he would have reached the level of ranking. The entry of
king AI into university campuses has also brought confidence to young scholars like Lu Yunlong. The upgrade of the course made him more determined that the reinforcement learning algorithms he studied were enough to be used in more complex game environments. In the past, due to lack of computing power, I could not apply the intelligent agent I built myself to complex game training, and the exact level was difficult to verify. But now, he is much less confused.
Xie Ning also noticed the changes. After his research group adopted King AI as a daily scientific research project, the interest of graduate students increased significantly.In the past, they had no access to computing power and no direct experience. However, in the current "window period" of open "enlightenment", even on weekends and finals, they will seize the opportunity to train and experiment to verify their technical theories.
"Students have a more mature understanding of games than we do." He sometimes thinks that this sense of closeness may be innate and engraved in the genes of a generation. A student told Xie Ning that his parents met in a game. Xie Ning encourages students to participate in enlightenment competitions. His students took the initiative to teach and teach, and the students who participated in the first batch shared their coding notes with their junior brothers and sisters. In future courses, he hopes to put more emphasis on practical combat - it may be a good idea to hold a campus competition of the King AI Competition, "using competition as a substitute for research", which can produce more practical results.
Turning games into scientific testing grounds
After taking charge of the King AI application development project, Lao Liu, the person in charge of King AI application development, has a lot more things to do as a university teacher on his schedule. He prepared lessons together with the teachers, discussed how to break down the knowledge points, and taught the engineering application knowledge in King AI so that undergraduates could understand it. In normal times, he also has to read many papers, update the knowledge system of artificial intelligence, and embody the emerging technologies of multi-intelligence learning in the "Enlightenment" platform, so that students can implement the overall artificial intelligence in their daily homework and examinations.
At first, he was surprised by the limited computing power of universities. The CPU of student computers only has 16 cores, and GPU only has 32 cores. Most of them are Windows systems, which cannot support the operation of the enlightenment platform at all. Therefore, the project team mobilized 25 programmers to design the experimental platform client so that students can learn more efficiently.
"Provide some help in the trend development of artificial intelligence, so that students can learn more efficiently, teachers can convey knowledge more efficiently, and also allow students to avoid detours when doing similar engineering applications in future work. "Every time the students call him "Teacher Liu," it makes Lao Liu feel that this job has created a little more value. The research and training of artificial intelligence models often require countless iterations and trial and error. The accumulation of small, fast steps can lead to qualitative change. Enlightenment competitions and courses, and step-by-step exploration, are all aimed at improving the AI talent training system. When more and more scientific and technological young people participate in AI innovation, innovators and leaders will continue to emerge in the development wave of artificial intelligence.
Xie Ning’s research team is currently studying how to use the enlightenment platform to simulate fire evacuation and other scenarios, trying to solve some new problems in the field of emergency systems and social governance. In his view, “This means that games empower other industries.” The combination of games and AI will have broad prospects in the future. If AI can simulate fierce confrontations in the complex game environment of Honor of Kings and learn to make decisions like humans, then the large-scale collaborative algorithms can also be migrated to changeable and complex real environments.
This is also the original intention of the Enlightenment Project. They hope to link up with universities to use the complex environment of Honor of Kings to turn the game into a scientific testing ground and promote the use of artificial intelligence to create various possibilities. Their cooperation with Southwest Jiaotong University ’s smart transportation is about to begin. In the future, in the virtual environment of Honor of Kings, "heroes" will be simulated and given traffic light-like roles. After a series of complex transformation procedures, their traffic lights have become "skills one, two, and three," which can be used to experiment with when to "release skills" to optimize traffic efficiency.
In the future, more similar realistic simulation scenes will appear in the Canyon of Kings, covering medical, industrial, agricultural, transportation and other industries. Lu Yunlong envisions allowing “heroes” to simulate autonomous driving in future research. Trial and error in the virtual environment, train stable agents without accidents, and apply the algorithms generated in the process to practice, "providing other fields with a low-cost place for trial and error."
"We are actually studying the operating rules of the world in the game." Chao Ge, technical director of Tencent AI Lab's "Enlightenment" platform, believes that there are still a lot of problems that still need to be overcome in the multiplayer competitive game environment. Only when more interested universities and scholars join in to discuss and study these problems can the entire industry-university-research community move forward.
In the AI academic community, the influence of the Enlightenment Platform continues to expand. After the first competition, students from overseas universities successively signed up to participate. Xie Ning hopes that more and more people will know about it. The combination of games and artificial intelligence will allow people to further update their understanding of games. It also means more possibilities for radiating reality. Xie Ning feels that the characters active in the game canyon are given a more realistic "meaning". A small step for AI in programming may become a giant step in reality.
Article source: Guangming.com "Let AI learn to play, a new battlefield for scientific research in universities"
Please contact: jhm9 9 9 9 9 8
Industry exchange/revelation/business cooperation/submission: add WeChat zhizuen9 5
