Like Jack, Shiyou Technology uses real-time virtual technology to resurrect nearly 400 "Avatars"

Dahe Financial Cube reporter Pei Rongrong

A cute and energetic two-dimensional beautiful girl, singing and dancing on the screen, makes people unable to move their eyes. But when the camera turned, the painting style suddenly changed, and the real body of this two-dimensional beautiful girl turned out to be a big five big and three big masters.

is an amazing reversal?

This is the direct embodiment of the motion capture technology and the real-time digital human technology of Seyou (Beijing) Technology Co., Ltd. (hereinafter referred to as Seyou Technology). Brand virtual mascots can interact with people in shows and evening parties; animation characters can jump out of the plot and have a cordial conversation with fans. Just like Jack uses his mind to control Avatar, it can be done in real time without post-production.

In the "virtual factory" of Shiyou Technology, nearly 400 virtual characters have been successfully resurrected in this way.

And all of this stems from the seemingly absurd idea of ​​the founder Ji Zhihui.

"We can not only do it, but also do better"

In 2010, Hollywood director Cameron 's 3D masterpiece "Avatar" was released in mainland China, with cool graphics and technology The domestic audience was greatly shocked. Many people stay up late to queue up to buy tickets, and it's hard to get a ticket.

is like a seed. "Avatar" kicked off the 3D version of the stereoscopic movie. It also gave Ji Zhihui, who has been in business for many years, another dream to sprout: "The virtual technology inside, I think we can not only do it, but also do it. Better."

At first glance, is Ji Zhihui's idea a bit absurd?

is known as "Avatar", which was produced in ten years, four years of production, and a cost of 500 million US dollars. Where did Ji Zhihui's prediction come from?

In fact, Ji Zhihui and his partners set up a virtual technology laboratory as early as 2004, and began to research cutting-edge technologies including data mining, graphics and image algorithms, workflow engines, real-time motion capture technology, etc., becoming one of the earliest domestic research and development real-time digital human technology's top-notch team,However, there was no mature application in the market at that time.

Until the emergence of "Avatar", he smelled the potential market of virtual technology, and the technology accumulated for many years finally had the opportunity to realize.

"Avatar mainly uses motion capture technology. The actor wears a special costume with many sensor balls on it to record the expression and movement data of the actor, but it is not real-time full-body motion capture. At that time The facial expression capture technology is very limited, it is difficult to track the lip movement in real time. Instead, it first records the lower part and body data to perform the capture data, and then puts the captured actor data on the workstation for later calculation, and then undergoes artificial refinement , And finally offline rendering and other processes can output the final picture." Ji Zhihui said.

has been engaged in the research of motion capture technology in the laboratory for a long time, making him realize that this technology can be further optimized to achieve "real-time" replication, without the need for post-rendering and processing.

So in 2011, Shiyou Technology was born.

"Like Jack, use your mind to control Avatar"

At the end of 2020, when a group of Chinese fans were chasing dramas and urging "Fox Fairy Little Matchmaker", they were pleasantly surprised to find that their idol was Tu Shan Susu went out of the animation and started a live broadcast.

A head of golden hair, two pink fox ears, a pink and white girlish skirt, "original" Tushan Susu , selling cute, dancing, singing, and interacting with fans in real time in the live broadcast room. Many fans exclaimed: I saw the "alive" goddess in my lifetime.

The supporting technology behind it is Ji Zhihui's original vision-real-time digital human technology.

is like a double reed or puppet show. The virtual character in front of the screen poses in various poses, jumping, spinning, singing, and making faces; behind the screen is a real actor wearing a motion capture suit and head. Facial equipment, hands wearing data gloves, point-to-point binding with any virtual image, real-time anti-shake calculation, animation correction, processing facial expression body finger synchronization, and send the data to the rendering engine in real time,Perform real-time rendering and environment simulation, and easily manipulate the virtual characters in front of the screen.

"It's like Jack lying in the cabin, controlling an avatar with his mind, and letting the avatar act according to his wishes. It is such a concept." Ji Zhihui said.

In the virtual world, in addition to virtual digital people, there are also virtual scenes that can constitute the virtual world.

On the stage of CCTV's 2021 Year of the Ox Spring Festival Gala, a song " ox up " impressed many people. On the virtual stage of the red Chinese elements, Andy Lau sometimes placed himself on the red platform of Phnom Penh, and sometimes placed himself on the "Chinese knot" and danced with the mechanical cows. It was so cool to have no friends. At the end of the song, the stage changes from virtual to real, and Andy Lau interacts with Wang Yibo and Guan Xiaotong .

The seamless scenes make many people think that Liu Tianwang has also come to the Spring Festival Gala. But in fact, with the exception of Wang Yibo, Guan Xiaotong and the real Spring Festival Gala stage, part of Andy Lau's performances are all made by the real-time virtual synthesis system of Shiyou Technology.

In addition, TCL double 11 president live broadcast, Yili gold collar crown Nianwei Street goods live broadcast, Netease live broadcast of Westward Journey Youth Same Road, UFO car new product launch conference, Tencent King Honor KPL virtual live broadcast, etc., are all produced by Shiyou Technology.

As the pioneer of real-time virtual technology in China, Shiyou Technology has now become the preferred technology provider for many brands and platforms. So far, it has served more than 1,000 top brands such as Tencent, Tmall, JD, NetEase, , Huawei, , Xiaomi, etc., and has reached in-depth cooperation with CCTV, Phoenix Satellite TV, Dragon TV, Hunan Satellite TV, Zhejiang Satellite TV, etc. ,Nearly 400 virtual characters have been successfully resurrected. In 2020, a revenue of nearly 30 million yuan has been achieved, and it is expected to exceed 50 million yuan this year.

"There is a hammer but no nails"

In less than 10 years, Ji Zhihui's ideals have been reflected in reality, leading Shiyou Technology to become an industry leader. But looking back on the road to entrepreneurship, it is actually not that easy. Shiyou Technology has even sat on the "cold bench" for several years.

In 2011, the newly born Seyou Technology quickly fell into an embarrassing situation: it has accumulated technology, but no hardware equipment; when it has both, there is no suitable market.

"We are one of the earliest people involved in virtual technology. At that time, related wearable devices were not available in the world, and there was no proven business model. The equipment used in the production of "Avatar" was developed by them internally, and only some of the physical equipment was used. . So we all do our own design and research and development." Ji Zhihui said.

Because of the technical reserves of many years in the early stage, in 2012, Shiyou Technology independently developed the Puppeteer virtual factory system, which is the first domestic real-time & fast virtual content production, virtual IP creation platform, real-time digital human and fast animation The production technology is at the leading level both at home and abroad.

But the market at that time was not strong.

"Before 2014, 3G networks were still used in China, and there was no Wi-Fi. Such basic communication conditions simply cannot support the large-scale implementation of virtual technology, and there are no such types of movies in China. So few people use us. In general, the traditional CG (computer graphics) artificial production method is used for animation." Ji Zhihui said.

There is no first-class technology, but there is no market. Many investors ridiculed Shiyou Technology: "Typically there are hammers but no nails." However, because of its firm optimism about the virtual market, Shiyou Technology, which "only exports but does not enter", does not Not abandoned.

With the launch of 4G and 5G networks, short videos, live broadcasts and other new formats have been waved. Especially with the rise of Generation Z (that is, the generation born between 1995 and 2009), the two-dimensional and three-dimensional culture has become a trend among young people. Real-life interpretation and traditional CG animation cannot restore the exquisiteness of the virtual world in real time.Moreover, the production cost is high and the efficiency is low, and the demand for virtual technology in the market is getting higher and higher.

By 2017, Shiyou Technology finally has the opportunity to make a big splash.

"A trillion-dollar blue ocean market in the future"

Today's World You Technology, real-time virtual technology has been applied to education, media, film and television animation, games, variety entertainment, finance, brand marketing, live broadcast/short video, etc. In this area, in Ji Zhihui’s words, "At this stage, Shiyou Technology can provide commercial applications of the entire system of technology and products in terms of technical algorithms, hardware equipment, software systems, and technical production."

However, in his opinion, real-time virtual technology is still in its early stage, and many industries and fields have not been rolled out. There will be a trillion blue ocean market in the future.

From the perspective of the live broadcast industry alone, according to the data of the Prospective Industry Research Institute, the revenue of China's live video industry in 2019 reached 108.2 billion yuan, and 390 million people across the country are paying attention to virtual idols. Among them, the largest two-dimensional activity community B station monthly active users Up to 11.4 million, virtual anchors accounted for 40% of live broadcast revenue. During the same period, large-scale live broadcast platforms such as Station B and Huya also launched online and offline virtual anchor activities. During the same period, the size of the game market reached 230 billion yuan, of which the RPG game , which is more relevant to digital humans, accounted for about 30% of the market.

Regarding the company's next development, Ji Zhihui said that on the basis of continuously improving the level of research and development and service capabilities, it will expand more cooperative companies and subsidiaries across the country, and promote the implementation of real-time virtual technology on a larger scale.

At the beginning of this year, Shiyou Technology established a branch in Zhengzhou-Zhengzhou Shiyou Huaguan Animation Technology Co., Ltd., aiming to deeply explore the rich cultural resources of the Central Plains, create a cultural IP of the Yellow River, and make Henan characteristic culture out of the Central Plains and move towards world. In the 2021 National Entrepreneurship and Entrepreneurship Week, Shiyou Huaguan will release the IP image of Shaolin Temple-Kung Fu Baby.

"Based on the industry's first IPCT model, Shiyou Huaguan has carried out in-depth development of "Kung Fu Baby". "Kung Fu Baby" is cute and cute.Eighteen martial arts are proficient in everything. In his bones is the kindness, simplicity, bravery and integrity, willingness to help, and hospitable character of the people of the Central Plains. He always uses a contented and happy mentality to bring positive energy to more people by creating happiness and passing on good luck. "Ji Zhihui said.

According to reports, in the future, Shiyou Huaguan will develop a large number of creative content around the "Kung Fu Baby" brand, and through the spread of a series of micro-contents on new media platforms, it will effectively enhance users' recognition of brand culture; at the same time; Continue to hold various theme activities offline to further enhance the interaction between the image and the audience, and achieve a double harvest of economic and social benefits.

Further reading

Shiyou Technology is a leading provider of virtual technology, focusing on The development and application of real-time virtual technology and AI digital humans. The Puppeteer virtual factory system independently developed by the company is the first domestic (2012) real-time & fast virtual content production and virtual IP creation platform. It is in the field of real-time digital humans and fast animation production. It is at the leading level at home and abroad.

In terms of AI applications, Shiyou Technology is positioned as the "UI" of AI, deeply integrated with AI technology, and has developed AI applications such as a certain brand of mobile phone smart assistants, AI companions, 7 *24-hour AI virtual virtual anchors, holographic AI interactive digital humans and other products provide intelligent and diversified applications for mobile phones, education, live delivery, digital exhibition hall virtual reception and other fields.

The company’s main business is to provide B-side customers Software and hardware system products and technical services, typical customer industries are , radio and television , the Internet, short video/live broadcast, brand marketing, education, film and television animation, etc.

Typical cases in the radio and television industry, such as the virtual technology and support and services of the CCTV Spring Festival Gala over the years (Andy Lau "bull up" in the 2021 Spring Festival Gala), Phoenix Satellite TV, Dragon TV, Hunan Satellite TV, Shandong Satellite TV, Zhejiang Satellite TV, Guangxi Satellite TV, etc. have also purchased or used the technical services of the old World You Technology. In addition, the World You Technology is Huawei, Xiaomi, Tencent, Baidu, NetEase, JD.com, Ali, Suning, Haier, TCL, L'Oreal and other leading companies provide virtual IP to create services or related products.

Currently, there are nearly 400 "resurrected" virtual IP images in the real-time IP library of Shiyou Technology.

Editor: Liu Anqi | Review: Li Zhen | Director: Wan Junwei