Since its release in 2016, the AR game "Pokémon Go" has created countless miracles, which not only refreshed the public's perception of the AR experience, but also achieved great commercial success. Even after the global COVID-19 pandemic, players can still invest about US$1.3 billion in 2021, and is about US$400 million higher than in 2019.
But the mobile phone screen is neither designed for AR nor the ultimate belonging to AR.
1 On November 17, at the 2022 Snapdragon Summit held by Qualcomm , not only did the Snapdragon AR2 Gen 1 built on the Snapdragon XR2 Gen 1 platform be released—Qualcomm's first processing chip specifically for the next generation of AR smart glasses, Qualcomm also jointly with Niantic, a developer of "Pokémon Go", to showcase the latest reference design of the AR headset.
AR glasses reference design, picture/Qualcomm
According to the introduction, the design adopts a conventional AR glasses design at the front, the main body is an ultra-thin transparent display screen, and is equipped with multiple relatively hidden external and internal cameras and an AR coprocessor, with a distinct mask design to fit the face. The AR main computing unit is placed at the rear and then connected to the front display screen through the headband. The wireless connection module seems to be placed inside the headband.
At the same time, Niantic showed more details in a live AR prototype demonstration video, such as the battery being obviously placed at the back, and a hand controller is also included.
picture/Niantic
Niantic emphasized at the meeting that AR glasses or headsets should not weigh at least 250g and provide at least 1-2 hours of continuous battery life. To do these two things, the AR chip needs to be smaller, while also providing lower power consumption and stronger processing performance.
This is exactly what Qualcomm did on Snapdragon AR2 Gen 1, but compared with the existing performance improvement, the technical path Qualcomm envisioned for AR is more worth looking forward to.
computing power is not enough, and mobile phones come to make up
An objective fact is that the current real AR glasses are far from reaching the stage of industrialization and mass production.
At this stage, some AR head-mounted display products exist, such as Nreal, Thunderbird, etc., but it is more of them just as a "movie viewing device" and does not have much interactive capabilities. On the other side of the spectrum, such as Microsoft HoloLens 2 and recently launched Magic Leap 2, the high cost and large size and weight limit their possibility in the consumer AR market.
Currently, the core of AR devices is two points - optical display and computing performance. The latter has to face the impossible triangle of computing power, battery life and volume. Qualcomm is approaching this goal and shows a feasible future.
According to Qualcomm, Snapdragon AR2 Gen 1 adopts a multi-chip architecture, consisting of an AR processor, an AR coprocessor and a wireless connection module, distributed in different locations of the AR device and plays different roles. Compared with Snapdragon XR2 Gen 1, the PCB of Snapdragon AR2 Gen 1 has reduced space by 40%, AI performance has been improved by 2.5 times, and power consumption has been reduced by 50%. The distributed method also prevents the waste heat generated by the chip operation from gathering, thereby achieving better heat dissipation effect.
Picture/Qualcomm
These improvements make it possible to have a lighter long-range AR glasses, or in Qualcomm's words: "This allows AR glasses to be worn comfortably for a long time and meet the needs of consumers and enterprises."
specifically to multi-chip architecture, where AR coprocessor is responsible for collecting data from connected cameras and sensors, supporting eye tracking and iris recognition. Since the design focuses on data collection, power consumption is also reduced.
AR processor is the main computing unit . Qualcomm said that the AR processor is "optimized for the delay from low motion (hand movements, etc.) to photons (data transmission)" and supports up to 9 concurrent cameras. It also enhances a series of perception capabilities, including: a dedicated hardware acceleration engine that improves user motion tracking and positioning; an AIR accelerator that reduces the latency of sensitive input interactions such as manual tracking or 6DoF; a projection engine for a smoother experience.
But apart from that, many people ignore the wireless connection module of Snapdragon AR2 Gen 1.
Snapdragon AR2 Gen 1 Wireless connection module uses WiFi 7 to connect AR glasses to smartphones or other host devices. The data and perception results collected by the glasses - 6DoF, eyeball and gesture tracking, etc. - will be transmitted to the host side, and the chip on the host side will be rendered in the picture, and then transmitted through the encoder and compression, and finally transmitted back to the glasses side to display.
Picture/Qualcomm
From the perspective of data circulation, when users use AR glasses equipped with Snapdragon AR2 Gen 1, the AR coprocessor will collect a large amount of local data from local sensors, such as cameras, and then use the AR processor to transmit it to the flagship phone equipped with Snapdragon 8 Gen 2 through Qualcomm's FastConnect 7800 (WiFi 7 chip). The screen is rendered by Snapdragon 8 Gen 2, and is encoded and compressed, and is returned to the AR glasses for decoding, and finally passed through the reprojection engine and displayed on the screen of the AR glasses.
Qualcomm claims that the delay of the entire process will be less than: 9ms.
High-speed and low-latency wireless connection can already allow mobile phones to share the computing power requirements for some scenarios of AR glasses. Of course, the 9ms delay is far from satisfying all scenarios. Qualcomm pointed out that some tasks that are extremely sensitive to delay will be processed directly on the AR processor and displayed on the screen of AR glasses, such as stock trading, multiplayer online games, etc. Other more complex data needs will be transferred to host devices such as smartphones and PCs equipped with Snapdragon processors.
However, Qualcomm still drew a larger pie, or described a greater possibility.
mobile phone, also the host of AR,
In 2016, a video of a whale jumping and falling in the gym hitting all viewers who have watched it.
Picture/Magic Leap
At that time, the AR industry was still in its highlight moment, and The shock brought by Microsoft HoloLens has not yet dissipated. AR startup Magic Leap once again aroused our expectations and excitement for AR with this video and Magic Leap One. Soon after, it also received more than US$2.6 billion in investments from investors such as Google and Ali .
Wait until the first Magic Leap One creator version was actually released in 2018. The first batch of media found that Magic Leap One was far from the effect they promoted. The disadvantages were that the field of view FoV was small, the brightness was low, and it was not clear enough... But compared to Microsoft HoloLens, the weight of the Magic Leap One headset was only a little more than half of the former - 316g. By the time the Magic Leap 2 released this year, it was further reduced to 248g, which is largely due to the external processor box - a miniature PC or a special-shaped phone, because the system has shifted to Android.
Magic Leap 2's external host is equipped with Android custom system. Picture/Magic Leap
But if it is to be targeted at the consumer market, such a design also makes it difficult to bring Magic Leap to outdoor scenes. Qualcomm's idea is to replace wired with high-speed and low-latency wireless, and use a mobile phone equipped with a Snapdragon flagship processor as the "host", making full use of the high-performance computing platform in our pockets.
even goes even further. Qualcomm hopes to achieve complete distributed processing after 2025, distributing most of the computing power needs of AR glasses to various smart terminals around us, including smartphones, tablets, laptops, smart watches, etc.
In fact, ordinary users will not pay attention to the computing processing terminals during use, just like when using a PC, we will only pay attention to the monitor, keyboard and mouse, and when using a mobile phone, we will only pay attention to that screen. computing power does not matter to users, what is important is the interaction and display interface.
HoloLens 2, picture/Microsoft
The liberation of computing power can not only alleviate the contradiction between battery life and volume of AR glasses, but also have greater room for improvement in the optical display solution. Qualcomm's idea for Snapdragon AR chip is also: "To completely change the appearance of head-mounted glasses and create a new era of spatial computing experience for the combination of real-world/virtual worlds."
It is true that even Snapdragon AR2 Gen 1 is still far from the "new era" described by Qualcomm, and the core is a breakthrough in wireless connectivity and distributed technology, but Qualcomm is indeed getting closer step by step. Current split AR glasses almost all adopt wired connection designs, and only support wireless in simple tasks such as screen projection. Qualcomm understands that wireless is the future when supporting wired connections, and plans to promote low-latency wireless connections between AR glasses and mobile phones.
This is of course also the consensus of the entire AR industry. AR glasses will eventually evolve towards lightweight and miniaturization. But the difference is that Qualcomm believes that the technical path to achieve the goal should be distributed multi-terminal support, rather than actively entering the impossible triangle of "computing power, battery life and volume" in the context of Moore's Law failure.
Question picture comes from Niantic.
