Virtual reality videos are being paid attention to by more and more people as a new generation of video types. Different from traditional videos, their good immersion and greater freedom bring people a next-generation video experience. However, the bandwidth required for 360° immersive video transmission is much higher than that of traditional videos. The existing video transmission system defaults to the user's perception of 360° videos as the traditional videos, which is seriously inconsistent with the way the actual user watches immersive videos, resulting in serious bandwidth waste. Peking University proposed a new generation of 360° video transmission system Pano based on the immersive video perception quality model, aiming to maximize user perception quality under limited bandwidth, and significantly reduce bandwidth consumption while providing a high-quality viewing experience. At present, this achievement has been included in the world's top academic conference ACM SIGCOMM 2019, in the direction of computer networks, and has attracted widespread attention at home and abroad.
360° immersive video, also known as panoramic video, is different from the single viewing angle of traditional videos. People can watch any area of the video at 360 degrees freely. In the past few years, more and more video content providers (such as iQiyi , Youku, Youtube, etc.) have provided 360° online video services, but due to the ultra-high bandwidth demand of 360° video, online user experience is still at a low level. According to relevant literature research, 360° video requires 80 times the bandwidth of traditional video transmission to achieve a similar experience as watching traditional videos. Although some existing 360° video transmission systems propose transmission methods based on user windows, which reduce bandwidth consumption to a certain extent, this is still very different from the way users actually perceive 360° video quality, causing serious bandwidth waste. Researchers from Peking University and the University of Chicago recently conducted in-depth research on the mechanism of human eye sensing 360° video quality and proposed a 360° video perceived quality model, which greatly reduced the bandwidth consumption of 360° video.
360° video perceived quality model
Pano proposed that when users watch 360° videos, their perceived quality is related to the following six factors. These include three traditional video factors and three 360° video-specific factors.
Three traditional video factors include: 1) the brightness of video objects. According to relevant literature research, when the brightness of video objects is too high and too low, the user's visual sensitivity to the object will drop significantly. 2) Texture complexity of video objects. According to relevant literature research, when the texture complexity of video objects is high, the user's visual sensitivity to objects will be greatly reduced. 3) The angle between the video object and the user's central viewpoint. According to relevant literature research, users only have high visual sensitivity to objects around the gaze point. For other areas, as the angle between the video object and the user's central viewpoint increases, the user's visual sensitivity will gradually decrease. Different from traditional videos, since the 360° video has a wider viewing angle and a larger range, the influence of angle factors is also more significant.
In particular, according to Pano's research, there are also three factors that are unique to human eye perception quality in 360° videos: 1) the speed of video object movement. Traditional human visual system research shows that human visual sensitivity to moving objects will decrease. Similarly, Pano's study found that in 360° video, there is also a decrease in visual sensitivity due to user's head rotation and relative movement of objects. 2) The depth of field difference between the user's viewpoint and the video object. In reality, when people stare at an object of a fixed distance, the remaining objects of different distances will become blurred. Pano found that this feature of the human visual system also exists when watching 360° videos. 3) The light and dark adaptation of the human eye. Research on traditional human visual systems shows that the visual sensitivity of human visual systems to objects will drop significantly after a period of time after a large change in light and darkness. Pano found that the impact of light and dark changes in 360° videos also exists.
Pano for the first time introduced the above factors of human eye visual perception 360° video quality into network video transmission. A quantitative mathematical model of user visual sensitivity was established using scientific statistics. Secondly, the 360° video is divided according to different visual sensitivity levels to form spatial video chunking driven by user visual sensitivity.Finally, according to the actual viewing scenario (such as the user's viewpoint position, the user's viewpoint movement speed, etc.), the transmission code rate is dynamically allocated to different regions, such as the lower sensitivity area transmitting a lower code rate, and the higher sensitivity area transmitting a higher code rate.
The research team of Peking University has implemented Pano's system prototype. According to a large number of subjective experiments, Pano's 360° video perception quality model can greatly improve the user's viewing experience and reduce bandwidth consumption. Under the same quality experience conditions, bandwidth consumption can be saved by 41%-46% compared to the existing 360° video transmission technology.
research value
Currently, 360° video has been provided by major video content service providers, but due to the ultra-high bandwidth demand, its quality experience is far inferior to traditional videos, which is also an important reason that hinders the further use and popularization of immersive videos. The value of Pano lies in introducing the visual characteristics of the human eye into network video transmission, which can make full use of the current limited bandwidth and provide users with the highest possible video viewing experience, which is of great significance to the further promotion and popularization of immersive videos.
conclusion
This project is still a long way from the realization of industrialization. More accurate perceived quality models and more realistic system optimization are goals that still need to be further improved in the future.
At present, the academic paper "Pano: Optimizing 360° Video Streaming with a Better Understanding of Quality Perception" of this research has been accepted by ACM SIGCOMM 2019. It was jointly completed by Zhang Xinggong's team from the Institute of Computer Science and Technology of Peking University and Jiang Jingchen's team from the University of Chicago, and has been supported by institutions such as Ali , iQiyi .
About SIGCOMM
SIGCOMM is an international top academic conference in the field of the Internet, with extremely high requirements for the quality of papers. In 2019, only 32 papers of the main conference were hired. Over the past 30 years, many research results of SIGCOMM have been widely used, such as the Internet Domain Name Management System (DNS), TCP protocol, etc.
Paper address: https://people.cs.uchicago.edu/~junchenj/docs/360StreamingQuality_SIGCOMM.pdf
