It is often said in the photography circle, "The bigger the bottom, the bigger the bottom will crush people to death."
This generally refers to the CMOS image sensor of a digital camera. The larger the size, the better the imaging effect. It is the key imaging component of the camera and the core of shaping the competitiveness of the image.
Therefore, we see that as the competition in smartphones becomes more and more fierce, and imaging technology plays an increasingly important role in mobile phones, sensor technology has become a battleground for many mobile phone manufacturers and is where differentiated competition lies.
However, mobile phones often pursue thinness, lightness and low power consumption, which naturally imposes physical limitations on the size of mobile phone image sensors. How to resolve this contradiction has become an important direction for the industry.
Recently, SenseTime Intelligent Industry Research Institute released the white paper "AI Sensor: The New Core of Smartphone Imaging", which proposed that as the smartphone market slowly enters a bottleneck period, imaging functions have become the focus of industry disruption, and AI software and CMOS image sensors The integration of hardware will be the solution to the continuous improvement of smartphone imaging capabilities.
Image sensors have entered the AI era
In the past few years, AI algorithms have greatly improved the imaging capabilities of mobile phones. However, in this solution, the image sensing and AI algorithm operations are completed on different hardware. The sensor provides image signals, and the processor or AI acceleration chip executes the AI algorithm. This will cause a waste of energy resources and make it difficult to handle some scenarios that require timely response. The
white paper points out that CMOS image sensors incorporating AI technology can maximize the value of original light signals.
At the beginning of the device acquiring visual signals, the AI sensor can be optimized and processed to enhance real-world perception, improve image and video quality, enrich content details, while minimizing device power consumption and enhancing data security. . The
white paper points out that since the emergence of optical multiplier tubes (Photo Multiplier Tube, PMT) in the 1950s, image sensors have continued to evolve with the development of technology.
Today, with the blessing of AI technology, the development of CMOS image sensors has begun to enter the era of AI software definition, pushing the overall capabilities of the imaging system to new heights.
turns your mobile phone into a "personal smart imaging studio"
With its leading AI technology and software and hardware integration capabilities, SenseTime has developed technologies in super-resolution, AI super-image quality, night scene imaging, high dynamic range, ultra-clear portraits, and image blur. , RGB ToF 3D and other key image scenes, it has accumulated a lot of algorithm capabilities.
At the same time, as early as 2019, SenseTime has been working closely with the world's leading image sensor manufacturers to directly integrate AI algorithms and sensor hardware. At present, SenseTime AI sensors have completed a number of products and successfully launched a number of high-end flagship mobile phones.
With the trinity of AI algorithm, AI sensor and AI ISP, SenseTime has realized the integrated empowerment of AI technology for the entire smart phone intelligent imaging system. On the one hand,
uses AI technology to fully tap the potential of lens modules, various new sensors and other hardware, maximize the use of the underlying characteristics of imaging equipment, and achieve optimal imaging quality.
On the other hand, AI technology is used to organically combine the independent components in the imaging system and fully link them, turning the originally independently operating imaging modules into a collaborative imaging platform.
From top to bottom are the comparisons of super-resolution, AI super-quality, and night scene image effects. The left side is the shooting effect of the traditional solution, and the right side is the shooting effect after SenseTime is empowered. The pictures are all partial enlargements of the original image
SenseTime's The solution has comprehensively improved the quality of mobile phone photography.
For example, AI can learn the different characteristics of multiple light sources to understand the natural spectral characteristics, and then bring AI noise reduction and AI super-resolution to improve imaging quality according to actual application scenarios; in special shooting modes, AI night scene enhancement, AI portrait enhancement gives smartphones a variety of "magic scenes".
These experiences make smartphones equipped with AI sensors become everyone's exclusive "personal smart imaging studio". Consumers can easily shoot high-quality works that are comparable to professional imaging equipment during daily use.
The infrastructure for machines to "cognize" the world
Looking to the future, the value of AI sensors is not only to improve the imaging capabilities of smartphones, but also to become the infrastructure for machines to "cognize" the world, giving more Internet of Things terminals intelligent perception. Enhanced capabilities with content.
With a huge base of 4 billion mobile phone users around the world, the widespread application of AI sensors will significantly reduce the cost of world modeling, become a bridge connecting the physical world and the virtual world, and accelerate the construction of " metaverse ".
AI sensors will also play an important role in emerging application scenarios. For example, in the field of smart cars, AI sensors will become the core components for vehicles to perceive the world; in the field of smart cities, AI sensors will play an important role in mining the value of video information.
Gartner data shows that the shipment volume of CMOS image sensors will exceed 8.4 billion units in 2022, and will exceed 10.7 billion units by 2026, growing into a semiconductor market segment with tens of billions of shipments.
adheres to the long-term layout in the field of AI sensor technology. SenseTime will work closely with partners to accelerate the penetration of market of AI sensors into , and jointly "redefine" the image sensor trend with AI software, giving hardware more innovative possibilities.
