When it comes to computer vision, as a very popular subject now, it is actually very close to our daily lives. For example, face recognition, roadside signs captured by cameras can be directly translated into Chinese text, or text information in pictures can be selected and copied. These can be considered typical applications of computer vision/machine vision.
Early experiments in computer vision date back to the 1950s; in the 1970s, there were commercial applications of machine vision to distinguish handwriting and written text. Simply put, computer vision allows computers or machines to understand and interpret images and visual data, and to some extent replace the human eye to perform tasks such as observation, identification, positioning, inspection, measurement, and decision-making. Machine vision systems can automatically acquire and analyze visual images, provide information and control machines or workflows.
Its application scope is of course not limited to face recognition for taking pictures. For example, when we visited Infineonās factory before, we found that they use machine vision to check the yield problems of semiconductor products in certain processes - which is more time-consuming than manual inspection. It is very efficient; at the micro level, EDA and foundry factories also use machine vision to discover defects in chip manufacturing... Expanding further, when the term Industry 4.0 became popular earlier, we knew that machine vision is realizing industrial automation. plays a very important role in the process.
If we simply divide the computer vision workflow into three major blocks, it roughly includes image capture, image processing, and image analysis and understanding. Based on this, the cover story of this issue of "International Electronic Business News" interviews Smartway, Prophesee, Yinniu Microelectronics, and Imagination Technologies. Among them, SmartSmart and Prophesee are mainly image/vision sensor suppliers and related image capture; Imagination is involved in the latter two - in fact, Imagination also developed ISP (image processor) earlier; Yinniu Microelectronics focuses on 3D vision , its role positioning should cover 3 processes.
Although from the perspective of the industry chain, there are a large number of other market participants, such as upstream light sources and lenses, midstream system integrators, as well as software and algorithm suppliers other than hardware; however, we still look forward to this article It can briefly outline the development potential of the current machine vision market. Letās talk about this field, which has a development history of more than 70 years, and how it is doing now.
Discussing machine vision in various contexts
First of all, letās clarify what the words machine vision and computer vision are talking about. Regarding the definitions and differences between these two words, there are differences in the explanations given by various sources. iiMedia Consulting previously reported that machine vision is the engineering of computer vision technology. āComputer vision provides machine vision with the theoretical and algorithmic basis for image and scene analysis. Machine vision provides sensor models, system structures and means of realizationā.
seems to make some sense. It sounds like different dimensions in different contexts. Judging from information such as Wikipedia, the term "machine vision" is more biased towards the application of "computer vision" in the industrial field. Many experts we interviewed this time also said similar things. For example, Mr. He Huogao, co-founder and vice president of Yinniu Microelectronics, mentioned: "Machine vision technology gives industrial equipment the ability to 'see'" and "Machine vision is a very important application field of computer vision technology. Computer vision is An important part of machine vision technology. "Rob Fisher, product director of
Imagination Technologies, mentioned: "Machine vision can be considered a subset of computer vision, and computer vision includes a wider range of applications." Product Director Gilberto Rodriguez said: "Computer vision. The concepts of vision and machine vision are affected by machine learning and are changing rapidly. "
Prophesee Many experts mentioned in their answers that the boundaries between machine vision and computer vision are blurred, "In the same field, we can often see them being used interchangeably. In our opinion, computer vision belongs to the broader field of vision technology, and machine vision is a subset of computer vision."More specifically, we can think of machine vision as a set of task-oriented vision skills applied to some specific applications (presence detection of objects, quality control, dimensional inspection, automated inspection, pass/fail decision-making... .). Computer vision is an interdisciplinary field that includes the most advanced visual perception and computing at the technical level. "
This article will no longer specifically distinguish between the two words computer vision and machine vision, and the scope of discussion will only be limited to the meaning of "machine vision". (For example, many industry reports and this article do not count automotive ADAS systems as machine vision, even though It applies many computer vision technologies; but in some literature, these two words are indeed interchangeable)
The second question to clarify is what is the relationship between computer vision and AI. After all, we often see this now. The two words appear at the same time. And understanding the relationship between the two will help us understand the development prospects of computer vision. When we checked the information, what surprised us was that almost all experts mentioned that computer vision is the transformation of AI. When applied to the visual world, computer vision is even a subset of AI. According to our early understanding, computer vision does have some solutions for applying neural network technology, but this is not all of computer vision.
Later we discovered that the general public. The definition of "AI" is broader, and it generally refers to the imitation of human behavior or other human characteristics and intelligence. Then machine vision is originally a simulation of human eyes and understanding in a specific field, so it can naturally be classified as AI. In fact, what we often call AI now does not have such a broad definition.
Gilberto said: "Computer vision was originally used to describe algorithms written by humans and executed in general or specialized computing hardware. With the improvement of machine learning performance and the existence of efficient heterogeneous architecture, we can now obtain algorithms through training (AI training), and no longer need people to write code. This changes our understanding of computer vision concepts. He emphasized in particular, āWith the greater application of AI and machine vision technology, the adoption of traditional computer vision technology is becoming less and less. "The meaning of "AI" here has actually been narrowed.
When we say that in the ImageNet image recognition competition in 2012, the AlexNet convolutional neural network (CNN) stood out and was accelerated based on GPU. This is an innovation in AI. And AlexNet was also considered to be the most far-reaching technological innovation in the field of computer vision. The concept of AI has been narrowed to deep learning (which is also the narrow definition of AI by many people). The application of AI is the major trend in the development of computer vision technology in the past two years. In this context, it can no longer be said that computer vision is a subset of AI; rather, it should be said that AI is driving the development of computer vision technology
AI. Thrust
AlexNet should be a representative of the development of machine vision technology promoted by AI; the contemporary ResNet residual neural network has become a frequent visitor when we talk about AI and when AI chip companies release products. This actually shows the development of contemporary computer vision. It is driven by AI. As Gilberto said, the weight of traditional computer vision technology is significantly reduced; or from a programming perspective, clear rule judgments based on human experience will become more and more important. Not as good as AI technology.
āEspecially the craze of artificial intelligence in recent years has contributed to the development of machine vision technology. AI has entered a new level, not just comparing computing power and indicators, but making machines truly have human characteristics and attributes. In the future, AI will play an increasingly important role in the field of machine vision and lead its future development direction. "Mr. He Huogao said.
Regarding the application of deep learning in the field of machine vision, although we cannot provide exact figures. However, from the companies we interviewed, we can see that the proportion of AI is increasing. Imagination does not need to be more He said that in addition to the GPU, the company's current main product is the NNA neural network accelerator. The GPU itself is also used for computer vision tasks, "such as 360Ā° dewarping, overlapping, information display, etc." " Gilberto said.
And "Convolutional neural networks are very good at finding and classifying visual objects..." "NNA is one of our main computer vision processors. NNA is very suitable for deployment in high-efficiency machine learning calculations, realizing the transition from traditional computing to machine learning." The transition of learning.ā And of course its RISC-V CPU is also part of the pre-/post-processing of machine learning tasks.
Yinniu Microelectronics focuses on 3D binocular stereo vision technology. It is worth mentioning the NU4000 chip self-developed by this company. Mr. He Huogao said: "NU4000 not only integrates the depth engine, AI computing power engine, general CPU core, and SLAM engine, it enhances the scalability of application scenarios and improves integration. It has also been greatly improved, truly becoming a single-chip solution SoC from depth perception to AI computing to system control. After NU4000 integrates third-party DSP and CNN engines, other AI processing capabilities that Yinniu will develop in the future will also be integrated into it. On the next generation chip." He revealed that the chip products planned to be released in the future will not only have stronger main control CPU and better energy consumption ratio, but also have better 3D depth visual performance and increasingly powerful AI computing power.
Smartway, as a CIS (CMOS image sensor) supplier, has been exploring solutions to bring AI computing power closer to CIS in the past two years - this is actually what CIS manufacturers, including Sony, have been doing in the past two years. . In this interview, Smartway talked about the "AI smart sensor platform", that is, "the integration of edge AI computing on the image sensor can effectively improve the resolution of key areas (such as faces or license plates), reduce latency, and With high frame rate and ultra-low power consumption, it can solve the problems of slow response, presentation and recognition caused by insufficient frame rate and insufficient resolution for advanced artificial intelligence applications such as face recognition, advanced assisted driving systems, driverless driving and robots. It is actually quite common to apply AI to data generated by traditional CIS. But the event-based vision sensor promoted by Prophesee, another company we interviewed this time, is a type of sensor that we think may not be related to AI. Because this sensor is different from traditional frame-based image sensors, it senses scene change information and static parts will not be captured; it is more suitable for conventional simple rule judgments. However, Prophesee co-founder and CEO Luca Verre told us that iCatch and Japan's DMP and Restar have recently announced cooperation with Prophesee to develop event-based vision sensor AI solutions and the world's "first event-based Edge AI Vision Systems and Services".
Figure 1, computer vision accounts for nearly half of the AI āāResearch Funding Portfolios and Extreme Growth research
No matter how the AI āāis implemented or in which link, event-based visual sensors are being combined with (or assisted by) AI ), machine vision and AI have reached an inseparable point. After all, isnāt one of the hot spots in AI just computer vision?
A paper AI Research Funding Portfolios and Extreme Growth published by Georgetown University last year analyzed 600 large-scale AI research clusters and found that related computer vision research fields accounted for nearly half (Figure 1). Mr. He Huogao said by analogy: "Computer vision is the most important application scenario of AI, because vision accounts for 80% of all human sensory input." Based on the fact that AI is currently a hot spot in the market, we can all say that computer vision is currently "in full swing" " stage of development.
development space geometry?
There are many ways to judge the development stage of an industry, including looking at the current market growth rate, the iteration cycle of new technologies, and the share distribution of market participants. Whatās more incredible is that we have read about 10 industry reports related to machine vision. The market value estimates of many research institutions for the industry are very different in magnitude - the data span is as much as 4 times; and the estimates of the market value of the industry, There are also different interpretations of the importance of regions. This may be related to the differences in the definition of "machine vision" in different research institutions, or the different scope of statistics.
We speculate on the current development stage of the machine vision market from market size data, application directions, and new technology points.
takes the median value of report data from institutions including Grand View Research, The Business Research Company, Qianzhan Industry Research Institute and other institutions. The global market size of machine vision is approximately US$13-15 billion this year, with an expected compound annual growth rate from 2021-2026. (CAGR) at 8-12%, it should be said that it is still a rapidly growing industry.
Among them, reports from multiple research institutions mentioned that the market is "highly fragmented" with a large number of market participants. The top 10 market players accounted for less than 20% of the entire market in 2020. The "market participants" here should mainly refer to system-level suppliers (such as industrial cameras), typically such as Cognex, Keyence, etc. The general electronics industry is highly fragmented, which means there is still huge room for competition in this field. However, this question may require a closer look at the suppliers at different levels in this field, covering various related components such as light sources, lenses, sensors, algorithms and software.
As for the application markets and directions of machine vision, they mainly include automobiles, food and beverages, pharmaceuticals and medical care, electronics and semiconductors, industrial robots, packaging and printing, etc. Replacing manual inspection and measurement in industrial operations is a relatively large application market for machine vision - after all, the development direction of the entire society is a process in which labor costs are constantly rising. The essence of machine vision is an integral part of automation, so we always say that Industry 4.0 is closely related to machine vision.
Figure 2, 2020-2026 industrial and automation camera market outlook; Source: Yole DĆ©veloppement
When it comes to which industry has become the focus of machine vision development, the opinions of research institutions are quite different. For example, Qianzhan Industry Research Institute believes that electronics and semiconductors (manufacturing) are currently the largest downstream markets for machine vision; while Mordor Intelligence and Grand View Research believe that the largest application direction is automobile manufacturing. Judging from the data, the two should indeed be on par (although there may still be a gap in magnitude). At the same time, the development potential of the automobile industry is generally considered to be the greatest.
In addition, there are also differences on the question of who is bigger, the North American or the Asia-Pacific market, but it is also a consensus that the Asia-Pacific market has greater development potential... To verify the reliability of these data, Cognex's financial report is probably a way. Judging from Cognex's 2020 annual report, 30% of the company's revenue comes from consumer electronics, 20% from the logistics industry, 20% from automobiles, and 30% from others. The United States is its largest market, followed by Europe. However, if you add its revenue from Greater China and other regions in Asia, it is indeed second only to the U.S. market.
Of course, one company's situation cannot explain the entire industry. In our experience, the vagueness of the data in the analysis reports, the differences in magnitude, and the different conclusions all indicate that the industry is still in the early stages of its rise. We believe that at the current stage of technological development, there is still a huge demand waiting to be filled; or at least the market is still far from being "mature".
As mentioned above, the development of computer vision as an interdisciplinary subject itself has been around for some years. But in fact, many technologies started very early, such as event-based vision sensors, which have only become very popular in the past two years. Luca told us that this technology was proposed as early as the 1980s, but it took a long time for the technology to truly mature. It's been these two years. As for this type of sensor that seems relatively new and has great development prospects, Yole DĆ©veloppement predicts that neuromorphic AI will reach a market size of US$7 billion by 2030. By then, about 9% of the CIS market will be taken up by this type of sensor. . This shows that the emergence of new technologies is promoting the development of the entire industry; and that the machine vision industry itself is also trying various new technologies.
Figure 3, the 3D vision market will reach its maximum growth in the next few years; Source: Allied Market Research
3D visual perception Needless to say, several reports have specifically mentioned the development of smart camera systems, which will be largely attributed to the growth of the 3D imaging market Growth - This is something we mentioned in our previous 3D ToF technology report. Because 3D machine vision can now provide accurate and real-time information, giving machine vision more data. 3D perception and imaging are inherently fields that can be discussed separately.In the field of 3D vision/imaging, Yinniu Microelectronics focuses on stereoscope, specifically binocular vision.
"The biggest difference in machine vision today is that future development will increasingly apply 3D vision technology, especially dense depth stereo." Mr. He Huogao said, "Just like Just like the Cambrian period when the earth's species exploded, the evolution and communication of species flourished rapidly because of the ability of binocular stereoscopic vision. The "
" machine did not know the mutual positional relationship and distance between the objects in the image before. Visual illusions and misjudgments often occur. With the emergence and development of 3D/stereoscopic vision perception technology, especially the development of binocular stereovision technology, there are very effective tools to avoid illusions and misjudgments in machine vision. . As a result, 3D/stereo visual perception technology is increasingly used in machine vision, and its importance and efficiency are constantly recognized and valued by the industry. As the leader of the AIoT heterogeneous platform, Yinniu is the core. Competitive advantages include not only AI computing processing capabilities, but also a very unique binocular stereoscopic 3D vision algorithm engine and integrated general-purpose CPU core, which are indispensable and important components in the field of machine vision. "
From this perspective. , it is self-evident that 3D vision technology, represented by Yinniu Microelectronics' products, will win more markets in the future. In fact, even for traditional CIS, SmartSite is also trying to enhance machine vision technology, including what we know as high frame rate, global shutter, imaging under non-visible light (far-infrared enhancement technology), etc. Judging from Smartway's official website, "machine vision" is displayed as one of its major application directions. Drones, sweeping robots, AR/VR, smart code scanning, face recognition, industrial cameras and smart transportation systems are all the future prospects of the machine vision industry that Smartway sees.
From the perspective of technological innovation and the new market forces currently emerging, if the sum of these machine visions is accurately called an "industry", then the industry still has a lot of room for development even at the technical level.
and its subordinate branches have room for further oligarchization, especially as AI technology becomes more and more mainstream. Although the requirements for machine vision technology may vary widely when facing different downstream application fields: Prophesee said that based on its cooperation experience, even if it is also an event-based vision sensor, different industries have different requirements for technology. So there may be room for further discussion on this issue.
When we observe the field of machine vision, there are actually many other gains, such as certain technical trends: such as the fusion of multiple sensors (including visual sensors and other types of sensors), the rise of edge computing, and the direction of computing power units. Sensors are getting closer together, and energy efficiency continues to improve. But the real general direction is still: as AI technology continues to advance, even though machine vision has accumulated the development process of the past few decades, it is really just the beginning at this point in time.
Editor: Clover.li
