In the field of intelligent surface defect detection and classification, my country's industrial manufacturing has long been highly dependent on industrial intelligence software provided by foreign manufacturers such as the United States, Europe, Japan, and South Korea. Its introduction threshold is high, its autonomy and controllability are low, and there is a risk of being restricted and disabled. Many domestic scientific research institutes and large enterprises are investing heavily in the research and development of their own products in this field. However, due to the reality of strong background interference, few key samples, difficult grading, and high latency in surface defect detection and classification in industrial manufacturing scenarios, they have been unable to break through the accuracy and efficiency index requirements on actual production lines.
This project has carried out in-depth research on the intelligent detection, classification, and grading of surface defects. It has achieved innovative results in defect detection under complex background conditions, defect classification under small sample conditions, intelligent determination of defect severity, and model lightweighting. It has broken through the existing technical bottlenecks and has been implemented and demonstrated in many leading manufacturing and benchmark companies.
Technical Description
Intelligent detection and classification of product surface defects is an important part of realizing "intelligent manufacturing", especially in the pan-semiconductor industry such as panels, where the manufacturing process of products is complex and cumbersome. Quickly and accurately locating defects, determining the severity level of defects and handling them efficiently are necessary conditions to improve the international competitiveness of products. In recent years, with the continuous development of intelligent manufacturing theory and technology, machine vision technology based on deep learning has been increasingly used in panel, integrated circuit and other manufacturing industries to replace manual surface defect detection and classification. However, due to the complexity and particularity of industrial production scenarios, surface defect detection and classification technology based on deep learning faces different challenges from other people in actual industrial production scenarios. Problems and challenges in the application field of artificial intelligence :
1. Complex defects are difficult to detect : There are many types of industrial product defect detection objects with great differences. The objects to be detected usually have complex textures and backgrounds, which leads to problems such as difficulty in detecting subtle defects, confusion and mutual interference of different defects, which greatly affects the defect detection effect.
2. Key defects are difficult to identify : Training the intelligent defect detection model requires a large amount of annotated data. However, in actual production, the incidence rate of key serious defects is often low and the number of key samples is small, making it difficult for the model to accurately identify such key defects.
3. It is difficult to determine the severity of defects : The same type of defects occur in critical areas and non-critical areas, which have different impacts on product quality. The layout of critical areas in different product designs varies greatly. There is currently no effective method to identify critical areas, and the severity of defects is difficult to determine.
4. Intelligent model has high latency : The intelligent model has a large number of parameters and high computational complexity, which results in the time and computing resources required for the actual running of the model being unable to meet the real-time and economical requirements of the production line. In order to solve the above problems, Shuzhilian took the lead in forming a scientific and technological research team and carried out a series of technical research. The relevant results have been implemented on a large scale in panel and other manufacturing companies, ranking first in the domestic market share.
Its main key core technologies:
In view of the problems and challenges faced by surface defect detection and classification technology based on deep learning in actual industrial production scenarios, the project team conducted in-depth research and achieved innovative results in intelligent detection of defects under complex background conditions, intelligent classification of surface defects under small sample conditions, determination of defect severity levels, and lightweight surface defect detection models, breaking through existing technical bottlenecks.
1. Proposed an intelligent defect detection technology under complex background conditions, introduced the symmetric attention mechanism into the detection network, and adopted the symmetric data enhancement method to solve the problem of missed detection and misdetection of subtle, obstructed, and easily confused surface defects, and improved the overall detection accuracy.
2. Proposed a small image sample intelligent classification modeling method, which constructs a multi-level label propagation network based on multi-scale features, and based on information diffusion theory, solves the problem of model construction when training samples are insufficient, and effectively improves the classification accuracy of imbalanced samples.
3. Proposed an intelligent defect severity grading technology that integrates the spatial distribution characteristics of defect severity with attention distribution, and adopts the idea of adaptive hierarchical clustering to solve the problem of intelligent determination of defect levels and greatly improve the efficiency of defect handling.
4. A lightweight technology for surface defect detection models in industrial manufacturing is proposed. It combines knowledge distillation and channel pruning, and adopts a joint knowledge and model-driven approach to solve the problem of high delay in defect detection. On the premise of ensuring accuracy, it effectively improves the real-time detection and reduces resource consumption.
The results of this project have been successfully applied to leading enterprises such as BOE , Tianma Microelectronics , CSOT , Huike , AT&S , etc. It can effectively improve defect detection efficiency and accuracy, save detection labor costs, and achieve significant economic and social benefits. The project has achieved a cumulative sales revenue of 150 million yuan, successfully introduced into 25 large factories, ranking first in domestic market share, and has established a competitive advantage in the new display field. It is expected to save customers approximately 2.5 billion yuan in costs in the next 3-5 years. In the future, there will also be huge market space in industries such as chips and new energy.
relied on the technology of this project to successfully develop a cloud-edge integrated platform for intelligent detection and classification of defects, breaking through the technology monopoly of foreign manufacturers and realizing independent controllable and large-scale application of defect detection industrial software. At the same time, it has greatly promoted the intelligentization process of domestic manufacturing enterprises, effectively improved their international competitiveness, and made significant contributions to the implementation of the manufacturing power strategy and the Encore project.
★Patent application number/publication number : CN113920117B
Development Team
· Name of team leader: Sun Chongjing
Sun Chongjing, senior algorithm expert, Ph.D. of University of Electronic Science and Technology of China. Responsible for the development and application implementation of system products, he has made important contributions in three directions: intelligent detection of defects under complex background conditions, intelligent classification of small samples of surface defects, and determination of defect severity levels. Names of other important members of the
team: Peng Xiangnan, Fang Yuke.
·Affiliated organization: Shuzhilian
Chengdu Shuzhilian Technology Co., Ltd. is a big data and artificial intelligence product and solution service provider with computer vision technology, multi-dimensional data analysis and mining technology and natural language processing technology as its core. The company focuses on the two major areas of "intelligent manufacturing and smart city " and relies on the "integrated cloud-native digital intelligence service platform" to provide governments, enterprises and national defense units with data governance, data visualization analysis, data mining and other platform- and scenario-oriented digital intelligence technical services to help customers reduce costs, increase efficiency and improve quality.
related comments
" Wuhan Tianma automatic identification classification system (ADC) project has completed the online operation of multiple site models."
—— Wuhan Tianma Microelectronics Subsidiary Co., Ltd.
Xiamen Tianma Microelectronics Co., Ltd. user evaluation: "This project uses algorithm models to achieve automatic defect location, defect classification, defect grade determination and other functions of AOI equipment camera inspection pictures covering 97 sites in four factories. Through the application of the system in enterprises, the defect coverage rate has been improved, which is equivalent to the personnel replacement rate. More than 80%, that is, saving 80% of labor costs; in the production site of the formal production environment, the factory's production capacity and full production demand are guaranteed. ”
——Xiamen Tianma Microelectronics Co., Ltd.
