Recently, NeurIPS (Neural Information Processing Systems), one of the world's most prestigious AI academic conferences, announced the results of the 2022 paper acceptance. The paper "An Embarrassingly Simple Approach to Semi-Supervised Few-Shot Learning" was successfully accepted by NeurIPS 2022.
As one of the most prestigious AI academic conferences in the world, NeurIPS is an important event in the academic community every year. NeurIPS’ full name is Neural Information Processing Systems, the NeurIPS Foundation is usually hosted in December each year. The content discussed at the conference includes many subdivided fields such as deep learning, computer vision , large-scale machine learning, learning theory, optimization, sparse theory, etc. NeurIPS is in its 36th edition this year and will be held from November 28 to December 9 for a two-week period. The first week will be held at the Ernest N.Morial Convention Center in New Orleans, USA, and the second week will be changed to an online meeting. NeurIPS 2022 paper submission has ended on May 19, and today the official has finally announced the recruitment results. According to data given in the official website email, there were 10,411 papers contributed at this session, with a reception rate of 25.6%, slightly lower than last year's 26%.
Paper Interpretation:
Figure 1: Paper Summary
Paper Overview:
This paper proposes a semi-supervised low-sample image classification learning method based on anti-label learning, including the following steps: constructing metatasks, using pre-trained neural network as feature extractor, extracting features corresponding to support sets, query sets and unlabeled image data sets in the metatask, and training a classifier on the support set for subsequent classification tasks; the anti-label learning module inverses the labelless data with a higher accuracy rate, and the classifier learns and updates on the inverse tags, and iterates continuously until the anti-label cannot be selected. The positive label learning module, after the iteration of the anti-label module, obtains positive labels with a high accuracy rate, and uses a classifier to learn and update.
This paper extracts the characteristics of the corresponding data in the metatask through convolutional neural network , uses labelless data at a higher accuracy rate through the inverse tag construction module, and uses classifiers to learn and update on the inverse tag data. After iterating, the positive tag learning module is designed to obtain positive tags with balanced categories and high accuracy rates. Use classifiers to learn and update on the positive tag data to make more fully and high-quality use of labelless data, and can obtain higher accuracy of classification of few samples to learn images.
Innovation Background:
With the development of deep learning, convolutional neural networks have surpassed the human level in multiple image tasks, but the training of these models depends on a large amount of data. In real life, some data collection is difficult, such as the collection of all kinds of defective data on LCD screens. In addition, the annotation of these data also requires a lot of manpower and financial resources. By contrast, human vision system can quickly learn new concepts and features from a small number of examples, and then identify similar objects in new data. In order to imitate this ability of humans to learn quickly and reduce the dependence of methods on data, few-sample learning has attracted more and more attention in recent years. Small sample learning aims to quickly generalize to a new task that contains only a small number of samples with supervised information in combination with prior knowledge . Under this setting, only a very small or even one labeled sample is required for each category, so it can greatly reduce the cost of manual labeling.
is based on learning a setting with a small amount of data, and one problem that needs to be faced is that it is difficult to fit the model well to the distribution of the data with very few annotated data. Therefore, in order to solve such problems, a research direction combined with semi-supervised emerged in small sample learning. In addition, in order to solve the problem of data labeling difficulties, anti-label learning methods have emerged. As the name suggests, anti-labeling is to label the data opposite, which is an indirect way to indicate that the data does not belong to a certain category. This approach can greatly reduce data annotation errors. For example, for a 5-category problem, the probability of labeling the data is 4 times the probability of labeling the data incorrectly.In addition, in semi-supervised and small sample learning, since there is very little labeled data, it is difficult for the model to have good results in the initial stage. Using such a model to label pseudo-labels for unlabeled data will lead to a large number of errors and category imbalances. In such a situation, combined with anti-label learning methods can solve such problems. The semi-supervised and few-sample learning method based on anti-label learning research in the present invention is designed to design a suitable anti-label annotation method for semi-supervised and few-sample learning, and combined with anti-label learning, solve problems such as insufficient utilization of labelless data that occurs in semi-supervised and few-sample learning.
At present, many methods have been studied to study semi-supervised few-sample learning, but there are still some problems: 1) The accuracy of labeling pseudo-labels on unlabeled data is low, and incorrectly labeled samples will affect the final result; 2) There is category imbalance in the pseudo-labels marked on unlabeled data; 3) The method is relatively complicated.
Main contributions of this paper:
This paper proposes a semi-supervised low-sample image classification learning method based on anti-label learning. The method is as follows:
Step 1, construct a metatask, use a pre-trained neural network as a feature extractor to extract image data, extract features corresponding to the support set, query set and unlabeled data set in the metatask, and train a classifier on the support set for subsequent image classification tasks;
Step 2, the anti-label learning module inversely labels the unlabeled image data with a higher accuracy rate of 95%, and uses a classifier to learn and update on the inverse labels, and continuously iterate until the anti-label cannot be selected;
Step 3, the positive label learning module obtains positive labels with a category balanced and correct rate as high as 85%, and uses a classifier to learn and update;
Step 4, the trained classifier predicts the final image classification results on the query set.
Compared with the existing technology, the method proposed in this paper has significant advantages:
(1) The anti-label learning module designed by the present invention, by labeling the unlabeled image data and learning it, the error rate of labeling labels on the unlabeled image data is greatly reduced in the initial stage of poor model effect;
(2) After the anti-label learning module, the positive label learning module designed by the present invention can obtain positive labels with high accuracy and balanced categories, and continue to train the model;
(3) The process proposed in this invention is simpler than the previous method, and can use labelless image data to learn more fully and with high quality, and finally achieve better results in the image classification task.
Innovation Qizhi CTO Zhang Faen (one of the authors of the paper) said: "The current deep learning technology has a great dependence on the number of manually labeled data samples (i.e., labeled data samples). How to reduce the dependence on labeled data samples and use fewer labeled data samples to train an ideal visual algorithm model has become a technical difficulty that needs to be broken through at present. The learning of few sample size aims to learn prior knowledge from existing categories of data, and then use very few labeled data to complete the identification of new categories, breaking the constraints on sample data volume, and has practical value in areas where samples are generally missing, which helps promote the implementation of AI."
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-