检索结果-内蒙古大学图书馆

KD-Crowd:a knowledge distillation framework for learning from crowds

Frontiers of Computer Science 2025年第1期19卷 119-130页

作者： Shaoyuan LI Yuxiang ZHENG Ye SHI Shengjun HUANG Songcan CHEN College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsMIIT Key Laboratory of Pattern Analysis and Machine IntelligenceNanjing 211106China

Recently, crowdsourcing has established itself as an efficient labeling solution by distributing tasks to crowd workers. As the workers can make mistakes with diverse expertise, one core learning task is to estimate each worker’s expertise, and aggregate over them to infer the latent true labels. In this paper, we show that as one of the major research directions, the noise transition matrix based worker expertise modeling methods commonly overfit the annotation noise, either due to the oversimplified noise assumption or inaccurate estimation. To solve this problem, we propose a knowledge distillation framework (KD-Crowd) by combining the complementary strength of noise-model-free robust learning techniques and transition matrix based worker expertise modeling. The framework consists of two stages: in Stage 1, a noise-model-free robust student model is trained by treating the prediction of a transition matrix based crowdsourcing teacher model as noisy labels, aiming at correcting the teacher’s mistakes and obtaining better true label predictions;in Stage 2, we switch their roles, retraining a better crowdsourcing model using the crowds’ annotations supervised by the refined true label predictions given by Stage 1. Additionally, we propose one f-mutual information gain (MIG^(f)) based knowledge distillation loss, which finds the maximum information intersection between the student’s and teacher’s prediction. We show in experiments that MIG^(f) achieves obvious improvements compared to the regular KL divergence knowledge distillation loss, which tends to force the student to memorize all information of the teacher’s prediction, including its errors. We conduct extensive experiments showing that, as a universal framework, KD-Crowd substantially improves previous crowdsourcing methods on true label prediction and worker expertise estimation.

关键词： crowdsourcing label noise worker expertise knowledge distillation robust learning

来源：评论

学校读者我要写书评

暂无评论

A Deep Model for Partial Multi-label Image Classification with Curriculum-based Disambiguation

引用

machine intelligence Research 2024年第4期21卷 801-814页

作者： Feng Sun Ming-Kun Xie Sheng-Jun Huang MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and TechnologyNanjing University of AeronauticsandAstronauticsNanjing211106China

In this paper,we study the partial multi-label(PML)image classification problem,where each image is annotated with a candidate label set consisting of multiple relevant labels and other noisy *** PML methods typically design a disambiguation strategy to filter out noisy labels by utilizing prior knowledge with extra assumptions,which unfortunately is unavailable in many real ***,because the objective function for disambiguation is usually elaborately designed on the whole training set,it can hardly be optimized in a deep model with stochastic gradient descent(SGD)on *** this paper,for the first time,we propose a deep model for PML to enhance the representation and discrimination *** the one hand,we propose a novel curriculum-based disambiguation strategy to progressively identify ground-truth labels by incorporating the varied difficulties of different *** the other hand,consistency regularization is introduced for model training to balance fitting identified easy labels and exploiting potential relevant *** experimental results on the commonly used benchmark datasets show that the proposed method significantlyoutperforms the SOTA methods.

关键词： Partial multi-label image classification curriculum-based disambiguation consistency regularization label difficulty candidatelabel set.

来源：评论

学校读者我要写书评

暂无评论

Relative difficulty distillation for semantic segmentation

引用

Science China(Information Sciences) 2024年第9期67卷 126-145页

作者： Dong LIANG Yue SUN Yun DU Songcan CHEN Sheng-Jun HUANG MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Shenzhen Research Institute Nanjing University of Aeronautics and Astronautics

Current knowledge distillation(KD) methods primarily focus on transferring various structured knowledge and designing corresponding optimization goals to encourage the student network to imitate the output of the teacher network. However, introducing too many additional optimization objectives may lead to unstable training, such as gradient conflicts. Moreover, these methods ignored the guidelines of relative learning difficulty between the teacher and student networks. Inspired by human cognitive science, in this paper, we redefine knowledge from a new perspective — the student and teacher networks' relative difficulty of samples, and propose a pixel-level KD paradigm for semantic segmentation named relative difficulty distillation(RDD). We propose a two-stage RDD framework: teacher-full evaluated RDD(TFE-RDD) and teacher-student evaluated RDD(TSE-RDD). RDD allows the teacher network to provide effective guidance on learning focus without additional optimization goals, thus avoiding adjusting learning weights for multiple losses. Extensive experimental evaluations using a general distillation loss function on popular datasets such as Cityscapes, Cam Vid, Pascal VOC, and ADE20k demonstrate the effectiveness of RDD against state-ofthe-art KD methods. Additionally, our research showcases that RDD can integrate with existing KD methods to improve their upper performance bound. Codes are available at https://***/sunyueue/***.

关键词： knowledge distillation semantic segmentation relative difficulty sample weighting prediction discrepancy

来源：评论

学校读者我要写书评

暂无评论

Sequential Cooperative Distillation for Imbalanced Multi-Task Learning

引用

Journal of Computer Science & Technology 2024年第5期39卷 1094-1106页

作者： Quan Feng Jia-Yu Yao Ming-Kun Xie Sheng-Jun Huang Song-Can Chen College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing 211106China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing University of Aeronautics and Astronautics Nanjing 211106China

Multi-task learning(MTL)can boost the performance of individual tasks by mutual learning among multiple related ***,when these tasks assume diverse complexities,their corresponding losses involved in the MTL objective inevitably compete with each other and ultimately make the learning biased towards simple tasks rather than complex *** address this imbalanced learning problem,we propose a novel MTL method that can equip multiple existing deep MTL model architectures with a sequential cooperative distillation(SCD)***,we first introduce an efficient mechanism to measure the similarity between tasks,and group similar tasks into the same block to allow their cooperative learning from each *** on this,the grouped task blocks are sorted in a queue to determine the learning sequence of the tasks according to their complexities estimated with the defined performance ***,a distillation between the individual task-specific models and the MTL model is performed block by block from complex to simple manner,achieving a balance between competition and cooperation among learning multiple *** experiments demonstrate that our method is significantly more competitive compared with state-of-the-art methods,ranking No.1 with average performances across multiple datasets by improving 12.95%and 3.72%compared with OMTL and MTLKD,respectively.

关键词： multi-task learning(MIT) imbalanced learning similarity estimation knowledge distillation distillation queue

来源：评论

学校读者我要写书评

暂无评论

Pushing one pair of labels apart each time in multi-label learning: from single positive to full labels

引用

Science China(Information Sciences) 2025年第6期 268-285页

作者： Xiang LI Xinrui WANG Songcan CHEN MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology/College of Artificial Intelligence Nanjing University of Aeronautics and Astronautics

In multi-label learning(MLL), it is extremely challenging to accurately annotate every appearing object due to expensive costs and limited knowledge. When facing such a challenge, a more practical and cheaper alternative should be single positive multi-label learning(SPMLL), where only one positive label needs to be provided per sample. Existing SPMLL methods usually assume unknown labels as negatives, which inevitably introduces false negatives as noisy labels. More seriously, binary cross entropy(BCE) loss is often used for training, which is notoriously not robust to noisy labels. To mitigate this issue, we customize an objective function for SPMLL by pushing only one pair of labels apart each time to suppress the domination of negative labels, which is the main culprit of fitting noisy labels in SPMLL. To further combat such noisy labels, we explore the high-rankness of the label matrix, which can also push apart different labels. By directly extending from SPMLL to MLL with full labels, a unified loss applicable to both settings is derived. As a byproduct, the proposed loss can alleviate the imbalance inherent in MLL. Experiments on real datasets demonstrate that the proposed loss not only performs more robustly to noisy labels for SPMLL but also works well for full labels. Besides, we empirically discover that high-rankness can mitigate the dramatic performance drop in SPMLL. Most surprisingly, even without any regularization or fine-tuned label correction, only adopting our loss defeats state-of-the-art SPMLL methods on CUB, a dataset that severely lacks labels.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Learning multi-tasks with inconsistent labels by using auxiliary big task

引用

Frontiers of Computer Science 2023年第5期17卷 119-132页

作者： Quan FENG Songcan CHEN College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing 211106China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing University of Aeronautics and AstronauticsNanjing 211106China

Multi-task learning is to improve the performance of the model by transferring and exploiting common knowledge among *** MTL works mainly focus on the scenario where label sets among multiple tasks(MTs)are usually the same,thus they can be utilized for learning across the ***,the real world has more general scenarios in which each task has only a small number of training samples and their label sets are just partially overlapped or even *** such MTs is more challenging because of less correlation information available among these *** this,we propose a framework to learn these tasks by jointly leveraging both abundant information from a learnt auxiliary big task with sufficiently many classes to cover those of all these tasks and the information shared among those partiallyoverlapped *** our implementation of using the same neural network architecture of the learnt auxiliary task to learn individual tasks,the key idea is to utilize available label information to adaptively prune the hidden layer neurons of the auxiliary network to construct corresponding network for each task,while accompanying a joint learning across individual *** experimental results demonstrate that our proposed method is significantly competitive compared to state-of-the-art methods.

关键词： multi-task learning inconsistent labels auxiliary task

来源：评论

学校读者我要写书评

暂无评论

Multiband decomposition and spectral discriminative analysis for motor imagery BCI via deep neural network

引用

Frontiers of Computer Science 2022年第5期16卷 71-83页

作者： Pengpai WANG Mingliang WANG Yueying ZHOU Ziming XU Daoqiang ZHANG College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsMIIT Key Laboratory of Pattern Analysis and Machine IntelligenceNanjing 211106China

Human limb movement imagery,which can be used in limb neural disorders rehabilitation and brain-controlled external devices,has become a significant control paradigm in the domain of brain-computer interface(BCI).Although numerous pioneering studies have been devoted to motor imagery classification based on electroencephalography(EEG)signal,their performance is somewhat limited due to insufficient analysis of key effective frequency bands of EEG *** this paper,we propose a model of multiband decomposition and spectral discriminative analysis for motor imagery classification,which is called variational sample-long short term memory(VS-LSTM)***,we first use a channel fusion operator to reduce the signal channels of the raw EEG ***,we use the variational mode decomposition(VMD)model to decompose the EEG signal into six band-limited intrinsic mode functions(BIMFs)for further signal noise *** order to select discriminative frequency bands,we calculate the sample entropy(SampEn)value of each frequency band and select the maximum ***,to predict the classification of motor imagery,a LSTM model is used to predict the class of frequency band with the largest SampEn *** open-access public data is used to evaluated the effectiveness of the proposed *** the data,15 subjects performed motor imagery tasks with elbow flexion/extension,forearm supination/pronation and hand open/close of right upper *** experiment results show that the average classification result of seven kinds of motor imagery was 76.2%,the average accuracy of motor imagery binary classification is 96.6%(imagery ***),respectively,which outperforms the state-of-the-art deep learning-based *** framework significantly improves the accuracy of motor imagery by selecting effective frequency *** research is very meaningful for BCIs,and it is inspiring for end-to-end learning research.

关键词： brain computer interface EEG long short-term memory VMD sample entropy motor imagery

来源：评论

学校读者我要写书评

暂无评论

An Improved Algorithm for Spiking Neural Networks with Multi-Scale Attention Coding

An Improved Algorithm for Spiking Neural Networks with Multi...

引用

2024 International Conference on Cyber-Physical Social intelligence, ICCSI 2024

作者： Chen, Sisi Chen, Xiaofeng Li, Weikai Chongqing Jiaotong University Department of Mathematics Chongqing China Miit Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

ISBN: (纸本)9798350376739

Spiking Neural Networks (SNNs), driven by spike-based mechanisms, are known for their high efficiency and low energy consumption, which makes them ideal for applications like image classification, object detection, and medical diagnostics. However, challenges include training difficulties due to the non-differentiability of spiking activities and limitations in encoding schemes, which struggle to capture complex spatio-temporal dynamics. In this paper, we propose a novel algorithm, termed the Spatio-Temporal Backpropagation algorithm with Multi-Scale Attention Coding (STBP-MSC). STBP-MSC dynamically allocates attention across spatial and temporal domains, which improves coding efficiency and enhances the classification accuracy of SNNs. Furthermore, STBP-MSC incorporates the Complementary Leaky Integrate-and-Fire (CLIF) neuron model, which enhances temporal gradient computation. Experimental results on the MNIST and Fashion-MNIST datasets demonstrate that STBP-MSC achieves high coding efficiency and superior performance within minimal time windows. © 2024 IEEE.

关键词： Network coding

来源：评论

学校读者我要写书评

暂无评论

Robust AUC maximization for classification with pairwise confidence comparisons

引用

Frontiers of Computer Science 2024年第4期18卷 73-83页

作者： Haochen SHI Mingkun XIE Shengjun HUANG College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing 211106China College of Electronic and Information Engineering Nanjing University of Aeronautics and AstronauticsNanjing 211106China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing 211106China

Supervised learning often requires a large number of labeled examples,which has become a critical bottleneck in the case that manual annotating the class labels is *** mitigate this issue,a new framework called pairwise comparison(Pcomp)classification is proposed to allow training examples only weakly annotated with pairwise comparison,i.e.,which one of two examples is more likely to be *** previous study solves Pcomp problems by minimizing the classification error,which may lead to less robust model due to its sensitivity to class *** this paper,we propose a robust learning framework for Pcomp data along with a pairwise surrogate loss called *** provides an unbiased estimator to equivalently maximize AUC without accessing the precise class ***,we prove the consistency with respect to AUC and further provide the estimation error bound for the proposed *** studies on multiple datasets validate the effectiveness of the proposed method.

关键词： method pairwise weakly

来源：评论

学校读者我要写书评

暂无评论

Class-aware Learning for Imbalanced Multi-Label Classification 5

Class-aware Learning for Imbalanced Multi-Label Classificati...

引用

5th IEEE International Conference on Civil Aviation Safety and Information Technology, ICCASIT 2023

作者： Chen, Jiayao Li, Shaoyuan Nanjing University of Aeronautics and Astronautics Miit Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

ISBN: (纸本)9798350310603

Imbalanced multi-label image classification has gained increasing attention recently, in which each sample has multiple class labels, but the number of each category is unevenly distributed. It's common in practical applications but traditional multi-label learning methods can hardly deal with imbalance problems. In this paper, we propose an effective method to tackle imbalanced multi-label learning. The class-aware embedding network is proposed to learn robust class-based representation. Additionally, by using the distribution-balanced loss to weigh different samples, our model can improve the feature learning ability of minority classes. Extensive experiments on widely used long-tailed manual multi-label datasets like VOC-LT and COCO-LT explicitly validate the proposed good method. © 2023 IEEE.

关键词： Classification (of information)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：