Recently, crowdsourcing has established itself as an efficient labeling solution by distributing tasks to crowd workers. As the workers can make mistakes with diverse expertise, one core learning task is to estimate e...
详细信息
Recently, crowdsourcing has established itself as an efficient labeling solution by distributing tasks to crowd workers. As the workers can make mistakes with diverse expertise, one core learning task is to estimate each worker’s expertise, and aggregate over them to infer the latent true labels. In this paper, we show that as one of the major research directions, the noise transition matrix based worker expertise modeling methods commonly overfit the annotation noise, either due to the oversimplified noise assumption or inaccurate estimation. To solve this problem, we propose a knowledge distillation framework (KD-Crowd) by combining the complementary strength of noise-model-free robust learning techniques and transition matrix based worker expertise modeling. The framework consists of two stages: in Stage 1, a noise-model-free robust student model is trained by treating the prediction of a transition matrix based crowdsourcing teacher model as noisy labels, aiming at correcting the teacher’s mistakes and obtaining better true label predictions;in Stage 2, we switch their roles, retraining a better crowdsourcing model using the crowds’ annotations supervised by the refined true label predictions given by Stage 1. Additionally, we propose one f-mutual information gain (MIG^(f)) based knowledge distillation loss, which finds the maximum information intersection between the student’s and teacher’s prediction. We show in experiments that MIG^(f) achieves obvious improvements compared to the regular KL divergence knowledge distillation loss, which tends to force the student to memorize all information of the teacher’s prediction, including its errors. We conduct extensive experiments showing that, as a universal framework, KD-Crowd substantially improves previous crowdsourcing methods on true label prediction and worker expertise estimation.
In multi-label learning (MLL), it is extremely challenging to accurately annotate every appearing object due to expensive costs and limited knowledge. When facing such a challenge, a more practical and cheaper alterna...
In multi-label learning (MLL), it is extremely challenging to accurately annotate every appearing object due to expensive costs and limited knowledge. When facing such a challenge, a more practical and cheaper alternative should be single positive multi-label learning (SPMLL), where only one positive label needs to be provided per sample. Existing SPMLL methods usually assume unknown labels as negatives, which inevitably introduces false negatives as noisy labels. More seriously, binary cross entropy (BCE) loss is often used for training, which is notoriously not robust to noisy labels. To mitigate this issue, we customize an objective function for SPMLL by pushing only one pair of labels apart each time to suppress the domination of negative labels, which is the main culprit of fitting noisy labels in SPMLL. To further combat such noisy labels, we explore the high-rankness of the label matrix, which can also push apart different labels. By directly extending from SPMLL to MLL with full labels, a unified loss applicable to both settings is derived. As a byproduct, the proposed loss can alleviate the imbalance inherent in MLL. Experiments on real datasets demonstrate that the proposed loss not only performs more robustly to noisy labels for SPMLL but also works well for full labels. Besides, we empirically discover that high-rankness can mitigate the dramatic performance drop in SPMLL. Most surprisingly, even without any regularization or fine-tuned label correction, only adopting our loss defeats state-of-the-art SPMLL methods on CUB, a dataset that severely lacks labels.
Unsupervised Domain Adaptation(UDA)intends to achieve excellent results by transferring knowledge from labeled source domains to unlabeled target domains in which the data or label distribution *** UDA methods have ac...
详细信息
Unsupervised Domain Adaptation(UDA)intends to achieve excellent results by transferring knowledge from labeled source domains to unlabeled target domains in which the data or label distribution *** UDA methods have acquired great success when labels in the source domain are ***,even the acquisition of scare clean labels in the source domain needs plenty of costs as *** the presence of label noise in the source domain,the traditional UDA methods will be seriously degraded as they do not deal with the label *** this paper,we propose an approach named Robust Self-training with Label Refinement(RSLR)to address the above *** adopts the self-training framework by maintaining a Labeling Network(LNet)on the source domain,which is used to provide confident pseudo-labels to target samples,and a Target-specific Network(TNet)trained by using the pseudo-labeled *** combat the effect of label noise,LNet progressively distinguishes and refines the mislabeled source *** combination with class rebalancing to combat the label distribution shift issue,RSLR achieves effective performance on extensive benchmark datasets.
Active learning (AL) is an effective method to balance annotation costs and model performance under resource-constrained circumstances. Most existing AL studies are typically designed for class-balanced datasets. Howe...
详细信息
ISBN:
(数字)9798350368741
ISBN:
(纸本)9798350368758
Active learning (AL) is an effective method to balance annotation costs and model performance under resource-constrained circumstances. Most existing AL studies are typically designed for class-balanced datasets. However, the ubiquity of long-tailed distributions in real-world scenarios largely restricts the applicability of those AL methods. To tackle this problem, we propose a new active learning framework, namely long-tailed active learning (LTAL). The LTAL framework divides the long-tailed dataset into constantly evolving in-distribution (ID) and out-of-distribution (OOD) samples, and views the tail samples as OOD samples distinct from the head ones, thus intuitively converting the LTA problem into an iterative OOD detection task. We leverage an energy-based OOD detection approach with a well-designed class-imbalanced energy regularization loss to further extend the energy gap between head and tail classes, encouraging the model to select more unlabeled tail samples with higher free energy values. Experimental results show that despite its conceptual simplicity, the proposed method significantly outperforms competitive baselines.
Evaluating the performance of low-light image enhancement (LLE) is highly subjective, thus making integrating human preferences into LLE a necessity. Existing methods fail to consider this and present a series of pote...
详细信息
We address the challenge of offline reinforcement learning using realistic data, specifically non-expert data collected through sub-optimal behavior policies. Under such circumstance, the learned policy must be safe e...
Recent studies have verified that semi-supervised learning (SSL) is vulnerable to data poisoning backdoor attacks. Even a tiny fraction of contaminated training data is sufficient for adversaries to manipulate up to 9...
详细信息
Diffusion-based models have shown great promise in real-world image super-resolution (Real-ISR), but often generate content with structural errors and spurious texture details due to the empirical priors and illusions...
详细信息
Visual Commonsense Reasoning, which is regarded as one challenging task to pursue advanced visual scene comprehension, has been used to diagnose the reasoning ability of AI systems. However, reliable reasoning require...
详细信息
Current eye-gaze interaction technologies for smartphones are considered inflexible, inaccurate, and power-hungry. These methods typically rely on hand involvement and accomplish partial interactions. In this paper, w...
详细信息
暂无评论