检索结果-内蒙古大学图书馆

Denoising diffusion path: attribution noise reduction with an auxiliary diffusion model 24

学校读者我要写书评

暂无评论

Denoising diffusion path: attribution noise reduction with a...

Proceedings of the 38th International Conference on Neural information processing Systems

作者： Yiming Lei Zilong Li Junping Zhang Hongming Shan Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Institute of Science and Technology for Brain-Inspired Intelligence & MOE Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence & MOE Frontiers Center for Brain Science Fudan University

ISBN: (纸本)9798331314385

The explainability of deep neural networks (DNNs) is critical for trust and reliability in AI systems. Path-based attribution methods, such as integrated gradients (IG), aim to explain predictions by accumulating gradients along a path from a baseline to the target image. However, noise accumulated during this process can significantly distort the explanation. While existing methods primarily concentrate on finding alternative paths to circumvent noise, they overlook a critical issue: intermediate-step images frequently diverge from the distribution of training data, further intensifying the impact of noise. This work presents a novel Denoising Diffusion Path (DDPath) to tackle this challenge by harnessing the power of diffusion models for denoising. By exploiting the inherent ability of diffusion models to progressively remove noise from an image, DDPath constructs a piece-wise linear path. Each segment of this path ensures that samples drawn from a Gaussian distribution are centered around the target image. This approach facilitates a gradual reduction of noise along the path. We further demonstrate that DDPath adheres to essential axiomatic properties for attribution methods and can be seamlessly integrated with existing methods such as IG. Extensive experimental results demonstrate that DDPath can significantly reduce noise in the attributions—resulting in clearer explanations—and achieves better quantitative results than traditional path-based methods.

关键词：

Adaptive Split-Fusion Transformer

学校读者我要写书评

暂无评论

Adaptive Split-Fusion Transformer

IEEE International Conference on Multimedia and Expo (ICME)

作者： Zixuan Su Jingjing Chen Lei Pang Chong-Wah Ngo Yu-Gang Jiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing City University of Hong Kong Singapore Management University

Neural networks for visual content understanding have recently evolved from convolutional ones to transformers. The prior (CNN) relies on small-windowed kernels to capture the regional clues, demonstrating solid local expressiveness. On the contrary, the latter (transformer) establishes long-range global connections between localities for holistic learning. Inspired by this complementary nature, there is a growing interest in designing hybrid models which utilize both techniques. Current hybrids merely replace convolutions as simple approximations of linear projection or juxtapose a convolution branch with attention without considering the importance of local/global modeling. To tackle this, we propose a new hybrid named Adaptive Split-Fusion Transformer (ASF-former) that treats convolutional and attention branches differently with adaptive weights. Specifically, an ASF-former encoder equally splits feature channels into half to fit dual-path inputs. Then, the outputs of the dual-path are fused with weights calculated from visual cues. We also design a compact convolutional path from a concern of efficiency. Extensive experiments on standard benchmarks show that our ASF-former outperforms its CNN, transformer, and hybrid counterparts in terms of accuracy (83.9% on ImageNet-1K), under similar conditions (12.9G MACs / 56.7M Params, without large-scale pre-training). The code is available at: https://***/szx503045266/ASF-former.

关键词：

A Holistically Point-guided Text Framework for Weakly-Supervised Camouflaged Object Detection

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Mok, Tsui Qin Gao, Shuyong Xing, Haozhe He, Miaoyang Wang, Yan Zhang, Wenqiang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China Academy for Engineering & Technology The Yiwu Research Institute of Fudan University Chengbei Road Zhejiang Yiwu City322000 China

Weakly-Supervised Camouflaged Object Detection (WSCOD) has gained popularity for its promise to train models with weak labels to segment objects that visually blend into their surroundings. Recently, some methods using sparsely-annotated supervision shown promising results through scribbling in WSCOD, while point-text supervision remains under-explored. Hence, this paper introduces a novel holistically point-guided text framework for WSCOD by decomposing into three phases: segment, choose, train. Specifically, we propose Point-guided Candidate Generation (PCG), where the point’s foreground serves as a correction for the text path to explicitly correct and rejuvenate the loss detection object during the mask generation process (SEGMENT). We also introduce a Qualified Candidate Discriminator (QCD) to choose the optimal mask from a given text prompt using CLIP (CHOOSE), and employ the chosen pseudo mask for training with a self-supervised Vision Transformer (TRAIN). Additionally, we developed a new point-supervised dataset (P2C-COD) and a text-supervised dataset (T-COD). Comprehensive experiments on four benchmark datasets demonstrate our method outperforms state-of-the-art methods by a large margin, and also outperforms some existing fully-supervised camouflaged object detection methods. Copyright © 2025, The Authors. All rights reserved.

关键词： Object detection

Towards End-to-End Unsupervised Saliency Detection with Self-Supervised Top-Down Context

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Song, Yicheng Gao, Shuyong Xing, Haozhe Cheng, Yiting Wang, Yan Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Keenon Robotics Co. Ltd. Shanghai China Academy for Engineering & Technology Fudan University Shanghai China

Unsupervised salient object detection aims to detect salient objects without using supervision signals eliminating the tedious task of manually labeling salient objects. To improve training efficiency, end-to-end methods for USOD have been proposed as a promising alternative. However, current solutions rely heavily on noisy handcraft labels and fail to mine rich semantic information from deep features. In this paper, we propose a self-supervised end-to-end salient object detection framework via top-down context. Specifically, motivated by contrastive learning, we exploit the self-localization from the deepest feature to construct the location maps which are then leveraged to learn the most instructive segmentation guidance. Further considering the lack of detailed information in deepest features, we exploit the detail-boosting refiner module to enrich the location labels with details. Moreover, we observe that due to lack of supervision, current unsupervised saliency models tend to detect non-salient objects that are salient in some other samples of corresponding scenarios. To address this widespread issue, we design a novel Unsupervised Non-Salient Suppression (UNSS) method developing the ability to ignore non-salient objects. Extensive experiments on benchmark datasets demonstrate that our method achieves leading performance among the recent end-to-end methods and most of the multi-stage solutions. The code is available. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

D2SP: Dynamic Dual-Stage Purification Framework for Dual Noise Mitigation in Vision-based Affective Recognition.

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Haoran Mai, Xinji Tao, Zeng Tong, Xuan Lin, Junxiong Wang, Yan Yu, Jiawen Wang, Boyang Yan, Shaoqi Zhao, Qing Zhou, Ziheng Gao, Shuyong Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China School of Information Science and Technology Fudan University Shanghai China Fudan University Shanghai China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

The contemporary state-of-the-art of Dynamic Facial Expression Recognition (DFER) technology facilitates remarkable progress by deriving emotional mappings of facial expressions from video content, underpinned by training on voluminous datasets. Yet, the DFER datasets encompass a substantial volume of noise data. Noise arises from low-quality captures that defy logical labeling, and instances that suffer from mislabeling due to annotation bias, engendering two principal types of uncertainty: the uncertainty regarding data usability and the uncertainty concerning label reliability. Addressing the two types of uncertainty, we have meticulously crafted a two-stage framework aiming at Seeking Certain data In extensive Uncertain data (SCIU). This initiative aims to purge the DFER datasets of these uncertainties, thereby ensuring that only clean, verified data is employed in training processes. To mitigate the issue of low-quality samples, we introduce the Coarse-Grained Pruning (CGP) stage, which assesses sample weights and prunes those deemed unusable due to their low weight. For samples with incorrect annotations, the Fine-Grained Correction (FGC) stage evaluates prediction stability to rectify mislabeled data. Moreover, SCIU is conceived as a universally compatible, plug-and-play framework, tailored to integrate seamlessly with prevailing DFER methodologies. Rigorous experiments across prevalent DFER datasets and against numerous benchmark methods substantiates SCIU’s capacity to markedly elevate performance metrics. Copyright © 2024, The Authors. All rights reserved.

关键词： Face recognition

Semi-Supervised Self-Learning Enhanced Music Emotion Recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Yifu Zhang, Xulong Zhou, Monan Li, Wei School of Computer Science and Technology Fudan University Shanghai China Ping An Technology Co. Ltd. Shenzhen China Department of Music AI and Information Technology Central Conservatory of Music Beijing China Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China

Music emotion recognition (MER) aims to identify the emotions conveyed in a given musical piece. However, currently, in the field of MER, the available public datasets have limited sample sizes. Recently, segment-based methods for emotion-related tasks have been proposed, which train backbone networks on shorter segments instead of entire audio clips, thereby naturally augmenting training samples without requiring additional resources. Then, the predicted segment-level results are aggregated to obtain the entire song prediction. The most commonly used method is that the segment inherits the label of the clip containing it, but music emotion is not constant during the whole clip. Doing so will introduce label noise and make the training easy to overfit. To handle the noisy label issue, we propose a semi-supervised self-learning (SSSL) method, which can differentiate between samples with correct and incorrect labels in a self-learning manner, thus effectively utilizing the augmented segment-level data. Experiments on three public emotional datasets demonstrate that the proposed method can achieve better or comparable performance. © 2024, CC BY.

关键词： Semi-supervised learning

StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fu, Yuqian Xie, Yu Fu, Yanwei Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Purple Mountain Laboratories Nanjing China School of Data Science Fudan University China

Cross-Domain Few-Shot Learning (CD-FSL) is a recently emerging task that tackles few-shot learning across different domains. It aims at transferring prior knowledge learned on the source dataset to novel target datasets. The CD-FSL task is especially challenged by the huge domain gap between different datasets. Critically, such a domain gap actually comes from the changes of visual styles, and wave-SAN [10] empirically shows that spanning the style distribution of the source data helps alleviate this issue. However, wave-SAN simply swaps styles of two images. Such a vanilla operation makes the generated styles "real" and "easy", which still fall into the original set of the source styles. Thus, inspired by vanilla adversarial learning, a novel model-agnostic meta Style Adversarial training (StyleAdv) method together with a novel style adversarial attack method is proposed for CD-FSL. Particularly, our style attack method synthesizes both "virtual" and "hard" adversarial styles for model training. This is achieved by perturbing the original style with the signed style gradients. By continually attacking styles and forcing the model to recognize these challenging adversarial styles, our model is gradually robust to the visual styles, thus boosting the generalization ability for novel target datasets. Besides the typical CNN-based backbone, we also employ our StyleAdv method on large-scale pretrained vision transformer. Extensive experiments conducted on eight various target datasets show the effectiveness of our method. Whether built upon ResNet or ViT, we achieve the new state of the art for CD-FSL. Code is available at https://***/lovelyqian/StyleAdv-CDFSL. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

QMix: Quality-aware Learning with Mixed Noise for Robust Retinal Disease Diagnosis

学校读者我要写书评

暂无评论

IEEE Transactions on Medical Imaging 2025年 PP卷 PP页

作者： Hou, Junlin Xu, Jilan Feng, Rui Chen, Hao Hong Kong University of Science and Technology Department of Computer Science and Engineering Hong Kong Fudan University School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai China HKUST Department of Chemical and Biological Engineering Division of Life Science Hong Kong HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Shenzhen China

Due to the complex nature of medical image acquisition and annotation, medical datasets inevitably contain noise. This adversely affects the robustness and generalization of deep neural networks. Previous noise learning methods mainly considered noise arising from images being mislabeled, i.e., label noise, assuming all mislabeled images were of high quality. However, medical images can also suffer from severe data quality issues, i.e., data noise, where discriminative visual features for disease diagnosis are missing. In this paper, we propose QMix, a noise learning framework that learns a robust disease diagnosis model under mixed noise scenarios. QMix alternates between sample separation and quality-aware semisupervised training in each epoch. The sample separation phase uses a joint uncertainty-loss criterion to effectively separate (1) correctly labeled images, (2) mislabeled high-quality images, and (3) mislabeled low-quality images. The semi-supervised training phase then learns a robust disease diagnosis model from the separated samples. Specifically, we propose a sample-reweighing loss to mitigate the effect of mislabeled low-quality images during training, and a contrastive enhancement loss to further distinguish them from correctly labeled images. QMix achieved state-of-the-art performance on six public retinal image datasets and exhibited significant improvements in robustness against mixed noise. © 1982-2012 IEEE.

关键词： Self-supervised learning

FLIP-80M: 80 Million Visual-Linguistic Pairs for Facial Language-Image Pre-Training 24

学校读者我要写书评

暂无评论

FLIP-80M: 80 Million Visual-Linguistic Pairs for Facial Lang...

32nd ACM International Conference on Multimedia, MM 2024

作者： Li, Yudong Hou, Xianxu Dezhi, Zheng Shen, Linlin Zhao, Zhe School of Computer Science and Software Engineering Shenzhen University Shenzhen China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China School of AI and Advanced Computing Xi'an Jiaotong-Liverpool University Shenzhen China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen China Tencent AI Lab Beijing China

ISBN: (纸本)9798400706868

While significant progress has been made in multi-modal learning driven by large-scale image-text datasets, there is still a noticeable gap in the availability of such datasets within the facial domain. To facilitate and advance the field of facial representation learning, we present FLIP-80M, a large-scale visual-linguistic dataset comprising over 80 million face images paired with text descriptions. FLIP-80M is constructed by leveraging the large openly available image-text-pair dataset LAION-5B and a mixed-method approach to filter face-related pairs from both visual and linguistic perspectives. Our curation process involves face detection, face caption classification, text de-noising, and synthesis-based image augmentation. As a result, FLIP-80M stands as the largest face-text dataset to date. To evaluate the potential of our dataset, we fine-tune the CLIP model using the proposed FLIP-80M, to create FLIP (Facial Language-Image Pretraining) and assess its representation capabilities across various downstream tasks. Our experiments demonstrate that our FLIP model achieves state-of-the-art results in a range of face analysis tasks, including face parsing, face alignment, and face attribute classification. The dataset and models are available at https://***/ydli-ai/FLIP. © 2024 ACM.

关键词： Face recognition