检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Zhang, Chong Tu, Yi Zhao, Yixi Yuan, Chenshu Chen, Huan Zhang, Yue Chai, Mingxu Guo, Ya Zhu, Huijia Zhang, Qi Gui, Tao School of Computer Science Fudan University Shanghai China Shanghai Key Laboratory of Intelligent Information Processing Shanghai China Ant Tiansuan Security Lab Ant Group Hangzhou China School of Statistics and Data Science Nankai University Tianjin China Institute of Modern Languages and Linguistics Fudan University Shanghai China

Modeling and leveraging layout reading order in visually-rich documents (VrDs) is critical in document intelligence as it captures the rich structure semantics within documents. Previous works typically formulated layout reading order as a permutation of layout elements, i.e. a sequence containing all the layout elements. However, we argue that this formulation does not adequately convey the complete reading order information in the layout, which may potentially lead to performance decline in downstream VrD tasks. To address this issue, we propose to model the layout reading order as ordering relations over the set of layout elements, which have sufficient expressive capability for the complete reading order information. To enable empirical evaluation on methods towards the improved form of reading order prediction (ROP), we establish a comprehensive benchmark dataset including the reading order annotation as relations over layout elements, together with a relation-extraction-based method that outperforms previous methods. Moreover, to highlight the practical benefits of introducing the improved form of layout reading order, we propose a reading-order-relation-enhancing pipeline to improve model performance on any arbitrary VrD task by introducing additional reading order relation inputs. Comprehensive results demonstrate that the pipeline generally benefits downstream VrD tasks: (1) with utilizing the reading order relation information, the enhanced downstream models achieve SOTA results on both two task settings of the targeted dataset;(2) with utilizing the pseudo reading order information generated by the proposed ROP model, the performance of the enhanced models has improved across all three models and eight cross-domain VrD-IE/QA task settings without targeted optimization. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Wave-SAN: Wavelet based Style Augmentation Network for Cross-Domain Few-Shot Learning

arXiv

引用

arXiv 2022年

作者： Fu, Yuqian Xie, Yu Fu, Yanwei Chen, Jingjing Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China School of Data Science Fudan University China

Previous few-shot learning (FSL) works mostly are limited to natural images of general concepts and categories. These works assume very high visual similarity between the source and target classes. In contrast, the recently proposed crossdomain few-shot learning (CD-FSL) aims at transferring knowledge from general nature images of many labeled examples to novel domain-specific target categories of only a few labeled examples. The key challenge of CD-FSL lies in the huge data shift between source and target domains, which is typically in the form of totally different visual styles. This makes it very nontrivial to directly extend the classical FSL methods to address the CD-FSL task. To this end, this paper studies the problem of CD-FSL by spanning the style distributions of the source dataset. Particularly, wavelet transform is introduced to enable the decomposition of visual representations into low-frequency components such as shape and style and highfrequency components e.g., texture. To make our model robust to visual styles, the source images are augmented by swapping the styles of their low-frequency components with each other. We propose a novel Style Augmentation (StyleAug) module to implement this idea. Furthermore, we present a Self-Supervised Learning (SSL) module to ensure the predictions of style-augmented images are semantically similar to the unchanged ones. This avoids the potential semantic drift problem in exchanging the styles. Extensive experiments on two CD-FSL benchmarks show the effectiveness of our method. Our codes and models will be released. Copyright © 2022, The Authors. All rights reserved.

关键词： Textures

来源：评论

学校读者我要写书评

暂无评论

Hi-EF: Benchmarking Emotion Forecasting in Human-interaction

arXiv

引用

arXiv 2024年

作者： Wang, Haoran Mai, Xinji Tao, Zeng Wang, Yan Yu, Jiawen Zhou, Ziheng Tong, Xuan Yan, Shaoqi Zhao, Qing Gao, Shuyong Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China School of Information Science and Technology Fudan University Shanghai China

Affective Forecasting, a research direction in psychology that predicts individuals' future emotions, is often constrained by numerous external factors like social influence and temporal distance. To address this, we transform Affective Forecasting into a Deep Learning problem by designing an Emotion Forecasting paradigm based on two-party interactions. We propose a novel Emotion Forecasting (EF) task grounded in the theory that an individual's emotions are easily influenced by the emotions or other information conveyed during interactions with another person. To tackle this task, we have developed a specialized dataset, Humaninteraction- based Emotion Forecasting (Hi-EF), which contains 3069 two-party Multilayered-Contextual Interaction Samples (MCIS) with abundant affectiverelevant labels and three modalities. Hi-EF not only demonstrates the feasibility of the EF task but also highlights its potential. Additionally, we propose a methodology that establishes a foundational and referential baseline model for the EF task and extensive experiments are provided. The dataset and code is available HERE. Copyright © 2024, The Authors. All rights reserved.

关键词： Prediction models

来源：评论

学校读者我要写书评

暂无评论

(Image Present) ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection

arXiv

引用

arXiv 2024年

作者： Sun, Zhihao Jiang, Haoran Chen, Haoran Cao, Yixin Qiu, Xipeng Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China School of Computer Science Fudan University China

(Image Present) Multimodal large language models have unlocked new possibilities for various multimodal tasks. However, their potential in image manipulation detection remains unexplored. When directly applied to the IMD task, M-LLMs often produce reasoning texts that suffer from hallucinations and overthinking. To address this, in this work, we propose ForgerySleuth, which leverages M-LLMs to perform comprehensive clue fusion and generate segmentation outputs indicating specific regions that are tampered with. Moreover, we construct the ForgeryAnalysis dataset through the Chain-of-Clues prompt, which includes analysis and reasoning text to upgrade the image manipulation detection task. A data engine is also introduced to build a larger-scale dataset for the pre-training phase. Our extensive experiments demonstrate the effectiveness of ForgeryAnalysis and show that ForgerySleuth significantly outperforms existing methods in generalization, robustness, and explainability. The resource is available at https://***/sunzhihao18/ForgerySleuth. Copyright © 2024, The Authors. All rights reserved.

关键词： Large datasets

来源：评论

学校读者我要写书评

暂无评论

Towards End-to-End Unsupervised Saliency Detection with Self-Supervised Top-Down Context

arXiv

引用

arXiv 2023年

作者： Song, Yicheng Gao, Shuyong Xing, Haozhe Cheng, Yiting Wang, Yan Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Keenon Robotics Co. Ltd. Shanghai China Academy for Engineering & Technology Fudan University Shanghai China

Unsupervised salient object detection aims to detect salient objects without using supervision signals eliminating the tedious task of manually labeling salient objects. To improve training efficiency, end-to-end methods for USOD have been proposed as a promising alternative. However, current solutions rely heavily on noisy handcraft labels and fail to mine rich semantic information from deep features. In this paper, we propose a self-supervised end-to-end salient object detection framework via top-down context. Specifically, motivated by contrastive learning, we exploit the self-localization from the deepest feature to construct the location maps which are then leveraged to learn the most instructive segmentation guidance. Further considering the lack of detailed information in deepest features, we exploit the detail-boosting refiner module to enrich the location labels with details. Moreover, we observe that due to lack of supervision, current unsupervised saliency models tend to detect non-salient objects that are salient in some other samples of corresponding scenarios. To address this widespread issue, we design a novel Unsupervised Non-Salient Suppression (UNSS) method developing the ability to ignore non-salient objects. Extensive experiments on benchmark datasets demonstrate that our method achieves leading performance among the recent end-to-end methods and most of the multi-stage solutions. The code is available. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

QMix: Quality-aware Learning with Mixed Noise for Robust Retinal Disease Diagnosis

引用

IEEE Transactions on Medical Imaging 2025年 PP卷 PP页

作者： Hou, Junlin Xu, Jilan Feng, Rui Chen, Hao Hong Kong University of Science and Technology Department of Computer Science and Engineering Hong Kong Fudan University School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai China HKUST Department of Chemical and Biological Engineering Division of Life Science Hong Kong HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Shenzhen China

Due to the complex nature of medical image acquisition and annotation, medical datasets inevitably contain noise. This adversely affects the robustness and generalization of deep neural networks. Previous noise learning methods mainly considered noise arising from images being mislabeled, i.e., label noise, assuming all mislabeled images were of high quality. However, medical images can also suffer from severe data quality issues, i.e., data noise, where discriminative visual features for disease diagnosis are missing. In this paper, we propose QMix, a noise learning framework that learns a robust disease diagnosis model under mixed noise scenarios. QMix alternates between sample separation and quality-aware semisupervised training in each epoch. The sample separation phase uses a joint uncertainty-loss criterion to effectively separate (1) correctly labeled images, (2) mislabeled high-quality images, and (3) mislabeled low-quality images. The semi-supervised training phase then learns a robust disease diagnosis model from the separated samples. Specifically, we propose a sample-reweighing loss to mitigate the effect of mislabeled low-quality images during training, and a contrastive enhancement loss to further distinguish them from correctly labeled images. QMix achieved state-of-the-art performance on six public retinal image datasets and exhibited significant improvements in robustness against mixed noise. © 1982-2012 IEEE.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

TEMPORALLY COARSE TO FINE SNIPPETS RELATIONSHIP LEARNING WITH GRAPH CONVOLUTION FOR TEMPORAL ACTION PROPOSAL GENERATION

TEMPORALLY COARSE TO FINE SNIPPETS RELATIONSHIP LEARNING WIT...

引用

2021 IEEE International Conference on Multimedia and Expo, ICME 2021

作者： Li, Shuaicheng Zhao, Rui-Wei Miao, Shuyu Feng, Rui Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University China Academy for Engineering and Technology Fudan University China

ISBN: (纸本)9781665438643

Previous works have shown that explicit snippets relationship modeling can be helpful for feature learning on untrimmed action videos. However, the snippets relationship learning in these methods are far from optimal in that they failed to consider the valuable temporally coarse-grained features, learnable soft relationship weights, and separate relationship learning in different temporal orders. To address this issue, we proposed a novel SGC-Block for improved snippet relationship learning, which enables the temporally coarse-to-fine soft valued snippet-wise relationship learning in different temporal directions. The SGC-Block constructs the snippets graph and explicitly models the (1) temporal relations (TPR);(2) coarse-grained snippet-wise relations (CSR);(3) fine-grained snippet-wise relations (FSR);and an additional (4) adaptive relations (ADR). Especially, the novel CSR is inspired by the feature pyramid pooling structure to obtain the coarse feature presentations in the temporal dimension. Experimental results showed that our proposed approach outperforms most state-of-the-art methods on the THUMOS14 and ActivityNet-1.3 benchmarks. © 2021 IEEE

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

Denoising diffusion path: attribution noise reduction with an auxiliary diffusion model 24

Denoising diffusion path: attribution noise reduction with a...

引用

Proceedings of the 38th International Conference on Neural information processing Systems

作者： Yiming Lei Zilong Li Junping Zhang Hongming Shan Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Institute of Science and Technology for Brain-Inspired Intelligence & MOE Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence & MOE Frontiers Center for Brain Science Fudan University

ISBN: (纸本)9798331314385

The explainability of deep neural networks (DNNs) is critical for trust and reliability in AI systems. Path-based attribution methods, such as integrated gradients (IG), aim to explain predictions by accumulating gradients along a path from a baseline to the target image. However, noise accumulated during this process can significantly distort the explanation. While existing methods primarily concentrate on finding alternative paths to circumvent noise, they overlook a critical issue: intermediate-step images frequently diverge from the distribution of training data, further intensifying the impact of noise. This work presents a novel Denoising Diffusion Path (DDPath) to tackle this challenge by harnessing the power of diffusion models for denoising. By exploiting the inherent ability of diffusion models to progressively remove noise from an image, DDPath constructs a piece-wise linear path. Each segment of this path ensures that samples drawn from a Gaussian distribution are centered around the target image. This approach facilitates a gradual reduction of noise along the path. We further demonstrate that DDPath adheres to essential axiomatic properties for attribution methods and can be seamlessly integrated with existing methods such as IG. Extensive experimental results demonstrate that DDPath can significantly reduce noise in the attributions—resulting in clearer explanations—and achieves better quantitative results than traditional path-based methods.

关键词：

来源：评论

学校读者我要写书评

暂无评论

MPN: MULTIMODAL PARALLEL NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION

MPN: MULTIMODAL PARALLEL NETWORK FOR AUDIO-VISUAL EVENT LOCA...

引用

2021 IEEE International Conference on Multimedia and Expo, ICME 2021

作者： Yu, Jiashuo Cheng, Ying Feng, Rui School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University China Academy for Engineering and Technology Fudan University China

ISBN: (纸本)9781665438643

Audio-visual event localization aims to localize an event that is both audible and visible in the wild, which is a widespread audio-visual scene analysis task for unconstrained videos. To address this task, we propose a Multimodal Parallel Network (MPN), which can perceive global semantics and unmixed local information parallelly. Specifically, our MPN framework consists of a classification subnetwork to predict event categories and a localization subnetwork to predict event boundaries. The classification subnetwork is constructed by the Multimodal Co-attention Module (MCM) and obtains global contexts. The localization subnetwork consists of Multimodal Bottleneck Attention Module (MBAM), which is designed to extract fine-grained segment-level contents. Extensive experiments demonstrate that our framework achieves the state-of-the-art performance both in fully supervised and weakly supervised settings on the Audio-Visual Event (AVE) dataset. © 2021 IEEE

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Semi-supervised Medical Image Segmentation with Distribution Calibration and Non-local Semantic Constraint

Semi-supervised Medical Image Segmentation with Distribution...

引用

2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021

作者： Zhang, Nan Hou, Junlin Zhao, Rui-Wei Feng, Rui Zhang, Yuejie Fudan University Academy for Engineering and Technology China Fudan University School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing China

ISBN: (纸本)9781665401265

The available medical images with accurate segmentation masks are usually limited due to the expensive and time-consuming annotation cost. Many semi-supervised approaches tried to exploit a large amount of unlabeled data together with the small number of labeled data. However, their learned segmentation models can easily become overfitted on the biased labeled data, mainly because of the misalignment between the labeled and unlabeled data distribution. To address this challenging problem, we propose a novel semi-supervised model with distribution calibration and non-local semantic constraint for medical image segmentation. In specific, we explicitly calibrate the learned feature distributions of the labeled and unlabeled data to make them aligned. Meanwhile, we add a special nonlocal semantic loss to encourage the learned features to be more discriminative for the segmentation task at the same time. Consequently, our final segmentation networks have the advantage to better generalize on the unlabeled data in both training and test set. Experimental results on three popular medical image segmentation benchmarks demonstrate that our proposed model achieves superior performance over other state-of-the-art methods. We will release our source code in this URL. © 2021 IEEE.

关键词： Calibration

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：