检索结果-内蒙古大学图书馆

Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective 29

学校读者我要写书评

暂无评论

Decorrelate Irrelevant, Purify Relevant: Overcome Textual Sp...

29th International Conference on Computational Linguistics, COLING 2022

作者： Dou, Shihan Zheng, Rui Wu, Ting Gao, Songyang Shan, Junjie Zhang, Qi Wu, Yueming Huang, Xuanjing School of Computer Science Fudan University Shanghai China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China KTH Royal Institute of Technology Stockholm Sweden Nanyang Technological University Singapore

Natural language understanding (NLU) models tend to rely on spurious correlations (i.e., dataset bias) to achieve high performance on in-distribution datasets but poor performance on out-of-distribution ones. Most of the existing debiasing methods often identify and weaken these samples with biased features (i.e., superficial surface features that cause such spurious correlations). However, down-weighting these samples obstructs the model in learning from the non-biased parts of these samples. To tackle this challenge, in this paper, we propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective. Specifically, we introduce Random Fourier Features and weighted re-sampling to decorrelate the dependencies between features to mitigate spurious correlations. After obtaining decorrelated features, we further design a mutual-information-based method to purify them, which forces the model to learn features that are more relevant to tasks. Extensive experiments on two well-studied NLU tasks demonstrate that our method is superior to other comparative approaches. © 2022 Proceedings - International Conference on Computational Linguistics, COLING. All rights reserved.

关键词：

Question Answering Systems Based on Pre-trained Language Models: Recent Progress 13th

学校读者我要写书评

暂无评论

Question Answering Systems Based on Pre-trained Language Mo...

13th IFIP TC 12 International Conference on intelligent information processing, IIP 2024

作者： Luo, Xudong Luo, Ying Yang, Binxia School of Computer Science and Engineering Guangxi Normal University Guilin541004 China Key Lab of Education Blockchain and Intelligent Technology of Ministry of Education Guilin541004 China Guangxi Key Lab of Multi-Source Information Mining and Security Guilin541004 China

ISBN: (纸本)9783031578076

Although Pre-trained Language Model (PLM) ChatGPT as a Question-Answering System (QAS) is so successful, it is still necessary to study further the QASs based on PLMs. In this paper, we survey state-of-the-art systems of this kind, identify the issues that current researchers are concerned about, explore various PLM-based methods for addressing them, and compare their pros and cons. We also discuss the datasets used for fine-tuning the corresponding PLMs and evaluating these PLM-based methods. Moreover, we summarise the criteria for evaluating these methods and compare their performance against these criteria. Finally, based on our analysis of the state-of-the-art PLM-based methods for QA, we identify some challenges for future research. © IFIP International Federation for information processing 2024.

关键词： Computational linguistics

SEMANTIC LATENT DECOMPOSITION WITH NORMALIZING FLOWS FOR FACE EDITING

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Binglei Huang, Zhizhong Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

Navigating in the latent space of StyleGAN has shown effectiveness for face editing. However, the resulting methods usually encounter challenges in complicated navigation due to the entanglement among different attributes in the latent space. To address this issue, this paper proposes a novel framework, termed SDFlow, with a semantic decomposition in original latent space using continuous conditional normalizing flows. Specifically, SDFlow decomposes the original latent code into different irrelevant variables by jointly optimizing two components: (i) a semantic encoder to estimate semantic variables from input faces and (ii) a flow-based transformation module to map the latent code into a semantic-irrelevant variable in Gaussian distribution, conditioned on the learned semantic variables. To eliminate the entanglement between variables, we employ a disentangled learning strategy under a mutual information framework, thereby providing precise manipulation controls. Experimental results demonstrate that SDFlow outperforms existing state-of-the-art face editing methods both qualitatively and quantitatively. The source code is made available at https://***/phil329/SDFlow. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

OneVOS: Unifying Video Object Segmentation with All-in-One Transformer Framework

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Wanyun Guo, Pinxue Zhou, Xinyu Hong, Lingyi He, Yangji Zheng, Xiangyu Zhang, Wei Zhang, Wenqiang School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing China Academy for Engineering and Technology Fudan University China

Contemporary Video Object Segmentation (VOS) approaches typically consist stages of feature extraction, matching, memory management, and multiple objects aggregation. Recent advanced models either employ a discrete modeling for these components in a sequential manner, or optimize a combined pipeline through substructure aggregation. However, these existing explicit staged approaches prevent the VOS framework from being optimized as a unified whole, leading to the limited capacity and suboptimal performance in tackling complex videos. In this paper, we propose OneVOS, a novel framework that unifies the core components of VOS with All-in-One Transformer. Specifically, to unify all aforementioned modules into a vision transformer, we model all the features of frames, masks and memory for multiple objects as transformer tokens, and integrally accomplish feature extraction, matching and memory management of multiple objects through the flexible attention mechanism. Furthermore, a Unidirectional Hybrid Attention is proposed through a double decoupling of the original attention operation, to rectify semantic errors and ambiguities of stored tokens in OneVOS framework. Finally, to alleviate the storage burden and expedite inference, we propose the Dynamic Token Selector, which unveils the working mechanism of OneVOS and naturally leads to a more efficient version of OneVOS. Extensive experiments demonstrate the superiority of OneVOS, achieving state-of-the-art performance across 7 datasets, particularly excelling in complex LVOS and MOSE datasets with 70.1% and 66.4% J&F scores, surpassing previous state-of-the-art methods by 4.2% and 7.0%, respectively. And our code will be available for reproducibility and further research. Copyright © 2024, The Authors. All rights reserved.

关键词： Feature extraction

M3-20M: A Large-Scale Multi-Modal Molecule Dataset for AI-driven Drug Design and Discovery

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guo, Siyuan Wang, Lexuan Jin, Chang Wang, Jinxian Peng, Han Shi, Huayang Li, Wengen Guan, Jihong Zhou, Shuigeng Department of Computer Science and Technology Tongji University No. 4800 Cao’an Road Shanghai201804 China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University 2005 Songhu Road Shanghai200438 China

This paper introduces M3-20M, a large-scale Multi-Modal Molecule dataset that contains over 20 million molecules, with the data mainly being integrated from existing databases and partially generated by large language models. Designed to support AI-driven drug design and discovery, M3-20M is 71 times more in the number of molecules than the largest existing dataset, providing an unprecedented scale that can highly benefit the training or fine-tuning of models, including large language models for drug design and discovery tasks. This dataset integrates one-dimensional SMILES, two-dimensional molecular graphs, three-dimensional molecular structures, physicochemical properties, and textual descriptions collected through web crawling and generated using GPT-3.5, offering a comprehensive view of each molecule. To demonstrate the power of M3-20M in drug design and discovery, we conduct extensive experiments on two key tasks: molecule generation and molecular property prediction, using large language models including GLM4, GPT-3.5, GPT-4, and Llama3-8b. Our experimental results show that M3-20M can significantly boost model performance in both tasks. Specifically, it enables the models to generate more diverse and valid molecular structures and achieve higher property prediction accuracy than existing single-modal datasets, which validates the value and potential of M3-20M in supporting AI-driven drug design and discovery. The dataset is available at https://***/bz99bz/M-3. Copyright © 2024, The Authors. All rights reserved.

关键词： Web crawler

CROSS-HEAD SUPERVISION FOR CROWD COUNTING WITH NOISY ANNOTATIONS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dai, Mingliang Huang, Zhizhong Gao, Jiaqi Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

Noisy annotations such as missing annotations and location shifts often exist in crowd counting datasets due to multi-scale head sizes, high occlusion, etc. These noisy annotations severely affect the model training, especially for density map-based methods. To alleviate the negative impact of noisy annotations, we propose a novel crowd counting model with one convolution head and one transformer head, in which these two heads can supervise each other in noisy areas, called Cross-Head Supervision. The resultant model, CHS-Net, can synergize different types of inductive biases for better counting. In addition, we develop a progressive cross-head supervision learning strategy to stabilize the training process and provide more reliable supervision. Extensive experimental results on shanghaiTech and QNRF datasets demonstrate superior performance over state-of-the-art methods. Code is available at https://***/RaccoonDML/CHSNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Trusted data access and authorization protocol

学校读者我要写书评

暂无评论

The Journal of China Universities of Posts and Telecommunications 2021年第5期28卷 18-26页

作者： Song Wenpeng Li Ming Liu Baixiang Shanghai Key Laboratory of Intelligent Information Processing Fudan UniversityShanghai 200433China School of Computer Science Fudan UniversityShanghai 200433China China Electronics Standardization Institute Beijing 100007China

Threshold proxy re-encryption(PRE) authorizes the data access right of data subject to multiple proxies, who authorize the right again to delegatee to accomplish the end-to-end data encryption process from storage to authorization. Based on threshold PRE algorithm, in order to build a complete trusted data storage and authorization system, the four protocols, which are data access protocol, authorization proxy protocol, authorization proxy cancellation protocol and data reading authorization protocol, are defined completely. On that basis, an efficient data searching method is constructed by specifying the data delegatee. At last, to ensure the right to know of data, the audit log is processed with trusted data right confirmation based on distributed ledger technology. Meanwhile, a parallel data right confirmation processing method is defined based on hierarchical derivation algorithm of public and private key. In the end, the performance evaluation analysis of the protocol are given. Trusted data access and authorization protocol is convenient to build a complete data processing system on the premise of protecting data privacy based on public cloud storage system or distributed storage system.

关键词： proxy re-encryption(PRE) data privacy distributed ledger technology blockchain

AUCSeg: AUC-oriented pixel-level long-tail semantic segmentation 24

学校读者我要写书评

暂无评论

AUCSeg: AUC-oriented pixel-level long-tail semantic segmenta...

Proceedings of the 38th International Conference on Neural information processing Systems

作者： Boyu Han Qianqian Xu Zhiyong Yang Shilong Bao Peisong Wen Yangbangyan Jiang Qingming Huang Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS and School of Computer Science and Tech. University of Chinese Academy of Sciences Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS and Peng Cheng Laboratory School of Computer Science and Tech. University of Chinese Academy of Sciences School of Computer Science and Tech. University of Chinese Academy of Sciences and Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS and Key Laboratory of Big Data Mining and Knowledge Management CAS

ISBN: (纸本)9798331314385

The Area Under the ROC Curve (AUC) is a well-known metric for evaluating instance-level long-tail learning problems. In the past two decades, many AUC optimization methods have been proposed to improve model performance under long-tail distributions. In this paper, we explore AUC optimization methods in the context of pixel-level long-tail semantic segmentation, a much more complicated scenario. This task introduces two major challenges for AUC optimization techniques. On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis. On the other hand, we find that mini-batch estimation of AUC loss in this case requires a larger batch size, resulting in an unaffordable space complexity. To address these issues, we develop a pixel-level AUC loss function and conduct a dependency-graph-based theoretical analysis of the algorithm's generalization ability. Additionally, we design a Tail-Classes Memory Bank (T-Memory Bank) to manage the significant memory demand. Finally, comprehensive experiments across various benchmarks confirm the effectiveness of our proposed AUCSeg method. The code is available at https://***/boyuh/AUCSeg.

关键词：

NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qian, Tianwen Chen, Jingjing Zhuo, Linhai Jiao, Yang Jiang, Yu-Gang Academy for Engineering and Technology Fudan University China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China

We introduce a novel visual question answering (VQA) task in the context of autonomous driving, aiming to answer natural language questions based on street-view clues. Compared to traditional VQA tasks, VQA in autonomous driving scenario presents more challenges. Firstly, the raw visual data are multi-modal, including images and point clouds captured by camera and LiDAR, respectively. Secondly, the data are multi-frame due to the continuous, real-time acquisition. Thirdly, the outdoor scenes exhibit both moving foreground and static background. Existing VQA benchmarks fail to adequately address these complexities. To bridge this gap, we propose NuScenes-QA, the first benchmark for VQA in the autonomous driving scenario, encompassing 34K visual scenes and 460K question-answer pairs. Specifically, we leverage existing 3D detection annotations to generate scene graphs and design question templates manually. Subsequently, the question-answer pairs are generated program-matically based on these templates. Comprehensive statistics prove that our NuScenes-QA is a balanced large-scale benchmark with diverse question formats. Built upon it, we develop a series of baselines that employ advanced 3D detection and VQA techniques. Our extensive experiments highlight the challenges posed by this new task. Codes and dataset are available at https://***/qiantianwen/NuScenes-QA. Copyright © 2023, The Authors. All rights reserved.

关键词： Autonomous vehicles