检索结果-内蒙古大学图书馆

EAT: Towards Long-Tailed Out-of-Distribution Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wei, Tong Wang, Bo-Lin Zhang, Min-Ling School of Computer Science and Engineering Southeast University Nanjing210096 China Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education China

Despite recent advancements in out-of-distribution (OOD) detection, most current studies assume a class-balanced in-distribution training dataset, which is rarely the case in real-world scenarios. This paper addresses the challenging task of long-tailed OOD detection, where the in-distribution data follows a long-tailed class distribution. The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes, as the ability of a classifier to detect OOD instances is not strongly correlated with its accuracy on the in-distribution classes. To overcome this issue, we propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes. This approach allows us to build a detector with clear decision boundaries by training on OOD data using virtual labels. (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data. This technique encourages the model to pay more attention to the discriminative features of the tail classes. We provide a clue for separating in-distribution and OOD data by analyzing gradient noise. Through extensive experiments, we demonstrate that our method outperforms the current state-of-the-art on various benchmark datasets. Moreover, our method can be used as an add-on for existing long-tail learning approaches, significantly enhancing their OOD detection performance. Code is available at: https://***/Stomachache/Long-Tailed-OOD-Detection. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Fine-grainedly Synthesize Streaming Data Based On Large Language Models With Graph Structure Understanding For Data Sparsity

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Xin Zhang, Linhai Zhou, Deyu Xu, Guoqiang School of Computer Science and Engineering Key Laboratory of Computer Network and Information Integration Ministry of Education Southeast University China SANY Group Co. Ltd China

Due to the sparsity of user data, sentiment analysis on user reviews in e-commerce platforms often suffers from poor performance, especially when faced with extremely sparse user data or long-tail labels. Recently, the emergence of LLMs has introduced new solutions to such problems by leveraging graph structures to generate supplementary user profiles. However, previous approaches have not fully utilized the graph understanding capabilities of LLMs and have struggled to adapt to complex streaming data environments. In this work, we propose a fine-grained streaming data synthesis framework that categorizes sparse users into three categories: Mid-tail, Long-tail, and Extreme. Specifically, we design LLMs to comprehensively understand three key graph elements in streaming data, including Local-global Graph Understanding, Second-Order Relationship Extraction, and Product Attribute Understanding, which enables the generation of high-quality synthetic data to effectively address sparsity across different categories. Experimental results on three real datasets demonstrate significant performance improvements, with synthesized data contributing to MSE reductions of 45.85%, 3.16%, and 62.21%, respectively. Copyright © 2024, The Authors. All rights reserved.

关键词： Sentiment analysis

PE: A Poincare Explanation Method for Fast Text Hierarchy Generation

学校读者我要写书评

暂无评论

PE: A Poincare Explanation Method for Fast Text Hierarchy Ge...

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Chen, Qian Li, Dongyang He, Xiaofeng Li, Hongzhao Yi, Hongyu School of Computer Science and Technology East China Normal University Shanghai China NPPA Key Laboratory of Publishing Integration Development ECNUP Shanghai China Sichuan Caizi Software Information Network Co. Ltd. China

ISBN: (纸本)9798891761681

The black-box nature of deep learning models in NLP hinders their widespread *** research focus has shifted to Hierarchical Attribution (HA) for its ability to model feature *** works model non-contiguous combinations with a time-costly greedy search in Euclidean spaces, neglecting underlying linguistic information in feature *** this work, we introduce a novel method, namely Poincare Explanation (PE), for modeling feature interactions with hyperbolic spaces in a time efficient ***, we take building text hierarchies as finding spanning trees in hyperbolic *** we project the embeddings into hyperbolic spaces to elicit inherit semantic and syntax hierarchical *** we propose a simple yet effective strategy to calculate Shapley *** we build the the hierarchy with proving the constructing process in the projected space could be viewed as building a minimum spanning tree and introduce a time efficient building *** results demonstrate the effectiveness of our approach. © 2024 Association for Computational Linguistics.

关键词： Semantics

Data Augmentation For Label Enhancement

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Kou, Zhiqiang Jia, Yuheng Wang, Jing Shi, Boyu Geng, Xin MOE Key Laboratory of Computer Network and Information Integration China School of Computer Science and Engineering Southeast University Nanjing210096 China

Label distribution (LD) uses the description degree to describe instances, which provides more fine-grained supervision information when learning with label ambiguity. Nevertheless, LD is unavailable in many real-world applications. To obtain LD, label enhancement (LE) has emerged to recover LD from logical label. Existing LE approach have the following problems: (i) They use logical label to train mappings to LD, but the supervision information is too loose, which can lead to inaccurate model prediction;(ii) They ignore feature redundancy and use the collected features directly. To solve (i), we use the topology of the feature space to generate more accurate label-confidence. To solve (ii), we proposed a novel supervised LE dimensionality reduction approach, which projects the original data into a lower dimensional feature space. Combining the above two, we obtain the augmented data for LE. Further, we proposed a novel nonlinear LE model based on the label-confidence and reduced features. Extensive experiments on 12 real-world datasets are conducted and the results show that our method consistently outperforms the other five comparing approaches. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Dynamic Heterogeneous Federated Learning with Multi-Level Prototypes

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Guo, Shunxin Wang, Hongsong Geng, Xin MOE Key Laboratory of Computer Network and Information Integration China School of Computer Science and Engineering Southeast University Nanjing210096 China

Federated learning shows promise as a privacy-preserving collaborative learning technique. Existing heterogeneous federated learning mainly focuses on skewing the label distribution across clients. However, most approaches suffer from catastrophic forgetting and concept drift, mainly when the global distribution of all classes is extremely unbalanced and the data distribution of the client dynamically evolves over time. In this paper, we study the new task, i.e., Dynamic Heterogeneous Federated Learning (DHFL), which addresses the practical scenario where heterogeneous data distributions exist among different clients and dynamic tasks within the client. Accordingly, we propose a novel federated learning framework named Federated Multi-Level Prototypes (FedMLP) and design federated multi-level regularizations. To mitigate concept drift, we construct prototypes and semantic prototypes to provide fruitful generalization knowledge and ensure the continuity of prototype spaces. To maintain the model stability and consistency of convergence, three regularizations are introduced as training losses, i.e., prototype-based regularization, semantic prototype-based regularization, and federated inter-task regularization. Extensive experiments show that the proposed method achieves state-of-the-art performance in various settings. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

Exploiting Multi-Label Correlation in Label Distribution Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Kou, Zhiqiang Wang, Jing Jia, Yuheng Geng, Xin School of Computer Science and Engineering Southeast University Nanjing210096 China MOE Key Laboratory of Computer Network and Information Integration China

Label Distribution Learning (LDL) is a novel machine learning paradigm that assigns label distribution to each instance. Many LDL methods proposed to leverage label correlation in the learning process to solve the exponential-sized output space;among these, many exploited the low-rank structure of label distribution to capture label correlation. However, recent studies disclosed that label distribution matrices are typically full-rank, posing challenges to those works exploiting low-rank label correlation. Note that multi-label is generally low-rank;low-rank label correlation is widely adopted in multi-label learning (MLL) literature. Inspired by that, we introduce an auxiliary MLL process in LDL and capture low-rank label correlation on that MLL rather than LDL. In such a way, low-rank label correlation is appropriately exploited in our LDL methods. We conduct comprehensive experiments and demonstrate that our methods are superior to existing LDL methods. Besides, the ablation studies justify the advantages of exploiting low-rank label correlation in the auxiliary MLL. © 2023, CC BY.

关键词： Learning systems

Spatio-Temporal Context Modeling for Road Obstacle Detection 1

学校读者我要写书评

暂无评论

4th International Conference on Machine Learning for Cyber Security, ML4CS 2022

作者： Wu, Xiuen Wang, Tao Liang, Lingyu Li, Zuoyong Ching, Fum Yew College of Computer and Data Science Fuzhou University Fuzhou350108 China Fujian Provincial Key Laboratory of Information Processing and Intelligent Control Minjiang University Fuzhou350108 China The Key Laboratory of Cognitive Computing and Intelligent Information Processing of Fujian Education Institutions Wuyi University Wuyishan354300 China School of Electronic and Information Engineering South China University of Technology Guangzhou510641 China Ministry of Education Key Laboratory of Computer Network and Information Integration Southeast University Nanjing211189 China Guangzhou510320 China Penang11800 Malaysia

ISBN: (数字)9783031200960

ISBN: (纸本)9783031200953

Road obstacle detection is an important problem for vehicle driving safety. In this paper, we aim to obtain robust road obstacle detection based on spatio-temporal context modeling. Firstly, a data-driven spatial context model of the driving scene is constructed with the layouts of the training data. Then, obstacles in the input image are detected via the state-of-the-art object detection algorithms, and the results are combined with the generated scene layout. In addition, to further improve the performance and robustness, temporal information in the image sequence is taken into consideration, and the optical flow is obtained in the vicinity of the detected objects to track the obstacles across neighboring frames. Qualitative and quantitative experiments were conducted on the Small Obstacle Detection (SOD) dataset and the Lost and Found dataset. The results indicate that our method with spatio-temporal context modeling is superior to existing methods for road obstacle detection. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Object detection

Transformer Helps Gnns Express Better Via Distillation of Long-Range Dependencies

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Li, Jiaxing Zhang, Xiaotong Zhang, Ke Kong, Youyong Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing School of Computer Science and Engineering Southeast University Jiangsu Province Nanjing China Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education Nanjing China

Graph neural networks (GNNs) are powerful architectures for graph representation learning. However, GNNs based on message-passing cannot make fully use of the valuable long-range dependencies in graph-structured data. Recently, researchers have designed Transformer-based graph learning models to expand the receptive field to capture long-range dependencies between pairwise nodes. Although the preliminary performance of the graph Transformer is encouraging and even surpasses GNNs, the quadratic complexity of the self-attention leads to much longer inference time than GNNs, which limits the deployment in practical applications. To address this issue, we propose a knowledge distillation framework to transfer long-range dependencies captured by graph Transformer to GNNs. Our framework extracts knowledge from a pre-trained graph Transformer teacher and transfer it to a GNN-based student. To better transfer the long-range dependencies, we further design a long-range dependencies transfer module, in which the intermediate graph representations of the graph Transformer and GNNs are extracted as distributions. Then we minimize the distance between the two distributions, thereby achieving the long-range dependencies transfer more accurately. The proposed method can be extended to any graph Transformer and GNNs. We evaluate our proposed method on a large number of graph classification benchmark datasets with different GNN-based baselines, and the experimental results demonstrate that our method can obtain the performance close to graph Transformer with faster inference speed and smaller memory usage. © 2024, The Authors. All rights reserved.

关键词： Distillation

Test-Time Training with Invariant Graph Learning for Out-of-Distribution Generalization

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Pi, Li Li, Jiaxing Song, Lei Yang, Guanyu Chen, Yang Kong, Youyong Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing School of Computer Science and Engineering Southeast University Nanjing China Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education Jiangsu Province Nanjing China

Graph Neural networks (GNNs) frequently do not perform optimally on out-of-distribution (OOD) data. Previous research on graph OOD generalization suggests extracting invariant subgraphs, which provide crucial categorical information even under test distributions. However, this strategy is not optimal due to two issues. Firstly, there is a significant distribution shift between the test and training sets, which may result in incorrect invariant subgraphs being extracted during the test phase. Secondly, the extraction of subgraphs can lead to the loss of important structural information. To address these issues, we propose a novel two-stage training paradigm named \textbf{I}nvariant \textbf{G}raph \textbf{T}est-\textbf{T}ime \textbf{T}raining (\textbf{IGT3}), which uses the training paradigm called Test-Time Training (TTT). We also propose invariant graph recognition to learn key parts of the graph without destroying the original graph structure. In addition, we introduce a multi-level graph contrastive learning method as an auxiliary self-supervised task. To prevent TTT failure, we use the feature constraint to enhance the robustness of the training process. We conduct experiments on seven common graph classification OOD benchmark datasets. The results demonstrate that IGT3 outperforms state-of-the-art (SOTA) methods by an average of 4.2\%, confirming the effectiveness of each module. The paper code is available at https://***/r/IGT3-3281/ © 2024, The Authors. All rights reserved.

关键词： Graph neural networks