检索结果-内蒙古大学图书馆

Decoupled Prototype Learning for Reliable Test-Time Adaptation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Guowei Ding, Changxing Tan, Wentao Tan, Mingkui the School of Electronic and Information Engineering South China University of Technology 381 Wushan Road Tianhe District Guangzhou510000 China the Pazhou Lab Guangzhou510330 China the School of Future Technology South China University of Technology 381 Wushan Road Tianhe District Guangzhou510000 China the School of Software Engineering the Key Laboratory of Big Data and Intelligent Robot Ministry of Education South China University of Technology Guangzhou510006 China

Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference. One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels. However, its performance is significantly affected by noisy pseudo-labels. This study reveals that minimizing the classification error of each sample causes the cross-entropy loss’s vulnerability to label noise. To address this issue, we propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation. First, we decouple the optimization of class prototypes. For each class prototype, we reduce its distance with positive samples and enlarge its distance with negative samples in a contrastive manner. This strategy prevents the model from overfitting to noisy pseudo-labels. Second, we propose a memory-based strategy to enhance DPL’s robustness for the small batch sizes often encountered in TTA. We update each class’s pseudo-feature from a memory in a momentum manner and insert an additional DPL loss. Finally, we introduce a consistency regularization-based approach to leverage samples with unconfident pseudo-labels. This approach transfers feature styles of samples with unconfident pseudo-labels to those with confident pseudo-labels. Thus, more reliable samples for TTA are created. The experimental results demonstrate that our methods achieve state-of-the-art performance on domain generalization benchmarks, and reliably improve the performance of self-training-based methods on image corruption benchmarks. The code will be released. Copyright © 2024, The Authors. All rights reserved.

关键词： Benchmarking

Syntax-enhanced pre-trained model 59

学校读者我要写书评

暂无评论

Syntax-enhanced pre-trained model

Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021

作者： Xu, Zenan Guo, Daya Tang, Duyu Su, Qinliang Shou, Linjun Gong, Ming Zhong, Wanjun Quan, Xiaojun Jiang, Daxin Duan, Nan School of Computer Science and Engineering Sun Yat-Sen University Guangzhou China Microsoft Research Asia Beijing China Microsoft Search Technology Center Asia Beijing China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Key Lab. of Machine Intelligence and Advanced Computing Ministry of Education China

ISBN: (纸本)9781954085527

We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages. Such a problem would lead to the necessity of having human-annotated syntactic information, which limits the application of existing methods to broader scenarios. To address this, we present a model that utilizes the syntax of text in both pre-training and fine-tuning stages. Our model is based on Transformer with a syntax-aware attention layer that considers the dependency tree of the text. We further introduce a new pre-training task of predicting the syntactic distance among tokens in the dependency tree. We evaluate the model on three downstream tasks, including relation classification, entity typing, and question answering. Results show that our model achieves state-of-the-art performance on six public benchmark datasets. We have two major findings. First, we demonstrate that infusing automatically produced syntax of text improves pre-trained models. Second, global syntactic distances among tokens bring larger performance gains compared to local head relations between contiguous tokens. © 2021 Association for Computational Linguistics

关键词： Syntactics

Efficient Adaptive label Refinement for label Noise Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Wenzhen Cheng, Debo Lu, Guangquan Zhou, Bo Li, Jiaye Zhang, Shichao School of Computer Science and Engineering Guangxi Normal University Guangxi Guilin541004 China Guangxi Key Lab of Multi-Source Information Mining & Security Guangxi Normal University Guangxi Guilin541004 China Key Lab of Education Blockchain and Intelligent Technology Ministry of Education Guangxi Normal University Guangxi Guilin541004 China UniSA STEM University of South Australia AdelaideSA5095 Australia Guangxi Collaborative Innovation Center of Modern Sericulture and Silk Hechi University Guangxi Hechi546300 China The State Key Laboratory of Blockchain and Data Security Zhejiang University Zhejiang Hangzhou310027 China

Deep neural networks are highly susceptible to overfitting noisy labels, which leads to degraded performance. Existing methods address this issue by employing manually defined criteria, aiming to achieve optimal partitioning in each iteration to avoid fitting noisy labels while thoroughly learning clean samples. However, this often results in overly complex and difficult-to-train models. To address this issue, we decouple the tasks of avoiding fitting incorrect labels and thoroughly learning clean samples and propose a simple yet highly applicable method called Adaptive label Refinement (ALR). First, inspired by label refurbishment techniques, we update the original hard labels to soft labels using the model’s predictions to reduce the risk of fitting incorrect labels. Then, by introducing the entropy loss, we gradually ‘harden’ the high-confidence soft labels, guiding the model to better learn from clean samples. This approach is simple and efficient, requiring no prior knowledge of noise or auxiliary datasets, making it more accessible compared to Copyright © 2025, The Authors. All rights reserved.

关键词： Self-supervised learning

Order-preserving pattern mining with forgetting mechanism

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Yan Ma, Chenyu Gao, Rong Wu, Youxi Li, Jinyan Wang, Wenjian Wu, Xindong School of Economics and Management Hebei University of Technology Tianjin300400 China School of Artificial Intelligence Hebei University of Technology Tianjin300400 China School of Computer Science and Control Engineering Shenzhen University of Advanced Technology Shenzhen518055 China Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China School of Computer and Information Technology Shanxi University Taiyuan237016 China Key Laboratory of Knowledge Engineering with Big Data The Ministry of Education of China Hefei University of Technology Hefei230009 China

Order-preserving pattern (OPP) mining is a type of sequential pattern mining method in which a group of ranks of time series is used to represent an OPP. This approach can discover frequent trends in time series. Existing OPP mining algorithms consider data points at different time to be equally important;however, newer data usually have a more significant impact, while older data have a weaker impact. We therefore introduce the forgetting mechanism into OPP mining to reduce the importance of older data. This paper explores the mining of OPPs with forgetting mechanism (OPF) and proposes an algorithm called OPF-Miner that can discover frequent OPFs. OPF-Miner performs two tasks, candidate pattern generation and support calculation. In candidate pattern generation, OPF-Miner employs a maximal support priority strategy and a group pattern fusion strategy to avoid redundant pattern fusions. For support calculation, we propose an algorithm called support calculation with forgetting mechanism, which uses prefix and suffix pattern pruning strategies to avoid redundant support calculations. The experiments are conducted on nine datasets and 12 alternative algorithms. The results verify that OPF-Miner is superior to other competitive algorithms. More importantly, OPF-Miner yields good clustering performance for time series, since the forgetting mechanism is employed. Copyright © 2024, The Authors. All rights reserved.

关键词： Miners

Discover the Binding Domain of Transmembrane Proteins Based on Structural Universality

学校读者我要写书评

暂无评论

Discover the Binding Domain of Transmembrane Proteins Based ...

2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021

作者： Bao, Yihang He, Fei Wang, Weixi Wang, Han Dong, Minglong Northeast Normal University Changchun School of Information Science and Technology China NE Norm. Univ. Key Lab. of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun School of Information Science and Technology China Nanyang Technological University Singapore School of Biological Sciences Singapore

ISBN: (纸本)9781665401265

Transmembrane proteins (TMPs) serve as drug targets for more than half of the drugs currently available in the market. However, it had not been clearly explained how they realize their drug effects through multiple complex molecules bindings actions. Research into TMPs bindings and corresponding structural basis will provide key information for drug research and new drug development. In this study, we defined the binding domain of TMPs according to the binding region investigation of multiple conjugate types. A 3D deep learning model was architected to discover the structural universality inside those domains. The experimental results proved such binding domains existing on the surface of TMPs, and they are structural specific distinguishing to the surface regions without any binding activities. This work provides a new theoretical basis for TMPs binding research and can greatly boost the development of the drug industry. © 2021 IEEE.

关键词： Proteins

AutoSTL: Automated Spatio-Temporal Multi-Task Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Zijian Zhao, Xiangyu Miao, Hao Zhang, Chunxu Zhao, Hongwei Zhang, Junbo College of Computer Science and Technology Jilin University China School of Data Science City University of Hong Kong Hong Kong Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University China Department of Computer Science Aalborg University Denmark JD Intelligent Cities Research China JD iCity JD Technology China Hong Kong Institute for Data Science City University of Hong Kong Hong Kong

Spatio-temporal prediction plays a critical role in smart city construction. Jointly modeling multiple spatio-temporal tasks can further promote an intelligent city life by integrating their inseparable relationship. However, existing studies fail to address this joint learning problem well, which generally solve tasks individually or a fixed task combination. The challenges lie in the tangled relation between different properties, the demand for supporting flexible combinations of tasks and the complex spatio-temporal dependency. To cope with the problems above, we propose an Automated Spatio-Temporal multi-task Learning (AutoSTL) method to handle multiple spatio-temporal tasks jointly. Firstly, we propose a scalable architecture consisting of advanced spatio-temporal operations to exploit the complicated dependency. Shared modules and feature fusion mechanism are incorporated to further capture the intrinsic relationship between tasks. Furthermore, our model automatically allocates the operations and fusion weight. Extensive experiments on benchmark datasets verified that our model achieves state-of-the-art performance. As we can know, AutoSTL is the first automated spatio-temporal multi-task learning method. Copyright © 2023, The Authors. All rights reserved.

关键词： Smart city

Multiagent Q-Learning for Multicrew Dynamic Scheduling and Routing in Road Network Restoration

学校读者我要写书评

暂无评论

Multiagent Q-Learning for Multicrew Dynamic Scheduling and R...

Chinese Control and Decision Conference, CCDC

作者： Yufeng Shen Lianqi Duan Qianshun Zhu Zhaopin Su Guofu Zhang School of Computer Science and Information Engineering Hefei University of Technology Hefei 230601 School of Computer Science and Information Engineering Hefei University of Technology Hefei Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology) Ministry of Education Hefei Intelligent Interconnected Systems Laboratory of Anhui Province (Hefei University of Technology) Hefei Anhui Province Key Laboratory of Industry Safety and Emergency Technology (Hefei University of Technology) Heifei

ISBN: (纸本)9781665478977

Road network restoration is an important issue in the post-disaster disposal and rescue, especially when extraordinarily serious natural disasters (e.g., floods and earthquakes) occur. Central to this endeavour is the problem of determining how to reasonably schedule and route the repair crew to quickly restore the damaged road network and establish reliable supply lines from supply nodes to demand nodes. However, most existing work focuses on the activities of the single repair crew in the static road network, which is unable to adapt to the dynamic changes of the road network caused by secondary disasters, such as rock slides and debris flows. Consequently, this work is concentrated on multicrew dynamic scheduling and routing in road network restoration. Specifically, a model of multicrew dynamic scheduling and routing is first presented. Next, the Markov decision process is adopted to construct a decision model for repair crews, based on which a multiagent Q-learning algorithm is developed to find the best scheduling and routing strategy. Finally, experimental results demonstrate that the proposed method can make repair crews adjust their scheduling and routing strategies according to the dynamic changes of the damaged road network and provides a useful attempt to restore the damaged road network in complex, dynamic emergency scenarios of post-disaster.

关键词： Schedules Q-learning Roads Heuristic algorithms Maintenance engineering Markov processes Dynamic scheduling

An Abnormal data Analysis and Processing Method for Genealogy Graph databases

学校读者我要写书评

暂无评论

An Abnormal Data Analysis and Processing Method for Genealog...

IEEE International Conference on Big knowledge (ICBK)

作者： Jianxuan Shao Guliu Liu Shengwei Ji Ministry of Education Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology) China

ISBN: (数字)9781728181561

ISBN: (纸本)9781728181578

Large amounts of data are generated continually in the real world. The objects in the data and the relationships between them have become increasingly complex. Graph is a powerful tool for representing these data and the complex relationships between them. To effectively describe the entities and connections in these data, the concept of a knowledge graph was proposed, and knowledge graph has become one of the bases for storing graph data, and the importance of it is self-evident. Genealogy data is a kind of graph data where nodes are used to represent the person in the genealogy data, and edges are used to represent the relationship between the person. Moreover, in a genealogy graph database, the "critical" relationship that is unique for an entity is defined. In contrast, the abnormal data derive from the existence of multiple "critical" relationships between entities. In a genealogy graph, abnormal data will cause the wrong relationships and redundant entities. To avoid such abnormal data in genealogy graph database, the abnormal data are categorized into four different types, and the corresponding processing methods are proposed for each type of abnormal data, respectively. The experiment results demonstrate that the processing method can effectively solve these abnormal data.

关键词： databases Biology Task analysis Anomaly detection data models knowledge engineering Big data

Web Log Analysis in Genealogy System

学校读者我要写书评

暂无评论

Web Log Analysis in Genealogy System

IEEE International Conference on Big knowledge (ICBK)

作者： Xiaojian Liu Yi Zhu Shengwei Ji Ministry of Education Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology) China

ISBN: (数字)9781728181561

ISBN: (纸本)9781728181578

A large amount of log data is generated every moment in many real-world applications. System logs, mobile application logs, web logs etc., all of which contain rich information and can be analyzed for valuable information. At the same time, genealogy attracts more and more attention in recent decades, especially for Chinese. People utilize the genealogy system to seek roots, find source of family, build a family tree and so on. Due to the complexity of genealogy data and specific features of genealogy system, how to extract and analyze the valued genealogy information effectively is a huge challenge. Motivated by this, we propose a novel analysis method using web logs in genealogy system for drawing user profile and providing personalized recommendations. More specifically, firstly, all web logs are extracted from genealogy system based on the enhanced logs which record the behaviors and data of user. Secondly, the extracted logs are formatted for analysis, and we analyze the logs to extract various user-related attributes for drawing user profile. Finally, personalized recommendations like interested people and family tree are provided for each user in system. The high availability and effectiveness of our proposed method is verified on genealogy system-Huapu system.

关键词： data mining databases data visualization Tools Big data Feature extraction knowledge engineering