检索结果-内蒙古大学图书馆

FedMatch: Federated learning over heterogeneous question answering data

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chen, Jiangui Zhang, Ruqing Guo, Jiafeng Fan, Yixing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Question Answering (QA), a popular and promising technique for intelligent information access, faces a dilemma about data as most other AI techniques. On one hand, modern QA methods rely on deep learning models which are typically data-hungry. Therefore, it is expected to collect and fuse all the available QA datasets together in a common site for developing a powerful QA model. On the other hand, real-world QA datasets are typically distributed in the form of isolated islands belonging to different parties. Due to the increasing awareness of privacy security, it is almost impossible to integrate the data scattered around, or the cost is prohibited. A possible solution to this dilemma is a new approach known as federated learning, which is a privacy-preserving machine learning technique over distributed datasets. In this work, we propose to adopt federated learning for QA with the special concern on the statistical heterogeneity of the QA data. Here the heterogeneity refers to the fact that annotated QA data are typically with non-identical and independent distribution (non-IID) and unbalanced sizes in practice. Traditional federated learning methods may sacrifice the accuracy of individual models under the heterogeneous situation. To tackle this problem, we propose a novel Federated Matching framework for QA, named FedMatch, with a backbone-patch architecture. The shared backbone is to distill the common knowledge of all the participants while the private patch is a compact and efficient module to retain the domain information for each participant. To facilitate the evaluation, we build a benchmark collection based on several QA datasets from different domains to simulate the heterogeneous situation in practice. Empirical studies demonstrate that our model can achieve significant improvements against the baselines over all the datasets. © 2021, CC BY-NC-ND.

关键词： Deep learning

Group RandAugment: Video Augmentation for Action Recognition 5

学校读者我要写书评

暂无评论

Group RandAugment: Video Augmentation for Action Recognition

5th International Conference on data science and Information technology, DSIT 2022

作者： An, Fengmin Zhang, Bingbing Wang, Zhenwei Dong, Wei Zhang, Jianxin School of Computer Science and Engineering Dalian Minzu University Dalian China Institute of Machine Intelligence and Bio-computing Dalian Minzu University Dalian China SEAC Key Lab of Big Data Applied Technology Dalian Minzu University Dalian China School of Information and Communication Engineering Dalian University of Technology Dalian China

ISBN: (数字)9781665498685

ISBN: (纸本)9781665498685

data augmentation, as a critical strategy in deep learning, well improves the sample diversity for network training, leading to the obvious improvement of model generalization ability. Besides, automatic data augmentation, while sparking in image tasks, has attracted too little attention in video recognition task. Therefore, this work explores a novel group random augmentation (GRA) that automatically augments video data for recognition. GRA first collects 21 augmentation transformations to enrich the sample diversity, and then divides these transformations into four groups (i.e., pixel-transform, rigidtransform, erasing-transform, environment-transform) to reduce the excessive regularization caused by similar augmentation. To reduce the computational cost of finding the optimal setting, GRA adopts the combination form to select the optimal situation. For frames in the same video clip, GRA uses synchronous GRA. Additionally, the proposed GRA can be integrated into any existing video frameworks. To prove the effectiveness of GRA, we conduct experiments on two commonly used video action recognition benchmarks (Something-Something V1 & V2) and three typical frameworks (TSM, GSM, and TEA), whose results demonstrate GRA can improve performance in all cases without adding additional computational cost. © 2022 IEEE.

关键词： Deep learning

B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ma, Xinyu Guo, Jiafeng Zhang, Ruqing Fan, Yixing Li, Yingyan Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Pre-training and fine-tuning have achieved remarkable success in many downstream natural language processing (NLP) tasks. Recently, pre-training methods tailored for information retrieval (IR) have also been explored, and the latest success is the PROP method which has reached new SOTA on a variety of ad-hoc retrieval benchmarks. The basic idea of PROP is to construct the representative words prediction (ROP) task for pre-training inspired by the query likelihood model. Despite its exciting performance, the effectiveness of PROP might be bounded by the classical unigram language model adopted in the ROP task construction process. To tackle this problem, we propose a bootstrapped pre-training method (namely B-PROP) based on BERT for ad-hoc retrieval. The key idea is to use the powerful contextual language model BERT to replace the classical unigram language model for the ROP task construction, and re-train BERT itself towards the tailored objective for IR. Specifically, we introduce a novel contrastive method, inspired by the divergence-from-randomness idea, to leverage BERT's self-attention mechanism to sample representative words from the document. By further fine-tuning on downstream ad-hoc retrieval tasks, our method achieves significant improvements over PROP and other baselines, and further pushes forward the SOTA on a variety of ad-hoc retrieval tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： Natural language processing systems

How to use open-pFind in deep proteomics data analysis?—A protocol for rigorous identification and quantitation of peptides and proteins from mass spectrometry data

学校读者我要写书评

暂无评论

Biophysics Reports 2021年第3期7卷 207-226页

作者： Guangcan Shao Yong Cao Zhenlin Chen Chao Liu Shangtong Li Hao Chi Meng-Qiu Dong School of Life Sciences Peking University National Institute of Biological Sciences Beijing Tsinghua Institute of Multidisciplinary Biomedical Research Tsinghua University Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) University of CASInstitute of Computing TechnologyCAS University of Chinese Academy of Sciences Beijing Advanced Innovation Center for Big Data-Based Precision Medicine School of Medicine and EngineeringBeihang University

High-throughput proteomics based on mass spectrometry（MS） analysis has permeated biomedical science and propelled numerous research projects. p Find 3 is a database search engine for high-speed and in-depth proteomics data analysis. p Find 3 features a swift open search workflow that is adept at uncovering less obvious information such as unexpected modifications or mutations that would have gone unnoticed using a conventional data analysis pipeline. In this protocol, we provide step-by-step instructions to help users mastering various types of data analysis using p Find 3 in conjunction with p Parse for data pre-processing and if needed, p Quant for quantitation. This streamlined p Parse-p Findp Quant workflow offers exceptional sensitivity, precision, and speed. It can be easily implemented in any laboratory in need of identifying peptides, proteins, or post-translational modifications, or of quantitation based on15N-labeling, SILAC-labeling, or TMT/i TRAQ labeling.

关键词： Mass spectrometry Protein identification Search engine Open-pFind Quantitation

PEDNet: A Persona Enhanced Dual Alternating Learning network for Conversational Response Generation 28

学校读者我要写书评

暂无评论

PEDNet: A Persona Enhanced Dual Alternating Learning Network...

28th International Conference on Computational Linguistics, COLING 2020

作者： Jiang, Bin Zhou, Wanyue Yang, Jingxu Yang, Chao Wang, Shihan Pang, Liang College of Computer Science and Electronic Engineering Hunan University China Intelligent Systems Utrecht University Netherlands CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China

ISBN: (纸本)9781952148279

Endowing a chatbot with a personality is essential to deliver more realistic conversations. Various persona-based dialogue models have been proposed to generate personalized and diverse responses by utilizing predefined persona information. However, generating personalized responses is still a challenging task since the leverage of predefined persona information is often insufficient. To alleviate this problem, we propose a novel Persona Enhanced Dual Alternating Learning network (PEDNet) aiming at producing more personalized responses in various open-domain conversation scenarios. PEDNet consists of a Context-Dominated network (CDNet) and a Persona-Dominated network (PDNet), which are built upon a common encoder-decoder backbone. CDNet learns to select a proper persona as well as ensure the contextual relevance of the predicted response, while PDNet learns to enhance the utilization of persona information when generating the response by weakening the disturbance of specific content in the conversation context. CDNet and PDNet are trained alternately using a multi-task training approach to equip PEDNet with the both capabilities they have learned. Both automatic and human evaluations on a newly released dialogue dataset Persona-chat demonstrate that our method could deliver more personalized responses than baseline methods. © 2020 COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference. All rights reserved.

关键词： Learning systems

METNet: A Mutual Enhanced Transformation network for Aspect-based Sentiment Analysis 28

学校读者我要写书评

暂无评论

METNet: A Mutual Enhanced Transformation Network for Aspect-...

28th International Conference on Computational Linguistics, COLING 2020

作者： Jiang, Bin Hou, Jing Zhou, Wanyue Yang, Chao Wang, Shihan Pang, Liang College of Computer Science and Electronic Engineering Hunan University China Intelligent Systems Utrecht University Netherlands CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China

ISBN: (纸本)9781952148279

Aspect-based sentiment analysis (ABSA) aims to determine the sentiment polarity of each specific aspect in a given sentence. Existing researches have realized the importance of the aspect for the ABSA task and have derived many interactive learning methods that model context based on specific aspect. However, current interaction mechanisms are ill-equipped to learn complex sentences with multiple aspects, and these methods underestimate the representation learning of the aspect. In order to solve the two problems, we propose a mutual enhanced transformation network (METNet) for the ABSA task. First, the aspect enhancement module in METNet improves the representation learning of the aspect with contextual semantic features, which gives the aspect more abundant information. Second, METNet designs and implements a hierarchical structure, which enhances the representations of aspect and context iteratively. Experimental results on SemEval 2014 datasets demonstrate the effectiveness of METNet, and we further prove that METNet is outstanding in multi-aspect scenarios. © 2020 COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference. All rights reserved.

关键词： Sentiment analysis

Search from history and reason for future: Two-stage reasoning on temporal knowledge graphs

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Zixuan Jin, Xiaolong Guan, Saiping Li, Wei Guo, Jiafeng Wang, Yuanzhuo Cheng, Xueqi School of Computer Science and Technology University of Chinese Academy of Sciences CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Baidu Inc

Temporal Knowledge Graphs (TKGs) have been developed and used in many different areas. Reasoning on TKGs that predicts potential facts (events) in the future brings great challenges to existing models. When facing a prediction task, human beings usually search useful historical information (i.e., clues) in their memories and then reason for future meticulously. Inspired by this mechanism, we propose CluSTeR to predict future facts in a two-stage manner, Clue Searching and Temporal Reasoning, accordingly. Specifically, at the clue searching stage, CluSTeR learns a beam search policy via reinforcement learning (RL) to induce multiple clues from historical facts. At the temporal reasoning stage, it adopts a graph convolution network based sequence method to deduce answers from clues. Experiments on four datasets demonstrate the substantial advantages of CluSTeR compared with the state-of-the-art methods. Moreover, the clues found by CluSTeR further provide interpretability for the results. Copyright © 2021, The Authors. All rights reserved.

关键词： Knowledge graph

Adaptive Incentivize for Federated Learning with Cloud-Edge Collaboration under Multi-level Information Sharing

学校读者我要写书评

暂无评论

IEEE Transactions on Computers 2025年

作者： Yuan, Shijing Dong, Beiyu Li, Jie Guo, Song Chen, Hongyang Wu, Chentao Wu, Jie Zhao, Wei China Telecom Research Institute Department of Fundamental Network Operations Shanghai China Shanghai Jiao Tong University Department of Computer Science and Engineering Shanghai China Hong Kong University of Science and Technology Department of Computing Hong Kong Zhejiang Lab Hangzhou311121 China Temple University Department of Computer and Information Sciences United States CAS Shenzhen Institute of Advanced Technology Shenzhen518055 China

Federated Learning with Cloud-Edge Collaboration (FL-CEC) has emerged as a cutting-edge paradigm in distributed learning. Efficient resource investment incentive mechanisms are crucial to encouraging clients in FL-CEC to contribute the necessary data and computational resources for training. However, existing studies are inadequate in meeting the incentive design requirements under multi-level information-sharing scenarios. Moreover, current works often rely on specific functional relationships between resource investment and global model accuracy. To bridge these gaps, this paper investigates the incentive problem for data and computational resource investment under multi-level information-sharing levels. We design a resource investment incentive mechanism based on a weighted potential game without depending on any specific functional relationship between data investment and model accuracy. Furthermore, we propose four algorithms to solve resource investment strategies for different levels of information sharing. The complexity and convergence rates of the proposed algorithms are thoroughly analyzed. Finally, we construct a simulation incentive platform on the Aliyun. Extensive evaluations demonstrate that the proposed scheme effectively enhances social welfare, and improves collaborative training accuracy and efficiency. © 1968-2012 IEEE.

关键词： Open access

ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： He, Junwei Xu, Qianqian Jiang, Yangbangyan Wang, Zitai Huang, Qingming Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China Institute of Information Engineering CAS Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China

Graph anomaly detection is crucial for identifying nodes that deviate from regular behavior within graphs, benefiting various domains such as fraud detection and social network. Although existing reconstruction-based methods have achieved considerable success, they may face the Anomaly Overfitting and Homophily Trap problems caused by the abnormal patterns in the graph, breaking the assumption that normal nodes are often better reconstructed than abnormal ones. Our observations indicate that models trained on graphs with fewer anomalies exhibit higher detection performance. Based on this insight, we introduce a novel two-stage framework called Anomaly-Denoised Autoencoders for Graph Anomaly Detection (ADA-GAD). In the first stage, we design a learning-free anomaly-denoised augmentation method to generate graphs with reduced anomaly levels. We pretrain graph autoencoders on these augmented graphs at multiple levels, which enables the graph autoencoders to capture normal patterns. In the next stage, the decoders are retrained for detection on the original graph, benefiting from the multi-level representations learned in the previous stage. Meanwhile, we propose the node anomaly distribution regularization to further alleviate Anomaly Overfitting. We validate the effectiveness of our approach through extensive experiments on both synthetic and real-world datasets. Copyright © 2023, The Authors. All rights reserved.

关键词： Anomaly detection