检索结果-内蒙古大学图书馆

MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cui, Wanqing Bi, Keping Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Since commonsense information has been recorded significantly less frequently than its existence, language models pre-trained by text generation have difficulty to learn sufficient commonsense knowledge. Several studies have leveraged text retrieval to augment the models' commonsense ability. Unlike text, images capture commonsense information inherently but little effort has been paid to effectively utilize them. In this work, we propose a novel Multi-mOdal REtrieval (MORE) augmentation framework, to leverage both text and images to enhance the commonsense ability of language models. Extensive experiments on the Common-Gen task have demonstrated the efficacy of MORE based on the pre-trained models of both single and multiple modalities. © 2024, CC BY-NC-SA.

关键词： Computational linguistics

M-DRL: Deep Reinforcement Learning Based Coflow Traffic Scheduler with MLFQ Threshold Adaption 17th

学校读者我要写书评

暂无评论

M-DRL: Deep Reinforcement Learning Based Coflow Traffic Sche...

17th IFIP WG 10.3 International Conference on network and Parallel Computing, NPC 2020

作者： Chen, Tianba Li, Wei Sun, YuKang Li, Yunchun Beijing Key Lab of Network Technology School of Computer Science and Engineering Beihang University Beijing China

ISBN: (纸本)9783030794774

The coflow scheduling in data-parallel clusters can improve application-level communication performance. The existing coflow scheduling method without prior knowledge usually uses Multi-Level Feedback Queue (MLFQ) with fixed threshold parameters, which is insensitive to coflow traffic characteristics. Manual adjustment of the threshold parameters for different application scenarios often has long optimization period and is coarse in optimization granularity. We propose M-DRL, a deep reinforcement learning based coflow traffic scheduler by dynamically setting thresholds of MLFQ to adapt to the coflow traffic characteristics, and reduces the average coflow completion time. Trace-driven simulations on the public dataset show that coflow communication stages using M-DRL complete 2.08 × (6.48 × ) and 1.36 × (1.25 × ) faster on average coflow completion time (95-th percentile) in comparison to per-flow fairness and Aalo, and is comparable to SEBF with prior knowledge. © 2021, IFIP International Federation for Information Processing.

关键词： Reinforcement learning

Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data

学校读者我要写书评

暂无评论

ACM Transactions on Intelligent Systems and technology 2024年第6期15卷 1-28页

作者： Liu, Ji Jia, Juncheng Zhang, Hong Yun, Yuhui Wang, Leye Zhou, Yang Dai, Huaiyu Dou, Dejing Hithink RoyalFlush Information Network Co. Ltd. Hangzhou China School of Computer Science and Technology Soochow University Suzhou China Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing China Baidu Research Beijing China Key Lab of High Confidence Software Technologies Ministry of Education and Software Institute Peking University Beijing China Department of Computer Science and Software Engineering Auburn University AuburnAL United States Department of Electrical and Computer Engineering North Carolina State University RaleighNC United States Bedi Cloud and School of Computer Science Fudan University Beijing China

Despite achieving remarkable performance, Federated Learning (FL) encounters two important problems, i.e., low training efficiency and limited computational resources. In this article, we propose a new FL framework, i.e., FedDUMAP, with three original contributions, to leverage the shared insensitive data on the server in addition to the distributed data in edge devices so as to efficiently train a global model. First, we propose a simple dynamic server update algorithm, which takes advantage of the shared insensitive data on the server while dynamically adjusting the update steps on the server in order to speed up the convergence and improve the accuracy. Second, we propose an adaptive optimization method with the dynamic server update algorithm to exploit the global momentum on the server and each local device for superior accuracy. Third, we develop a layer-adaptive model pruning method to carry out specific pruning operations, which is adapted to the diverse features of each layer so as to attain an excellent tradeoff between effectiveness and efficiency. Our proposed FL model, FedDUMAP, combines the three original techniques and has a significantly better performance compared with baseline approaches in terms of efficiency (up to 16.9 times faster), accuracy (up to 20.4% higher), and computational cost (up to 62.6% smaller). © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.

关键词： Federated learning

Nested Event Extraction upon Pivot Element Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ren, Weicheng Li, Zixuan Jin, Xiaolong Bai, Long Su, Miao Liu, Yantao Guan, Saiping Guo, Jiafeng Cheng, Xueqi School of Computer Science and Technology University of Chinese Academy of Sciences China Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China

Nested Event Extraction (NEE) aims to extract complex event structures where an event contains other events as its arguments recursively. Nested events involve a kind of Pivot Elements (PEs) that simultaneously act as arguments of outer-nest events and as triggers of inner-nest events, and thus connect them into nested structures. This special characteristic of PEs brings challenges to existing NEE methods, as they cannot well cope with the dual identities of PEs. Therefore, this paper proposes a new model, called PerNee, which extracts nested events mainly based on recognizing PEs. Specifically, PerNee first recognizes the triggers of both inner-nest and outer-nest events and further recognizes the PEs via classifying the relation type between trigger pairs. The model uses prompt learning to incorporate information from both event types and argument roles for better trigger and argument representations to improve NEE performance. Since existing NEE datasets (e.g., Genia11) are limited to specific domains and contain a narrow range of event types with nested structures, we systematically categorize nested events in the generic domain and construct a new NEE dataset, called ACE2005-Nest. Experimental results demonstrate that PerNee consistently achieves state-of-the-art performance on ACE2005-Nest, Genia11, and Genia13. The ACE2005-Nest dataset and the code of the PerNee model are available at https://***/waysonren/PerNee. Copyright © 2023, The Authors. All rights reserved.

关键词： Character recognition

L2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Cai, Yinqiong Bi, Keping Fan, Yixing Guo, Jiafeng Chen, Wei Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

First-stage retrieval is a critical task that aims to retrieve relevant document candidates from a large-scale collection. While existing retrieval models have achieved impressive performance, they are mostly studied on static data sets, ignoring that in the real-world, the data on the Web is continuously growing with potential distribution drift. Consequently, retrievers trained on static old data may not suit new-coming data well and inevitably produce sub-optimal results. In this work, we study lifelong learning for first-stage retrieval, especially focusing on the setting where the emerging documents are unlabeled since relevance annotation is expensive and may not keep up with data emergence. Under this setting, we aim to develop model updating with two goals: (1) to effectively adapt to the evolving distribution with the unlabeled new-coming data, and (2) to avoid re-inferring all embeddings of old documents to efficiently update the index each time the model is updated. We first formalize the task and then propose a novel Lifelong Learning method for the first-stage Retrieval, namely L2R. L2R adopts the typical memory mechanism for lifelong learning, and incorporates two crucial components: (1) selecting diverse support negatives for model training and memory updating for effective model adaptation, and (2) a ranking alignment objective to ensure the backward-compatibility of representations to save the cost of index rebuilding without hurting the model performance. For evaluation, we construct two new benchmarks from LoTTE and Multi-CPR © 2023, CC BY-NC-ND.

关键词： Learning systems

A Comparative Study of Training Objectives for Clarification Facet Generation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ni, Shiyu Bi, Keping Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

Due to the ambiguity and vagueness of a user query, it is essential to identify the query facets for the clarification of user intents. Existing work on query facet generation has achieved compelling performance by sequentially predicting the next facet given previously generated facets based on pre-trained language generation models such as BART. Given a query, there are mainly two types of training objectives to guide the facet generation models. One is to generate the default sequence of ground-truth facets, and the other is to enumerate all the permutations of ground-truth facets and use the sequence that has the minimum loss for model updates. The second is permutation-invariant while the first is not. In this paper, we aim to conduct a systematic comparative study of various types of training objectives, with different properties of not only whether it is permutation-invariant but also whether it conducts sequential prediction and whether it can control the count of output facets. To this end, we propose another three training objectives of different aforementioned properties. For comprehensive comparisons, besides the commonly used evaluation that measures the matching with ground-truth facets, we also introduce two diversity metrics to measure the diversity of the generated facets. Based on an open-domain query facet dataset, i.e., MIMICS, we conduct extensive analyses and show the pros and cons of each method, which could shed light on model training for clarification facet generation. The code can be found at https://***/ShiyuNee/Facet-Generation. © 2023, CC BY.

关键词： Clarifiers

Multi-granular Adversarial Attacks against Black-box Neural Ranking Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Yu-An Zhang, Ruqing Guo, Jiafeng de Rijke, Maarten Fan, Yixing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China University of Amsterdam Amsterdam Netherlands

Adversarial ranking attacks have gained increasing attention due to their success in probing vulnerabilities, and, hence, enhancing the robustness, of neural ranking models. Conventional attack methods employ perturbations at a single granularity, e.g., word or sentence level, to target documents. However, limiting perturbations to a single level of granularity may reduce the flexibility of adversarial examples, thereby diminishing the potential threat of the attack. Therefore, we focus on generating high-quality adversarial examples by incorporating multi-granular perturbations. Achieving this objective involves tackling a combinatorial explosion problem, which requires identifying an optimal combination of perturbations across all possible levels of granularity, positions, and textual pieces. To address this challenge, we transform the multi-granular adversarial attack into a sequential decision-making process, where perturbations in the next attack step build on the perturbed document in the current attack step. Since the attack process can only access the final state without direct intermediate signals, we use reinforcement learning to perform multi-granular attacks. During the reinforcement learning process, two agents work cooperatively to identify multi-granular vulnerabilities as attack targets and organize perturbation candidates into a final perturbation sequence. Experimental results show that our attack method surpasses prevailing baselines in both attack effectiveness and imperceptibility. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

Inducing Causal Structure for Abstractive Text Summarization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Lu Zhang, Ruqing Huang, Wei Chen, Wei Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China

The mainstream of data-driven abstractive summarization models tends to explore the correlations rather than the causal relationships. Among such correlations, there can be spurious ones which suffer from the language prior learned from the training corpus and therefore undermine the overall effectiveness of the learned model. To tackle this issue, we introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data. We assume several latent causal factors and non-causal factors, representing the content and style of the document and summary. Theoretically, we prove that the latent factors in our SCM can be identified by fitting the observed training data under certain conditions. On the basis of this, we propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors, guiding us to pursue causal information for summary generation. The key idea is to reformulate the Variational Auto-encoder (VAE) to fit the joint distribution of the document and summary variables from the training corpus. Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach. © 2023, CC BY-NC-ND.

关键词： Text processing

Robust Information Retrieval

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Yu-An Zhang, Ruqing Guo, Jiafeng de Rijke, Maarten CAS Key Lab of Network Data Science and Technology ICT CAS University of Chinese Academy of Sciences Beijing China University of Amsterdam Amsterdam Netherlands

Beyond effectiveness, the robustness of an information retrieval (IR) system is increasingly attracting attention. When deployed, a critical technology such as IR should not only deliver strong performance on average but also have the ability to handle a variety of exceptional situations. In recent years, research into the robustness of IR has seen significant growth, with numerous researchers offering extensive analyses and proposing myriad strategies to address robustness challenges. In this tutorial, we first provide background information covering the basics and a taxonomy of robustness in IR. Then, we examine adversarial robustness and out-of-distribution (OOD) robustness within IR-specific contexts, extensively reviewing recent progress in methods to enhance robustness. The tutorial concludes with a discussion on the robustness of IR in the context of large language models (LLMs), highlighting ongoing challenges and promising directions for future research. This tutorial aims to generate broader attention to robustness issues in IR, facilitate an understanding of the relevant literature, and lower the barrier to entry for interested researchers and practitioners. © 2024, CC BY.

关键词： Search engines