检索结果-内蒙古大学图书馆

Experimental Study on The Effect of Multi-step Deep Reinforcement Learning in POMDPs

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Meng, Lingheng Gorbet, Rob Burke, Michael Kulić, Dana CSIRO Data61 Private Bag 10 Clayton SouthVIC3169 Australia Faculty of Engineering Monash University ClaytonVIC3800 Australia Departments of Knowledge Integration and Electrical and Computer Engineering University of Waterloo WaterlooONN2L 3G1 Canada

Deep Reinforcement Learning (DRL) has made tremendous advances in both simulated and real-world robot control tasks in recent years. This is particularly the case for tasks that can be carefully engineered with a full state representation, and which can then be formulated as a Markov Decision Process (MDP). However, applying DRL strategies designed for MDPs to novel robot control tasks can be challenging, because the available observations may be a partial representation of the state, resulting in a Partially Observable Markov Decision Process (POMDP). This paper considers three popular DRL algorithms, namely Proximal Policy Optimization (PPO), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC), invented for MDPs, and studies their performance in POMDP scenarios. While prior work has found that SAC and TD3 typically outperform PPO across a broad range of tasks that can be represented as MDPs, we show that this is not always the case, using three representative POMDP environments. Empirical studies show that this is related to multi-step bootstrapping, where multi-step immediate rewards, instead of one-step immediate reward, are used to calculate the target value estimation of an observation and action pair. We identify this by observing that the inclusion of multi-step bootstrapping in TD3 (MTD3) and SAC (MSAC) results in improved robustness in POMDP settings. © 2022, CC BY-NC-ND.

关键词： Deep reinforcement learning

Pre-training for ad-hoc retrieval: Hyperlink is also you need

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ma, Zhengyi Dou, Zhicheng Xu, Wei Zhang, Xinyu Jiang, Hao Cao, Zhao Wen, Ji-Rong Gaoling School of Artificial Intelligence Renmin University of China Beijing China Distributed and Parallel Software Lab Huawei Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China Key Laboratory of Data Engineering and Knowledge Engineering MOE Beijing China

Designing pre-training objectives that more closely resemble the downstream tasks for pre-trained language models can lead to better performance at the fine-tuning stage, especially in the ad-hoc retrieval area. Existing pre-training approaches tailored for IR tried to incorporate weak supervised signals, such as query-likelihood based sampling, to construct pseudo query-document pairs from the raw textual corpus. However, these signals rely heavily on the sampling method. For example, the query likelihood model may lead to much noise in the constructed pre-training data. In this paper, we propose to leverage the large-scale hyperlinks and anchor texts to pre-train the language model for ad-hoc retrieval. Since the anchor texts are created by webmasters and can usually summarize the target document, it can help to build more accurate and reliable pre-training samples than a specific algorithm. Considering different views of the downstream ad-hoc retrieval, we devise four pre-training tasks based on the hyperlinks. We then pre-train the Transformer model to predict the pair-wise preference, jointly with the Masked Language Model objective. Experimental results on two large-scale ad-hoc retrieval datasets show the significant improvement of our model compared with the existing methods. Copyright © 2021, The Authors. All rights reserved.

关键词： Hypertext systems

Block-Structured Optimization for Subgraph Detection in Interdependent Networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jie, Fei Wang, Chunpai Chen, Feng Li, Lei Wu, Xindong Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology Ministry of Education Hefei China School of Computer Science and Information Engineering Hefei University of Technology Hefei China Department of Computer Science University at Albany – SUNY AlbanyNY United States Erik Jonsson School of Engineering & Computer Science The University of Texas at Dallas DallasTX United States Mininglamp Academy of Sciences Mininglamp Technologies Beijing China Institute of Big Knowledge Science Hefei University of Technology Hefei China

We propose a generalized framework for block-structured nonconvex optimization, which can be applied to structured subgraph detection in interdependent networks, such as multi-layer networks, temporal networks, networks of networks, and many others. Specifically, we design an effective, efficient, and parallelizable projection algorithm, namely Graph Block-structured Gradient Projection (GBGP), to optimize a general non-linear function subject to graph-structured constraints. We prove that our algorithm: 1) runs in nearly-linear time on the network size;2) enjoys a theoretical approximation guarantee. Moreover, we demonstrate how our framework can be applied to two very practical applications and conduct comprehensive experiments to show the effectiveness and efficiency of our proposed algorithm. Copyright © 2022, The Authors. All rights reserved.

关键词： Functions

Group based personalized search by integrating search behaviour and friend network

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhou, Yujia Dou, Zhicheng Wei, Bingzheng Xie, Ruobing Wen, Ji-Rong Gaoling School of Artificial Intelligence Renmin University of China School of Information Renmin University of China WeChat Search Application Department Tencent China Beijing Key Laboratory of Big Data Management and Analysis Methods Key Laboratory of Data Engineering and Knowledge Engineering MOE

The key to personalized search is to build the user profile based on historical behaviour. To deal with the users who lack historical data, group based personalized models were proposed to incorporate the profiles of similar users when re-ranking the results. However, similar users are mostly found based on simple lexical or topical similarity in search behaviours. In this paper, we propose a neural network enhanced method to highlight similar users in semantic space. Furthermore, we argue that the behaviour-based similar users are still insufficient to understand a new query when user's historical activities are limited. To tackle this issue, we introduce the friend network into personalized search to determine the closeness between users in another way. Since the friendship is often formed based on similar background or interest, there are plenty of personalized signals hidden in the friend network naturally. Specifically, we propose a friend network enhanced personalized search model, which groups the user into multiple friend circles based on search behaviours and friend relations respectively. These two types of friend circles are complementary to construct a more comprehensive group profile for refining the personalization. Experimental results show the significant improvement of our model over existing personalized search models. Copyright © 2021, The Authors. All rights reserved.

关键词： User profile

Improving Generalization of Neural Vehicle Routing Problem Solvers Through the Lens of Model Architecture

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xiao, Yubin Wang, Di Wu, Xuan Wu, Yuesong Li, Boyang Du, Wei Wang, Liupu Zhou, You Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education College of Computer Science and Technology Jilin University Changchun130012 China Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly Nanyang Technological University 639956 Singapore WeBank-NTU Joint Research Institute on Fintech Nanyang Technological University 639956 Singapore College of Computing and Data Science Nanyang Technological University 639956 Singapore

Neural models produce promising results when solving Vehicle Routing Problems (VRPs), but may often fall short in generalization. Recent attempts to enhance model generalization often incur unnecessarily large training cost or cannot be directly applied to other models solving different VRP variants. To address these issues, we take a novel perspective on model architecture in this study. Specifically, we propose a plug-and-play Entropy-based Scaling Factor (ESF) and a Distribution-Specific (DS) decoder to enhance the size and distribution generalization, respectively. ESF adjusts the attention weight pattern of the model towards familiar ones discovered during training when solving VRPs of varying sizes. The DS decoder explicitly models VRPs of multiple training distribution patterns through multiple auxiliary light decoders, expanding the model representation space to encompass a broader range of distributional scenarios. We conduct extensive experiments on both synthetic and widely recognized real-world benchmarking datasets and compare the performance with seven baseline models. The results demonstrate the effectiveness of using ESF and DS decoder to obtain a more generalizable model and showcase their applicability to solve different VRP variants, i.e., travelling salesman problem and capacitated VRP. Notably, our proposed generic components require minimal computational resources, and can be effortlessly integrated into conventional generalization strategies to further elevate model generalization. Copyright © 2024, The Authors. All rights reserved.

关键词： Combinatorial optimization

UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Saichao Sun, Geng Li, Jiahui Liang, Shuang Wu, Qingqing Wang, Pengfei Niyato, Dusit The College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University Changchun130012 China The College of Computing and Data Science Nanyang Technological University 639798 Singapore The School of Information Science and Technology Northeast Normal University Changchun130117 China The Department of Electronic Engineering Shanghai Jiao Tong University Shanghai200240 China The School of Computer Science and Technology Dalian University of Technology Dalian116024 China

In this paper, we investigate an unmanned aerial vehicle (UAV)-assistant air-to-ground communication system, where multiple UAVs form a UAV-enabled virtual antenna array (UVAA) to communicate with remote base stations by utilizing collaborative beamforming. To improve the work efficiency of the UVAA, we formulate a UAV-enabled collaborative beamforming multi-objective optimization problem (UCBMOP) to simultaneously maximize the transmission rate of the UVAA and minimize the energy consumption of all UAVs by optimizing the positions and excitation current weights of all UAVs. This problem is challenging because these two optimization objectives conflict with each other, and they are non-concave to the optimization variables. Moreover, the system is dynamic, and the cooperation among UAVs is complex, making traditional methods take much time to compute the optimization solution for a single task. In addition, as the task changes, the previously obtained solution will become obsolete and invalid. To handle these issues, we leverage the multi-agent deep reinforcement learning (MADRL) to address the UCBMOP. Specifically, we use the heterogeneous-agent trust region policy optimization (HATRPO) as the basic framework, and then propose an improved HATRPO algorithm, namely HATRPO-UCB, where three techniques are introduced to enhance the performance. Simulation results demonstrate that the proposed algorithm can learn a better strategy compared with other methods. Moreover, extensive experiments also demonstrate the effectiveness of the proposed techniques. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

Multiagent Q-Learning for Multicrew Dynamic Scheduling and Routing in Road Network Restoration

学校读者我要写书评

暂无评论

Multiagent Q-Learning for Multicrew Dynamic Scheduling and R...

第34届中国控制与决策会议

作者： Yufeng Shen Lianqi Duan Qianshun Zhu Zhaopin Su Guofu Zhang School of Computer Science and Information Engineering Hefei University of Technology Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology) Ministry of Education Intelligent Interconnected Systems Laboratory of Anhui Province (Hefei University of Technology) Anhui Province Key Laboratory of Industry Safety and Emergency Technology (Hefei University of Technology)

Road network restoration is an important issue in the post-disaster disposal and rescue,especially when extraordinarily serious natural disasters(e.g.,floods and earthquakes) *** to this endeavour is the problem of determining how to reasonably schedule and route the repair crew to quickly restore the damaged road network and establish reliable supply lines from supply nodes to demand ***,most existing work focuses on the activities of the single repair crew in the static road network,which is unable to adapt to the dynamic changes of the road network caused by secondary disasters,such as rock slides and debris ***,this work is concentrated on multicrew dynamic scheduling and routing in road network ***,a model of multicrew dynamic scheduling and routing is first ***,the Markov decision process is adopted to construct a decision model for repair crews,based on which a multiagent Q-learning algorithm is developed to find the best scheduling and routing ***,experimental results demonstrate that the proposed method can make repair crews adjust their scheduling and routing strategies according to the dynamic changes of the damaged road network and provides a useful attempt to restore the damaged road network in complex,dynamic emergency scenarios of post-disaster.

关键词： Road Network Restoration Multiple Repair Crews Dynamic Scheduling and Routing Multiagent Q-Learning

Multi-fuzzy-constrained graph pattern matching with big graph data

学校读者我要写书评

暂无评论

Intelligent data Analysis 2020年第4期24卷 941-958页

作者： Liu, Guliu Li, Lei Wu, Xindong Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology) Ministry of Education Hefei Anhui 230009 China School of Computer Science and Information Engineering Hefei University of Technology Hefei Anhui China Mininglamp Academy of Sciences Mininglamp Technologies Beijing China

Graph pattern matching has been widespread used for protein structure analysis, expert finding and social group selection, ect. Recently, the study of graph pattern matching using the abundant attribute information of vertices and edges as constraint conditions has attracted the attention of scholars, and multi-constrained simulation has been proposed to address the problem in contextual social networks. Actually, multi-constrained graph pattern matching is an NP-complete problem and the fuzziness of constraint variables may exist in many applications. In this paper, we introduce a multi-fuzzy-constrained graph pattern matching problem in big graph data, and propose an efficient first-k algorithm Fuzzy-ETOF-K for solving it. Specifically, exploration-based method based on edge topology is adopted to improve the efficiency of edge connection, and breadth-first bounded search is used for edge matching instead of shortest path query between two nodes to improve the efficiency of edge matching. The results of our experiments conducted on three datasets of real social networks illustrate that our proposed algorithm Fuzzy-ETOF-K significantly outperforms existing approaches in efficiency and the introduction of fuzzy constraints makes our proposed algorithm more efficient and effective. © 2020 - IOS Press and the authors. All rights reserved.

关键词： big graph data Graph pattern matching multi-fuzzy-constrained

A new steganography without embedding based on adversarial training 20

学校读者我要写书评

暂无评论

A new steganography without embedding based on adversarial t...

2020 ACM Turing Celebration Conference - China, ACM TURC 2020

作者： Jiang, Wenjie Hu, Donghui Yu, Cong Li, Meng Zhao, Zhong-Qiu Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology Ministry of Education and College of Computer Science and Information Engineering !!!Hefei University of Technology Hefei China College of Computer Science and Information Engineering Hefei University of Technology Hefei230601 China

ISBN: (纸本)9781450375344

Steganography is an art to hide information in the carriers to prevent from being detected, while steganalysis is the opposite art to detect the presence of the hidden information. With the development of deep learning, several state-of-the-art steganography and steganalysis based on deep learning techniques have been proposed to improve hiding or detection capabilities. Generative Adversarial Networks (GANs) based steganography directly uses the minimax game between the generator and discriminator, to automatically generate steganography algorithms resisting being detected by powerful steganalysis. The steganography without embedding (SWE) based on GANs, where the generated cover images themselves are stego ones carrying secret information has shown its state-of-the-art steganography performance. However, SWE based on GANs has serious weaknesses, such as low information recovery accuracy, low steganography capacity and poor natural showing. To solve these problems, this paper proposes a new SWE based on adversarial training, with carefully designed generator, discriminator and extractor, as well as their loss functions and optimized training mode. The proposed method can achieve a very high information recovery accuracy (100% in some cases), and at the same time improve the steganography capacity and image quality. © 2020 ACM.

关键词： Steganography