检索结果-内蒙古大学图书馆

Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods

IEEE Transactions on Neural Networks and Learning Systems 2024年第6期36卷 9737-9757页

作者： Yuji Cao Huan Zhao Yuheng Cheng Ting Shu Yue Chen Guolong Liu Gaoqi Liang Junhua Zhao Jinyue Yan Yun Li Department of Mechanical and Automation Engineering The Chinese University of Hong Kong Hong Kong SAR China Department of Building Environment and Energy Engineering The Hong Kong Polytechnic University Hong Kong China School of Science and Engineering The Chinese University of Hong Kong Shenzhen China Center for Crowd Intelligence Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS) Shenzhen China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen China School of Electrical and Electronic Engineering Nanyang Technological University Jurong West Singapore School of Mechanical Engineering and Automation Harbin Institute of Technology Shenzhen China Shenzhen Institute for Advanced Study University of Electronic Science and Technology of China Shenzhen China i4AI Ltd. London U.K.

With extensive pretrained knowledge and high-level general capabilities, large language models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in aspects, such as multitask learning, sample efficiency, and high-level task planning. In this survey, we provide a comprehensive review of the existing literature in LLM-enhanced RL and summarize its characteristics compared with conventional RL methods, aiming to clarify the research scope and directions for future studies. Utilizing the classical agent-environment interaction paradigm, we propose a structured taxonomy to systematically categorize LLMs’ functionalities in RL, including four roles: information processor, reward designer, decision-maker, and generator. For each role, we summarize the methodologies, analyze the specific RL challenges that are mitigated and provide insights into future directions. Finally, the comparative analysis of each role, potential applications, prospective opportunities, and challenges of the LLM-enhanced RL are discussed. By proposing this taxonomy, we aim to provide a framework for researchers to effectively leverage LLMs in the RL field, potentially accelerating RL applications in complex applications, such as robotics, autonomous driving, and energy systems.

关键词： Taxonomy Surveys Visualization Reviews Planning Reinforcement learning Games Transformers Natural language processing Large language models

来源：评论

学校读者我要写书评

暂无评论

Incorporating Multiple Features to Predict Bug Fixing Time with Neural Networks

Incorporating Multiple Features to Predict Bug Fixing Time w...

引用

International Conference on Software Maintenance (ICSM)

作者： Wei Yuan Yuan Xiong Hailong Sun Xudong Liu SKLSDE Lab School of Computer Science and Engineering Beihang University Beijing China Beijing Advanced Innovation Center for Big Data and Brain Computing Beijing China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China SKLSDE Lab School of Software Beihang University Beijing China

Debugging is a well-known time-consuming task, and knowing how long it would take to resolve bugs is of great importance for allocating the limited resources in a software development team. However, it is challenging to predict bug fixing time since fixing bugs is subject to a plethora of uncertain factors such as types of bugs, program complexity and developers' abilities. Existing work mainly focuses on developers' activities in a bug lifecycle and ignores other important factors. In light of the limitations of existing work, we propose a novel approach to predicting the bug fixing time by incorporating a comprehensive set of relevant features. Specifically, we consider four types of features including developers' activities, developers' sentiments, semantics of bugs, and efforts caused by understanding and analyzing source code, and design particular neural networks to take advantage of these features and make them work efficiently. Experimental results on four real application datasets demonstrate that on the one hand, our approach outperforms the state-of-the-art by over 5% in accuracy and 7.3% in F1-score on average; on the other hand, each type of the features considered in our approach plays an important part in the prediction.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Deep Active Contour Network for Medical Image Segmentation 23rd

Deep Active Contour Network for Medical Image Segmentation

引用

23rd International Conference on Medical Image computing and Computer-Assisted Intervention, MICCAI 2020

作者： Zhang, Mo Dong, Bin Li, Quanzheng Center for Data Science Peking University Beijing100871 China Center for Data Science in Health and Medicine Peking University Beijing100871 China Laboratory for Biomedical Image Analysis Beijing Institute of Big Data Research Beijing100871 China Peking University Beijing100871 China Institute for Artificial Intelligence Peking University Beijing100871 China Department of Radiology Center for Advanced Medical Computing and Analysis MGH/BWH Center for Clinical Data Science Massachusetts General Hospital Harvard Medical School BostonMA02115 United States

ISBN: (纸本)9783030597184

Image segmentation is vital to medical image analysis and clinical diagnosis. Recently, convolutional neural networks (CNNs) have achieved tremendous success in this task, however, it performs poorly at recognizing precise object boundary due to the information loss in the successive downsampling layers. To overcome this problem, we integrate an active contour model (convexified Chan-Vese model) into the CNN structure (DenseUNet), forming a new framework called deep active contour network (DACN). Instead of manual setting, DACN applies a CNN backbone to learn the initialization and parameters of active contour model (ACM) automatically. The proposed DACN leverages the advantage of ACM to detect object boundaries accurately, which can be trained in an end-to-end differential manner. The experimental results on two public datasets demonstrate the effectiveness of DACN, and the trimap experiment confirms the superior ability of DACN to obtain precise boundary delineation. © 2020, Springer Nature Switzerland AG.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

Current switching of the antiferromagnetic Néel vector in Pd/CoO/MgO(001)

引用

Physical Review B 2022年第21期106卷 214405-214405页

作者： M. Yang Q. Li T. Wang B. Hong C. Klewe Z. Li X. Huang P. Shafer F. Zhang C. Hwang W. S. Yan R. Ramesh W. S. Zhao Y. Z. Wu Xixiang Zhang Z. Q. Qiu Institute of Physical Science and Information Technology Anhui University Hefei Anhui 230601 China National Synchrotron Radiation Laboratory University of Science and Technology of China Hefei Anhui 230029 China Department of Physics University of California Berkeley California 94720 USA Fert Beijing Research Institute School of Integrated Circuit Science and Engineering Beijing Advanced Innovation Center for Big Data and Brain Computing Beihang University Beijing China Advanced Light Source Lawrence Berkeley National Laboratory Berkeley California 94720 USA Department of Materials Science and Engineering University of California Berkeley California 94720 USA Korea Research Institute of Standards and Science Yuseong Daejeon 305-340 Korea Department of Physics State Key Laboratory of Surface Physics Fudan University Shanghai 200433 China Physical Science and Engineering Division King Abdullah University of Science and Technology Thuwal 23955-6900 Saudi Arabia

Recently, the electrical switching of antiferromagnetic (AFM) order has been intensively investigated because of its application potential in data storage technology. Herein, we report the current switching of the AFM Néel vector in epitaxial Pd/CoO films as a function of temperature. Using combined measurements of Hall resistance (HR) and x-ray magnetic linear dichroism (XMLD) below and above the AFM Néel temperature, we unambiguously identified both magnetic and nonmagnetic contributions to the current-induced HR change. Through magnetic field-induced HR measurements, we quantitatively determined the percentage of current-induced CoO spin switching. Further, we showed that the thermal effect dominated the CoO magnetic switching more in samples with a thinner Pd layer and that samples with a thicker CoO layer required higher thermal activation for current-induced magnetic switching. These results provide a clear and comprehensive picture of current-induced AFM spin switching across the AFM Néel temperature.

关键词： Hall effect Magnetism Spintronics X-ray magnetic linear dichroism

来源：评论

学校读者我要写书评

暂无评论

Holographic capture and projection system of real object based on tunable zoom lens

引用

PhotoniX 2020年第1期1卷 271-285页

作者： Di Wang Chao Liu Chuan Shen Yan Xing Qiong-Hua Wang School of Instrumentation and Optoelectronic Engineering Beihang UniversityBeijing 100191China Beijing Advanced Innovation Center for Big Data-based Precision Medicine Beihang UniversityBeijing 100191China Key Laboratory of Intelligent Computing&Signal Processing Ministry of EducationAnhui UniversityHefei 230039China.

In this paper, we propose a holographic capture and projection system of real objectsbased on tunable zoom lenses. Different from the traditional holographic system, aliquid lens-based zoom camera and a digital conical lens are used as key parts to reachthe functions of holographic capture and projection, respectively. The zoom camera isproduced by combing liquid lenses and solid lenses, which has the advantages of fastresponse and light weight. By electrically controlling the curvature of the liquid-liquidsurface, the focal length of the zoom camera can be changed easily. As anothertunable zoom lens, the digital conical lens has a large focal depth and the opticalproperty is perfectly used in the holographic system for adaptive projection, especiallyfor multilayer imaging. By loading the phase of the conical lens on the spatial lightmodulator, the reconstructed image can be projected with large depths. With theproposed system, holographic zoom capture and color reproduction of real objects canbe achieved based on a simple structure. Experimental results verify the feasibility ofthe proposed system. The proposed system is expected to be applied to microprojectionand three-dimensional display technology.

关键词： Spatial light modulator Holographic system Zoom lenses Display technology

来源：评论

学校读者我要写书评

暂无评论

Wireless Acoustic Sensor Networks and Edge computing for Rapid Acoustic Monitoring

引用

IEEE/CAA Journal of Automatica Sinica 2019年第1期6卷 64-74页

作者： Zhengguo Sheng Saskia Pfersich Alice Eldridge Jianshan Zhou Daxin Tian Victor C.M.Leung the Department of Engineering and Design University of Sussex Infineon Technologies AG the Department of Music University of Sussex Beijing Advanced Innovation Center for Big Data and Brain Computing (BDBC) Beijing Key Laboratory for Cooperative Vehicle Infrastructure Systems & Safety ControlSchool of Transportation Science and EngineeringBeihang University the Department of Electrical and Computer Engineering The University of British Columbia

Passive acoustic monitoring is emerging as a promising solution to the urgent, global need for new biodiversity assessment methods. The ecological relevance of the soundscape is increasingly recognised, and the affordability of robust hardware for remote audio recording is stimulating international interest in the potential for acoustic methods for biodiversity *** scale of the data involved requires automated methods,however, the development of acoustic sensor networks capable of sampling the soundscape across time and space and relaying the data to an accessible storage location remains a significant technical challenge, with power management at its core. Recording and transmitting large quantities of audio data is power intensive,hampering long-term deployment in remote, off-grid locations of key ecological interest. Rather than transmitting heavy audio data, in this paper, we propose a low-cost and energy efficient wireless acoustic sensor network integrated with edge computing structure for remote acoustic monitoring and in situ *** and computation of acoustic indices are carried out directly on edge devices built from low noise primo condenser microphones and Teensy microcontrollers, using internal FFT hardware support. Resultant indices are transmitted over a ZigBee-based wireless mesh network to a destination *** tests of audio quality, indices computation and power consumption demonstrate acoustic equivalence and significant power savings over current solutions.

关键词： Acoustic sensor networks edge computing energy efficiency

来源：评论

学校读者我要写书评

暂无评论

STRATLEARN-Z: IMPROVED PHOTO-Z ESTIMATION FROM SPECTROSCOPIC data SUBJECT TO SELECTION EFFECTS

arXiv

引用

arXiv 2024年

作者： Moretti, Chiara Autenrieth, Maximilian Serra, Riccardo Trotta, Roberto van Dyk, David A. Mesinger, Andrei SISSA - International School for Advanced Studies Via Bonomea 265 Trieste34136 Italy Centro Nazionale "High Performance Computer Big Data and Quantum Computing" INAF – Osservatorio Astronomico di Trieste Via Tiepolo 11 TriesteI-34143 Italy INFN sezione di Trieste Italy Statistic Section Department of Mathematics Imperial College London 180 Queen’s Gate LondonSW7 2AZ United Kingdom Department of Physics Imperial College London Blackett Laboratory Prince Consort Rd LondonSW7 2AZ United Kingdom Piazza dei Cavalieri 7 PI Pisa56125 Italy

A precise measurement of photometric redshifts (photo-z) is crucial for the success of modern photometric galaxy surveys. Machine learning (ML) methods show great promise in this context, but suffer from covariate shift in training sets due to selection bias where interesting sources, e.g., high redshift objects, are underrepresented, and the corresponding ML models exhibit poor generalisation properties. We present an application of the StratLearn method to the estimation of photo-z (StratLearn-z), validating against simulations where we enforce the presence of covariate shift to different degrees. StratLearn is a statistically principled approach which relies on splitting the combined source and target datasets into strata, based on estimated propensity scores. The latter is the probability for an object in the dataset to be in the source set, given its observed covariates. After stratification, two conditional density estimators are fit separately within each stratum, and then combined via a weighted average. We benchmark our results against the GPz algorithm, quantifying the performance of the two algorithms with a set of metrics. Our results show that the StratLearn-z metrics are only marginally affected by the presence of covariate shift, while GPz shows a significant degradation of performance, specifically concerning the photo-z prediction for fainter objects for which there is little training data. In particular, for the strongest covariate shift scenario considered, StratLearn-z yields a reduced fraction of catastrophic errors, a factor of 2 improvement for the RMSE as well as one order of magnitude improvement on the bias. We also assess the quality of the predicted conditional redshift estimates using the probability integral transform (PIT). The PIT distribution obtained from StratLearn-z features far fewer outliers and is symmetric, i.e., the predictions appear to be well-centered around the true redshift value, despite showing a rather conservative es

关键词： Risk assessment

来源：评论

学校读者我要写书评

暂无评论

Embedding dynamic attributed networks by modeling the evolution processes

arXiv

引用

arXiv 2020年

作者： Xu, Zenan Ou, Zijing Su, Qinliang Yu, Jianxing Quan, Xiaojun Lin, Zhenkun School of Data and Computer Science Sun Yat-sen University China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Key Lab. of Machine Intelligence and Advanced Computing Ministry of Education China

Network embedding has recently emerged as a promising technique to embed nodes of a network into low-dimensional vectors. While fairly successful, most existing works focus on the embedding techniques for static networks. But in practice, there are many networks that are evolving over time and hence are dynamic, e.g., the social networks. To address this issue, a high-order spatio-temporal embedding model is developed to track the evolutions of dynamic networks. Specifically, an activeness-aware neighborhood embedding method is first proposed to extract the high-order neighborhood information at each given timestamp. Then, an embedding prediction framework is further developed to capture the temporal correlations, in which the attention mechanism is employed instead of recurrent neural networks (RNNs) for its efficiency in computing and flexibility in modeling. Extensive experiments are conducted on four real-world datasets from three different areas. It is shown that the proposed method outperforms all the baselines by a substantial margin for the tasks of dynamic link prediction and node classification, which demonstrates the effectiveness of the proposed methods on tracking the evolutions of dynamic networks. Copyright © 2020, The Authors. All rights reserved.

关键词： Recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

Understanding collective behaviors in reinforcement learning evolutionary games via a belief-based formalization

引用

Physical Review E 2020年第4期101卷 042402-042402页

作者： Ji-Qiang Zhang Si-Ping Zhang Li Chen Xu-Dong Liu Beijing Advanced Innovation Center for Big Data and Brain Computing School of Comuter Science and Engineering Beihang University Beijing 100191 China The Key Laboratory of Biomedical Information Engineering of Ministry of Education The Key Laboratory of Neuro-informatics & Rehabilitation Engineering of Ministry of Civil Affairs and Institute of Health and Rehabilitation Science School of Life Science and Technology Xi'an Jiaotong University Xi'an 710049 China School of Physics and Information Technology Shaanxi Normal University Xi'an 710062 China Beijing Advanced Innovation Center for Big Data and Brain Computing School of Computer Science and Engineering Beihang University Beijing 100191 China

Collective behaviors by self-organization are ubiquitous in nature and human society and extensive efforts have been made to explore the mechanisms behind them. Artificial intelligence (AI) as a rapidly developing field is of great potential for these tasks. By combining reinforcement learning with evolutionary game (RLEG), we numerically discover a rich spectrum of collective behaviors—explosive events, oscillation, and stable states, etc., that are also often observed in the human society. In this work, we aim to provide a theoretical framework to investigate the RLEGs systematically. Specifically, we formalize AI-agents' learning processes in terms of belief switches and behavior modes defined as a series of actions following beliefs. Based on the preliminary results in the time-independent environment, we investigate the stability at the mixed equilibrium points in RLEGs generally, in which agents reside in one of the optimal behavior modes. Moreover, we adopt the maximum entropy principle to infer the composition of agents residing in each mode at a strictly stable point. When the theoretical analysis is applied to the 2×2 game setting, we can explain the uncovered collective behaviors and are able to construct equivalent systems intuitively. Also, the inferred composition of different modes is consistent with simulations. Our work may be helpful to understand the related collective emergence in human society as well as behavioral patterns at the individual level and potentially facilitate human-computer interactions in the future.

关键词： Collective dynamics Complex systems Evolutionary dynamics

来源：评论

学校读者我要写书评

暂无评论

Hyperbolic variational graph neural network for modeling dynamic graphs

arXiv

引用

arXiv 2021年

作者： Sun, Li Zhang, Zhongbao Zhang, Jiawei Wang, Feiyang Peng, Hao Su, Sen Yu, Philip S. State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications China IFM Lab Department of Computer Science Florida State University FL United States Beijing Advanced Innovation Center for Big Data and Brain Computing Beihang University China Department of Computer Science University of Illinois ChicagoIL United States

Learning representations for graphs plays a critical role in a wide spectrum of downstream applications. In this paper, we summarize the limitations of the prior works in three folds: representation space, modeling dynamics and modeling uncertainty. To bridge this gap, we propose to learn dynamic graph representation in hyperbolic space, for the first time, which aims to infer stochastic node representations. Working with hyperbolic space, we present a novel Hyperbolic Variational Graph Neural Network, referred to as HVGNN. In particular, to model the dynamics, we introduce a Temporal GNN (TGNN) based on a theoretically grounded time encoding approach. To model the uncertainty, we devise a hyperbolic graph variational autoencoder built upon the proposed TGNN to generate stochastic node representations of hyperbolic normal distributions. Furthermore, we introduce a reparameterisable sampling algorithm for the hyperbolic normal distribution to enable the gradient-based learning of HVGNN. Extensive experiments show that HVGNN outperforms state-of-the-art baselines on real-world datasets. © 2021, CC BY-NC-SA.

关键词： Normal distribution

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：