检索结果-内蒙古大学图书馆

A Reinforcement Learning Approach to Autonomous Decision Making of Intelligent Vehicles on Highways

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2020年第10期50卷 3884-3897页

作者： Xu, Xin Zuo, Lei Li, Xin Qian, Lilin Ren, Junkai Sun, Zhenping Natl Univ Def Technol Inst Unmanned Syst Coll Intelligence Sci & Technol Changsha 410073 Peoples R China

Autonomous decision making is a critical and difficult task for intelligent vehicles in dynamic transportation environments. In this paper, a reinforcement learning approach with value function approximation and feature learning is proposed for autonomous decision making of intelligent vehicles on highways. In the proposed approach, the sequential decision making problem for lane changing and overtaking is modeled as a Markov decision process with multiple goals, including safety, speediness, smoothness, etc. In order to learn optimized policies for autonomous decision-making, a multiobjective approximate policy iteration (MO-API) algorithm is presented. The features for value function approximation are learned in a data-driven way, where sparse kernel-based features or manifold-based features can be constructed based on data samples. Compared with previous RL algorithms such as multiobjective Q-learning, the MO-API approach uses data-driven feature representation for value and policy approximation so that better learning efficiency can be achieved. A highway simulation environment using a 14 degree-of-freedom vehicle dynamics model was established to generate training data and test the performance of different decision-making methods for intelligent vehicles on highways. The results illustrate the advantages of the proposed MO-API method under different traffic conditions. Furthermore, we also tested the learned decision policy on a real autonomous vehicle to implement overtaking decision and control under normal traffic on highways. The experimental results also demonstrate the effectiveness of the proposed method.

关键词： Autonomous decision-making intelligent driving vehicles Markov decision processes (MDPs) multiobjective,reinforcement learning (RL) value function approximation

来源：评论

学校读者我要写书评

暂无评论

Transfer learning via inter-task mappings for temporal difference learning

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2007年第9期8卷 2125-2167页

作者： Taylor, Matthew E. Stone, Peter Liu, Yaxin Univ Texas Austin Dept Comp Sci Austin TX 78712 USA

Temporal difference (TD) learning (Sutton and Barto, 1998) has become a popular reinforcement learning technique in recent years. TD methods, relying on function approximators to generalize learning to novel situations, have had some experimental successes and have been shown to exhibit some desirable properties in theory, but the most basic algorithms have often been found slow in practice. This empirical result has motivated the development of many methods that speed up reinforcement learning by modifying a task for the learner or helping the learner better generalize to novel situations. This article focuses on generalizing across tasks, thereby speeding up learning, via a novel form of transfer using handcoded task relationships. We compare learning on a complex task with three function approximators, a cerebellar model arithmetic computer (CMAC), an artificial neural network ( ANN), and a radial basis function (RBF), and empirically demonstrate that directly transferring the action-value function can lead to a dramatic speedup in learning with all three. Using transfer via inter-task mapping (TVITM), agents are able to learn one task and then markedly reduce the time it takes to learn a more complex task. Our algorithms are fully implemented and tested in the RoboCup soccer Keepaway domain. This article contains and extends material published in two conference papers (Taylor and Stone, 2005;Taylor et al., 2005).

关键词： transfer learning reinforcement learning temporal difference methods value function approximation inter-task mapping

来源：评论

学校读者我要写书评

暂无评论

Two-Stream Fused Fuzzy Deep Neural Network for Multiagent Learning

引用

IEEE TRANSACTIONS ON FUZZY SYSTEMS 2023年第2期31卷 511-520页

作者： Fang, Baofu Zheng, Caiming Wang, Hao Yu, Tingting Hefei Univ Technol Sch Comp Sci & Informat Engn Hefei 230601 Peoples R China

In multiagent reinforcement learning (RL), multilayer fully connected neural network is used for value function approximation, which solves large-scale or continuous space problems. However, it is easy to fall into a local optimal and overfitting under partially observed environments. Because each agent lacks the information that plays a key role in decision making beyond the observation field. Even if communication is allowed, the received informations in communication channel have large noise due to the observations of other agents and strong uncertainty if the agents policy is used as the communication information. To tackle this problem, two-stream fused fuzzy deep neural network (2s-FDNN) was proposed to reduce the uncertainty and noise of information in the communication channel. It is a parallel structure in which the fuzzy inference module reduces the uncertainty of information and the deep neural module reduces the noise of information. Then, we presented Fuzzy MA2C which integrates 2s-FDNN into multiagent deep RL to deal with uncertain communication informations for improving the robustness and generalization under partially observed environments. We empirically evaluate our methods in two large-scale traffic signal control environments using simulation of urban mobility (SUMO) simulator. Results demonstrate that our methods can achieve superior performance against existing RL algorithms.

关键词： Fuzzy logic Neural networks Deep learning Uncertainty Fuzzy neural networks Feature extraction Q-learning Deep reinforcement learning (RL) fuzzy logic inference multiagent systems traffic signal control value function approximation

来源：评论

学校读者我要写书评

暂无评论

Chaotic dynamics and convergence analysis of temporal difference algorithms with bang-bang control

引用

OPTIMAL CONTROL APPLICATIONS & METHODS 2016年第1期37卷 108-126页

作者： Tutsoy, Onder Brown, Martin Univ Manchester Control Syst Ctr Sch Elect & Elect Engn Manchester M13 9PL Lancs England Adana Sci & Technol Univ Elect & Elect Engn Dept Seyhan Adana Turkey

Reinforcement learning is a powerful tool used to obtain optimal control solutions for complex and difficult sequential decision making problems where only a minimal amount of a priori knowledge exists about the system dynamics. As such, it has also been used as a model of cognitive learning in humans and applied to systems, such as humanoid robots, to study embodied cognition. In this paper, a different approach is taken where a simple test problem is used to investigate issues associated with the value function's representation and parametric convergence. In particular, the terminal convergence problem is analyzed with a known optimal control policy where the aim is to accurately learn the value function. For certain initial conditions, the value function is explicitly calculated and it is shown to have a polynomial form. It is parameterized by terms that are functions of the unknown plant's parameters and the value function's discount factor, and their convergence properties are analyzed. It is shown that the temporal difference error introduces a null space associated with the finite horizon basis function during the experiment. The learning problem is only non-singular when the experiment termination is handled correctly and a number of (equivalent) solutions are described. Finally, it is demonstrated that, in general, the test problem's dynamics are chaotic for random initial states and this causes digital offset in the value function learning. The offset is calculated, and a dead zone is defined to switch off learning in the chaotic region. Copyright (C) 2015 John Wiley & Sons, Ltd.

关键词： badly conditioned learning polynomial basis functions rate of convergence temporal difference learning value function approximation

来源：评论

学校读者我要写书评

暂无评论

Gaussian Process Approximate Dynamic Programming for Energy-Optimal Supervisory Control of Parallel Hybrid Electric Vehicles

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2022年第8期71卷 8367-8380页

作者： Bae, Jin Woo Kim, Kwang-Ki K. Texas A&M Univ Dept Comp Sci & Engn College Stn TX 77843 USA Inha Univ Dept Elect & Comp Engn Incheon 22212 South Korea

We propose an energy-efficient supervisory control method for the power management of parallel hybrid electric vehicles (HEVs) to improve the fuel economy and reduce exhaust gas emissions. Plug-in HEVs ((P)HEVs) have multiple power sources (e.g., an engine and motor) that should be cooperatively operated to meet the required instantaneous traction power for the desired vehicle speed while satisfying their physical limits. Because the efficiencies of the engine and motor vary with different operating speeds and torques, the main issue of energy-efficient power management is to allocate the power demand among the power sources by achieving maximum power conversion efficiencies and satisfy the operating limits. For an efficient power allocation, an optimal control problem is formulated, and a global solution is found through deterministic dynamic programming (DP). Owing to the curse of dimensionality and uncertainties in real driving, DP solutions are not directly applicable in real time. To resolve the limitations of DP, we employ a non-parametric Bayesian function approximation technique using a Gaussian process (GP). The offline DP solutions obtained from a set of real vehicle driving test data were used to learn a state-dependent probabilistic value function through Gaussian process regression. For online implementations, a receding horizon control scheme was applied for the feedback control of the power management. In comparison with the existing charge sustaining strategy and charge depleting and charge sustaining mixed controllers, we recorded fuel efficiency improvements of over 4.8% and 7.3%, respectively, in a mixed urban-suburban route.

关键词： Batteries Vehicle dynamics Mathematical models Hybrid electric vehicles Energy management Optimal control Engines Approximate dynamic programming energy management gaussian process regression optimal control Parallel hybrid electric vehicles supervisory control value function approximation

来源：评论

学校读者我要写书评

暂无评论

Time-of-Day Pricing in Taxi Markets

引用

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2017年第6期18卷 1610-1622页

作者： Qian, Xinwu Ukkusuri, Satish V. Purdue Univ Dept Civil Engn W Lafayette IN 47906 USA

For a regular weekday in New York City, the number of taxi trips at 8 P. M. may be 10 times greater than that at 5 A. M., while passengers are charged under the same pricing scheme. Motivated by temporally non-stationary demand and supply in the taxi market, the time-of-day (TOD) pricing scheme for taxi industry is framed to vary trip cost dynamically over time, so that total market revenue is maximized. Temporal market dynamics is modeled as a semi-Markov process, which captures leftover of drivers, spillover of passengers, and restoration of drivers in service along the time horizon. The TOD pricing scheme is therefore formulated as discrete time stochastic dynamic programming with the goal to find the optimal sequence of price multipliers. The approximate dynamic programming (ADP) approach is introduced to solve the curse of dimensionality. Numerical experiments are conducted using New York City taxi trip data to illustrate the effectiveness of TOD price in real-world taxi market. The results suggest that TOD price may increase daily market revenue by over 10% using the ADP approach. Our experiments also show that TOD price may be even more effective if sudden surges in demand take place in the market.

关键词： Time-of-day pricing market dynamics revenue maximization value function approximation daily operation surge demand

来源：评论

学校读者我要写书评

暂无评论

A Q-learning algorithm for task scheduling based on improved SVM in wireless sensor networks

引用

COMPUTER NETWORKS 2019年 161卷 138-149页

作者： Wei, Zhenchun Liu, Fei Zhang, Yan Xu, Juan Ji, Jianjun Lyu, Zengwei Hefei Univ Technol Sch Comp Sci & Informat Engn Hefei Anhui Peoples R China Minist Educ Engn Res Ctr Safety Crit Ind Measurement & Contro Hefei Anhui Peoples R China Key Lab Ind Safety & Emergency Technol Hefei Anhui Peoples R China

Application performance and energy consumption deep exposed to task scheduling of nodes in wireless sensor networks (WSNs). Unreasonable task scheduling of nodes leads to excessive network energy consumption. Thus, a Q-learning algorithm for task scheduling based on Improved Support Vector Machine (ISVM) in WSNs, called ISVM-Q is proposed to optimize the application performance and energy consumption of networks. Energy consumption of task scheduling is associated with a reward of nodes in the learning process. To solve the "dimensionality disaster" problem of Q-learning, SVM is introduced as a value function approximation. Parameterizations of SVM function can strengthen the interpretation characteristics by using experience knowledge. Experiments show that ISVM-Q has the ability to make nodes perform tasks reasonably in a dynamic environment. Compared with classic task scheduling algorithms, ISVM-Q achieves better application performance with less energy consumption and keeps the learning system stable. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Wireless sensor networks Task scheduling value function approximation Q-learning

来源：评论

学校读者我要写书评

暂无评论

Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure

引用

NEUROCOMPUTING 2015年 156卷 157-165页

作者： Kiumarsi, Bahare Lewis, Frank L. Levine, Daniel S. Univ Texas Arlington UTA Res Inst Ft Worth TX 76118 USA Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang Peoples R China Univ Texas Arlington Arlington TX 76019 USA

In this paper motivated by recently discovered neurocognitive models of mechanisms in the brain, a new reinforcement learning (RL) method is presented based on a novel critic neural network (NN) structure to solve the optimal tracking problem of a nonlinear discrete time-varying system in an online manner. A multiple-model approach combined with an adaptive self-organizing map (ASOM) neural network is used to detect changes in the dynamics of the system. The number of sub-models is determined adaptively and grows once a mismatch between the stored sub-models and the new data is detected. By using the ASOM neural network, a novel value function approximation (VFA) scheme is presented. Each sub-model contributes into the value function based on a responsibility signal obtained by the ASOM. The responsibility signal determines how much each sub-model contributes to the general value function. Novel policy iteration and the value iteration algorithms are presented to find the optimal control for the partially-unknown nonlinear discrete time-varying systems in an online manner. Simulation results demonstrate the effectiveness of the proposed control scheme. (C) 2014 Elsevier B.V. All rights reserved.

关键词： Multiple-model Optimal control Adaptive self-organizing map Reinforcement learning value function approximation

来源：评论

学校读者我要写书评

暂无评论

A DEEP Q-LEARNING NETWORK FOR SHIP STOWAGE PLANNING PROBLEM

引用

POLISH MARITIME RESEARCH 2017年第s3期24卷 102-109页

作者： Shen, Yifan Zhao, Ning Xia, Mengjue Du, Xueqiang Shanghai Maritime Univ Sci Res Acad Shanghai Peoples R China Shanghai Maritime Univ Logist Engn Coll Shanghai 201306 Peoples R China

Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity greatly. Previous studies mainly focuses on solving stowage planning problem with online searching algorithm, efficiency of which is significantly affected by case size. In this study, a Deep Q-Learning Network (DQN) is proposed to solve ship stowage planning problem. With DQN, massive calculation and training is done in pre-training stage, while in application stage stowage plan can be made in seconds. To formulate network input, decision factors are analyzed to compose feature vector of stowage plan. States subject to constraints, available action and reward function of Q-value are designed. With these information and design, an 8-layer DQN is formulated with an evaluation function of mean square error is composed to learn stowage planning. At the end of this study, several production cases are solved with proposed DQN to validate the effectiveness and generalization ability. Result shows a good availability of DQN to solve ship stowage planning problem.

关键词： Deep Q-Leaning Network (DQN) Container terminal Ship stowage plan Markov decision process value function approximation Generalization

来源：评论

学校读者我要写书评

暂无评论

Dynamic Lookahead Policies for Stochastic-Dynamic Inventory Routing in Bike Sharing Systems

引用

COMPUTERS & OPERATIONS RESEARCH 2019年 106卷 260-279页

作者： Brinkmann, Jan Ulmer, Marlin W. Mattfeld, Dirk C. Tech Univ Carolo Wilhelmina Braunschweig D-38106 Braunschweig Germany

We present the stochastic-dynamic inventory routing problem for bike sharing systems (SDIRP). The objective of the SDIRP is to avoid unsatisfied demand by dynamically relocating bikes during the day. To anticipate potential future demands in the current inventory decisions, we present a dynamic lookahead policy (DLA). The policy simulates future demand over a predefined horizon. Because the heterogeneous demand patterns over the course of the day, the DLA horizons are time-dependent and autonomously parametrized by means of value function approximation, a method of approximate dynamic programming. We compare the DLA with conventional relocation strategies from the literature and lookahead policies with static horizons. Our study based on real-world data by the bike sharing system of Min-neapolis (Minnesota, USA) reveals the benefits of both anticipation by lookaheads as well as the time-dependent horizons of the DLA. We additionally show how the DLA is able to autonomously adapt to the demand patterns. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Bike sharing Approximate dynamic programming Dynamic vehicle routing Lookahead policies value function approximation Policy search

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：