检索结果-内蒙古大学图书馆

The dynamic shortest path problem with time-dependent stochastic disruptions

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2018年第Jul.期92卷 42-57页

作者： Sever, Derya Zhao, Lei Dellaert, Nico Demir, Emrah Van Woensel, Tom De Kok, Ton Eindhoven Univ Technol Sch Ind Engn Eindhoven Netherlands Tsinghua Univ Dept Ind Engn Beijing Peoples R China Cardiff Univ Panalpina Ctr Mfg & Logist Res Cardiff Business Sch Cardiff S Glam Wales

The dynamic shortest path problem with time-dependent stochastic disruptions consists of finding a route with a minimum expected travel time from an origin to a destination using both historical and real-time information. The problem is formulated as a discrete time finite horizon Markov decision process and it is solved by a hybrid Approximate Dynamic Programming (ADP) algorithm with a clustering approach using a deterministic lookahead policy and value function approximation. The algorithm is tested on a number of network configurations which represent different network sizes and disruption levels. Computational results reveal that the proposed hybrid ADP algorithm provides high quality solutions with a reduced computational effort.

关键词： Dynamic shortest path problem Approximate dynamic programming Time-dependent disruption Lookahead policy value function approximation

来源：评论

学校读者我要写书评

暂无评论

Horizontal combinations of online and offline approximate dynamic programming for stochastic dynamic vehicle routing

引用

CENTRAL EUROPEAN JOURNAL OF OPERATIONS RESEARCH 2020年第1期28卷 279-308页

作者： Ulmer, Marlin W. Tech Univ Carolo Wilhelmina Braunschweig Braunschweig Germany

Stochastic and dynamic vehicle routing problems gain increasing attention in the research community. In these problems, routing plans are dynamically updated based on realizations of stochastic information. Due to the complexity of the corresponding Markov decision processes (MDPs), the calculation of optimal policies for these problems is usually not possible and researchers draw on heuristical methods of approximate dynamic programming (ADP). These methods use simulation to approximate the value of a state and decision in the MDP. The simulations are either conducted offline or online. Offline methods such as value function approximations (VFAs) generally neglect the full detail of the state space due to aggregation. Online methods such as rollout algorithms (RAs) are often not able to capture decision and transition space sufficiently due to runtime limitations. In this paper, we alleviate this tradeoff by combining two methods of ADP, an online RA and an offline VFA in two ways. In addition to the integration of the VFA as a base policy into the online RA to strengthen the RA's simulations, we also limit the RA's simulation horizon, estimating the remaining reward-to-go again via the VFA. For two stochastic dynamic routing problems from the literature, we show how this combination outperforms state-of-the-art solutions while simultaneously reducing the required time for online calculations.

关键词： Approximate dynamic programming Rollout algorithm value function approximation Dynamic vehicle routing Stochastic requests Multi-period

来源：评论

学校读者我要写书评

暂无评论

Optimal Energy Management of a Residential Prosumer: A Robust Data-Driven Dynamic Programming Approach

引用

IEEE SYSTEMS JOURNAL 2022年第1期16卷 1548-1557页

作者： Guo, Zhongjie Wei, Wei Chen, Laijun Wang, Zhaojian Catalao, Joao P. S. Mei, Shengwei Tsinghua Univ Dept Elect Engn State Key Lab Power Syst Beijing 100084 Peoples R China Qinghai Univ New Energy Ind Res Ctr Xining 810016 Peoples R China Univ Porto Fac Engn P-4200465 Porto Portugal INESC TEC P-4200465 Porto Portugal

Prosumers are agents that both consume and produce energy. This article studies the optimal energy management of a residential prosumer which consists of a renewable power plant and an energy storage unit. Energy could stream among power grid, renewable plant, storage unit, and demand, providing a highly flexible energy supply and the opportunity of arbitrage. To capture the uncertainty of renewable generation and electricity price, as well as the rolling horizon feature of the multiperiod energy management, the problem is formulated as a robust data-driven dynamic programming (RDDP). Kernel regression is utilized to build the empirical conditional distribution in a data-driven manner, and all candidates that reside in a Wasserstein metric-based ambiguity set are taken into account to tackle the inexactness of the empirical distribution. The RDDP can be transformed into a series of convex optimization problems with cost-to-go functions in their constraints. The piecewise linear expression of the cost-to-go function is retrieved from dual linear programs. Through such an analytical expression of cost-to-go functions, the RDDP can be solved via backward induction, unlike the popular stochastic dual dynamic programming technique that incorporates forward and backward passes. Case studies validate the performance and advantage of the proposed RDDP approach.

关键词： Dynamic programming Optimization Energy management Wind turbines Uncertainty Stochastic processes Measurement Energy storage prosumer robust data-driven dynamic programming uncertainty value function approximation

来源：评论

学校读者我要写书评

暂无评论

Proto-value functions: A Laplacian framework for learning representation and control in Markov decision processes

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2007年第10期8卷 2169-2231页

作者： Mahadevan, Sridhar Maggioni, Mauro Univ Massachusetts Dept Comp Sci Amherst MA 01003 USA Duke Univ Dept Math & Comp Sci Durham NC 27708 USA

This paper introduces a novel spectral framework for solving Markov decision processes (MDPs) by jointly learning representations and optimal policies. The major components of the framework described in this paper include: (i) A general scheme for constructing representations or basis functions by diagonalizing symmetric diffusion operators (ii) A specific instantiation of this approach where global basis functions called proto-value functions (PVFs) are formed using the eigenvectors of the graph Laplacian on an undirected graph formed from state transitions induced by the MDP (iii) A three-phased procedure called representation policy iteration comprising of a sample collection phase, a representation learning phase that constructs basis functions from samples, and a final parameter estimation phase that determines an (approximately) optimal policy within the (linear) subspace spanned by the (current) basis functions. (iv) A specific instantiation of the RPI framework using least-squares policy iteration (LSPI) as the parameter estimation method (v) Several strategies for scaling the proposed approach to large discrete and continuous state spaces, including the Nystrom extension for out-of-sample interpolation of eigenfunctions, and the use of Kronecker sum factorization to construct compact eigenfunctions in product spaces such as factored MDPs (vi) Finally, a series of illustrative discrete and continuous control tasks, which both illustrate the concepts and provide a benchmark for evaluating the proposed approach. Many challenges remain to be addressed in scaling the proposed framework to large MDPs, and several elaboration of the proposed framework are briefly summarized at the end.

关键词： Markov decision processes reinforcement learning value function approximation manifold learning spectral graph theory

来源：评论

学校读者我要写书评

暂无评论

Hamiltonian-Driven Adaptive Dynamic Programming for Continuous Nonlinear Dynamical Systems

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2017年第8期28卷 1929-1940页

作者： Yang, Yongliang Wunsch, Donald Yin, Yixin Univ Sci & Technol Beijing Sch Automat & Elect Engn Beijing 100083 Peoples R China Missouri Univ Sci & Technol Dept Elect & Comp Engn Rolla MO 65401 USA

This paper presents a Hamiltonian-driven framework of adaptive dynamic programming (ADP) for continuous time nonlinear systems, which consists of evaluation of an admissible control, comparison between two different admissible policies with respect to the corresponding the performance function, and the performance improvement of an admissible control. It is showed that the Hamiltonian can serve as the temporal difference for continuous-time systems. In the Hamiltonian-driven ADP, the critic network is trained to output the value gradient. Then, the inner product between the critic and the system dynamics produces the value derivative. Under some conditions, the minimization of the Hamiltonian functional is equivalent to the value function approximation. An iterative algorithm starting from an arbitrary admissible control is presented for the optimal control approximation with its convergence proof. The implementation is accomplished by a neural network approximation. Two simulation studies demonstrate the effectiveness of Hamiltoniandriven ADP.

关键词： Adaptive dynamic programming (ADP) convergence proof Hamiltonian functional Hamiltonian-driven framework neural network (NN) approximation value function value function approximation

来源：评论

学校读者我要写书评

暂无评论

Gaussian Process Approximate Dynamic Programming for Energy-Optimal Supervisory Control of Parallel Hybrid Electric Vehicles

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2022年第8期71卷 8367-8380页

作者： Bae, Jin Woo Kim, Kwang-Ki K. Texas A&M Univ Dept Comp Sci & Engn College Stn TX 77843 USA Inha Univ Dept Elect & Comp Engn Incheon 22212 South Korea

We propose an energy-efficient supervisory control method for the power management of parallel hybrid electric vehicles (HEVs) to improve the fuel economy and reduce exhaust gas emissions. Plug-in HEVs ((P)HEVs) have multiple power sources (e.g., an engine and motor) that should be cooperatively operated to meet the required instantaneous traction power for the desired vehicle speed while satisfying their physical limits. Because the efficiencies of the engine and motor vary with different operating speeds and torques, the main issue of energy-efficient power management is to allocate the power demand among the power sources by achieving maximum power conversion efficiencies and satisfy the operating limits. For an efficient power allocation, an optimal control problem is formulated, and a global solution is found through deterministic dynamic programming (DP). Owing to the curse of dimensionality and uncertainties in real driving, DP solutions are not directly applicable in real time. To resolve the limitations of DP, we employ a non-parametric Bayesian function approximation technique using a Gaussian process (GP). The offline DP solutions obtained from a set of real vehicle driving test data were used to learn a state-dependent probabilistic value function through Gaussian process regression. For online implementations, a receding horizon control scheme was applied for the feedback control of the power management. In comparison with the existing charge sustaining strategy and charge depleting and charge sustaining mixed controllers, we recorded fuel efficiency improvements of over 4.8% and 7.3%, respectively, in a mixed urban-suburban route.

关键词： Batteries Vehicle dynamics Mathematical models Hybrid electric vehicles Energy management Optimal control Engines Approximate dynamic programming energy management gaussian process regression optimal control Parallel hybrid electric vehicles supervisory control value function approximation

来源：评论

学校读者我要写书评

暂无评论

High-Speed Finite Control Set Model Predictive Control for Power Electronics

引用

IEEE TRANSACTIONS ON POWER ELECTRONICS 2017年第5期32卷 4007-4020页

作者： Stellato, Bartolomeo Geyer, Tobias Goulart, Paul J. Univ Oxford Oxford OX1 3PJ England ABB Corp Res Ctr CH-5405 Baden Switzerland

Common approaches for direct model predictive control (MPC) for current reference tracking in power electronics suffer from the high computational complexity encountered when solving integer optimal control problems over long prediction horizons. We propose an efficient alternative method based on approximate dynamic programming, greatly reducing the computational burden and enabling sampling times below 25 mu s. Our approach is based on the offline estimation of an infinite horizon value function, which is then utilized as the tail cost of anMPC problem. This allows us to reduce the controller horizon to a very small number of stages, while simultaneously improving the overall controller performance. Our proposed algorithm was implemented on a small size FPGA and validated on a variable-speed drive system with a three-level voltage-source converter. Time measurements showed that our algorithm requires only 5.76 mu s for horizon N = 1 and 17.27 mu s for N = 2, in both cases outperforming state-of-the-art approaches with much longer horizons in terms of currents distortion and switching frequency. To the authors' knowledge, this is the first time direct MPC for current control has been implemented on an FPGA solving the integer optimization problem in real time and achieving comparable performance to formulations with long prediction horizons.

关键词： Approximate dynamic programming (ADP) drive systems finite control set model predictive control (MPC) value function approximation

来源：评论

学校读者我要写书评

暂无评论

Technical update: Least-squares temporal difference learning

引用

MACHINE LEARNING 2002年第2-3期49卷 233-246页

作者： Boyan, JA ITA Software Cambridge MA 02139 USA

TD(lambda) is a popular family of algorithms for approximate policy evaluation in large MDPs. TD(lambda) works by incrementally updating the value function after each observed transition. It has two major drawbacks: it may make inefficient use of data, and it requires the user to manually tune a stepsize schedule for good performance. For the case of linear value function approximations and lambda = 0, the Least-Squares TD (LSTD) algorithm of Bradtke and Barto (1996, Machine learning, 22:1-3, 33-57) eliminates all stepsize parameters and improves data efficiency. This paper updates Bradtke and Barto's work in three significant ways. First, it presents a simpler derivation of the LSTD algorithm. Second, it generalizes from lambda = 0 to arbitrary values of lambda;at the extreme of lambda = 1, the resulting new algorithm is shown to be a practical, incremental formulation of supervised linear regression. Third, it presents a novel and intuitive interpretation of LSTD as a model-based reinforcement learning technique.

关键词： reinforcement learning temporal difference learning value function approximation linear least-squares methods

来源：评论

学校读者我要写书评

暂无评论

Two-loop reinforcement learning algorithm for finite-horizon optimal control of continuous-time affine nonlinear systems

引用

INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL 2022年第1期32卷 393-420页

作者： Chen, Zhe Xue, Wenqian Li, Ning Lewis, Frank L. Shanghai Jiao Tong Univ Dept Automat Shanghai 200240 Peoples R China Minist Educ China Key Lab Syst Control & Informat Proc Shanghai Peoples R China Shanghai Engn Res Ctr Intelligent Control & Manag Shanghai Peoples R China Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang Peoples R China Northeastern Univ Int Joint Res Lab Integrated Automat Shenyang Peoples R China Univ Texas Arlington UTA Res Inst Arlington TX 76019 USA

This article proposes three novel time-varying policy iteration algorithms for finite-horizon optimal control problem of continuous-time affine nonlinear systems. We first propose a model-based time-varying policy iteration algorithm. The method considers time-varying solutions to the Hamiltonian-Jacobi-Bellman equation for finite-horizon optimal control. Based on this algorithm, value function approximation is applied to the Bellman equation by establishing neural networks with time-varying weights. A novel update law for time-varying weights is put forward based on the idea of iterative learning control, which obtains optimal solutions more efficiently compared to previous works. Considering that system models may be unknown in real applications, we propose a partially model-free time-varying policy iteration algorithm that applies integral reinforcement learning to acquiring the time-varying value function. Moreover, analysis of convergence, stability, and optimality is provided for every algorithm. Finally, simulations for different cases are given to verify the convenience and effectiveness of the proposed algorithms.

关键词： continuous-time nonlinear system finite-horizon optimal control iterative learning control policy iteration reinforcement learning value function approximation

来源：评论

学校读者我要写书评

暂无评论

Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives

引用

ROBOTICS AND AUTONOMOUS SYSTEMS 2011年第11期59卷 910-922页

作者： Tamosiunaite, Minija Nemec, Bojan Ude, Ales Woergoetter, Florentin Univ Gottingen Inst Phys Biophys 3 Bernstein Ctr Computat Neurosci D-37077 Gottingen Germany Vytautas Magnus Univ Dept Informat LT-44404 Kaunas Lithuania Jozef Stefan Inst Dept Automat Biocybernet & Robot Ljubljana 1000 Slovenia

When describing robot motion with dynamic movement primitives (DMPs), goal (trajectory endpoint), shape and temporal scaling parameters are used. In reinforcement learning with DMPs, usually goals and temporal scaling parameters are predefined and only the weights for shaping a DMP are learned. Many tasks, however, exist where the best goal position is not a priori known, requiring to learn it. Thus, here we specifically address the question of how to simultaneously combine goal and shape parameter learning. This is a difficult problem because learning of both parameters could easily interfere in a destructive way. We apply value function approximation techniques for goal learning and direct policy search methods for shape learning. Specifically, we use "policy improvement with path integrals" and "natural actor critic" for the policy search. We solve a learning-to-pour-liquid task in simulations as well as using a Pa10 robot arm. Results for learning from scratch, learning initialized by human demonstration, as well as for modifying the tool for the learned DMPs are presented. We observe that the combination of goal and shape learning is stable and robust within large parameter regimes. Learning converges quickly even in the presence of disturbances, which makes this combined method suitable for robotic applications. (C) 2011 Elsevier B.V. All rights reserved.

关键词： Reinforcement learning PI2-method Natural actor critic value function approximation Dynamic movement primitives

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：