检索结果-内蒙古大学图书馆

Smart Exploration in Reinforcement Learning using Bounded Uncertainty Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： van Hulst, J.S. Heemels, W.P.M.H. Antunes, D.J. Control Systems Technology Section Department of Mechanical Engineering Eindhoven University of Technology Netherlands

Reinforcement learning (RL) is a powerful tool for decision-making in uncertain environments, but it often requires large amounts of data to learn an optimal policy. We propose using prior model knowledge to guide the exploration process to speed up this learning process. This model knowledge comes in the form of a model set to which the true transition kernel and reward function belong. We optimize over this model set to obtain upper and lower bounds on the Q-function, which are then used to guide the exploration of the agent. We provide theoretical guarantees on the convergence of the Q-function to the optimal Q-function under the proposed class of exploring policies. Furthermore, we also introduce a data-driven regularized version of the model set optimization problem that ensures the convergence of the class of exploring policies to the optimal policy. Lastly, we show that when the model set has a specific structure, namely the bounded-parameter MDP (BMDP) framework, the regularized model set optimization problem becomes convex and simple to implement. In this setting, we also show that we obtain finite-time convergence to the optimal policy under additional assumptions. We demonstrate the effectiveness of the proposed exploration strategy in a simulation study. The results indicate that the proposed method can significantly speed up the learning process in reinforcement learning. © 2025, CC BY-NC-ND.

关键词： Uncertainty analysis

Scheduling of over-actuated networked control systems

学校读者我要写书评

暂无评论

IFAC-PapersOnLine 2022年第13期55卷 25-30页

作者： J. Kreiss R. Postoyan D. Nešić W.P.M.H. Heemels Université de Lorraine CNRS CRAN F-54000 Nancy France Department of Electrical and Electronic Engineering University of Melbourne Parkville VIC 3010 Australia Control Systems Technology Section Department of Mechanical Engineering Eindhoven University of Technology Netherlands

We investigate the scenario where an over-actuated plant is controlled over a network. We concentrate on the effect of two network-induced phenomena: varying transmission intervals and scheduling, in the sense that only one of the actuators receives new data at each transmission instant. We present an emulation-based solution for the controller design, i.e., the controller is first designed to stabilize the origin of the plant ignoring the network and second, the packet-based network is taken into account and conditions on the maximum allowable transmission interval (MATI) and the scheduling protocol are given to preserve the stability of the closed-loop system. Our results are tailored to over-actuated plants leading to significant improvements compared to applying off-the-shelf results available in the literature. In particular, a new model is derived and new conditions on the scheduling protocol are given, which lead to a two-measure stability property for the networked control system. We illustrate how new classes of scheduling protocols can be derived by exploiting over-actuation, which leads to larger MATI bounds compared to applying existing results, as shown on a numerical example.

关键词： Networked control systems over-actuated systems hybrid systems stability of hybrid systems Lyapunov methods

A Coverage control-based Idle Vehicle Rebalancing Approach for Autonomous Mobility-on-Demand systems

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhu, Pengbo Sirmatel, Isik Ilber Ferrari-Trecate, Giancarlo Geroliminis, Nikolas Urban Transport Systems Laboratory École Polytechnique Fédérale de Lausanne Lausanne1015 Switzerland Control Section Department of Electrical and Electronics Engineering Faculty of Engineering Trakya University Edirne22030 Turkey Dependable Control and Decision Group École Polytechnique Fédérale de Lausanne Lausanne1015 Switzerland

As an emerging mode of urban transportation, Autonomous Mobility-on-Demand (AMoD) systems show the potential in improving mobility in cities through timely and door-to-door services. However, the spatiotemporal imbalances between mobility demand and supply may lead to inefficiencies and a low quality of service. Vehicle rebalancing (i.e., dispatching idle vehicles to high-demand areas), is a potential solution for efficient AMoD fleet management. In this paper, we formulate the vehicle rebalancing problem as a coverage control problem for the deployment of a fleet of mobile agents for AMoD operation in urban areas. Performance is demonstrated via microscopic simulations representing a large urban road network of Shenzhen, China. Results reveal the potential of the proposed method in improving service rates and decreasing passenger waiting times. Copyright © 2023, The Authors. All rights reserved.

关键词： Urban transportation

Robustifying Event-Triggered control to Measurement Noise

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Scheres, Koen J.A. Postoyan, Romain Maurice Heemels, W.P.H. The Control Systems Technology Section Department Mechanical Engineering Eindhoven University of Technology Netherlands The Université de Lorraine CNRS CRAN NancyF-54000 France

While many event-triggered control strategies are available in the literature, most of them are designed ignoring the presence of measurement noise. As measurement noise is omnipresent in practice and can have detrimental effects, for instance, by inducing Zeno behavior in the closed-loop system and with that the lack of a positive lower bound on the inter-event times, rendering the event-triggered control design practically useless, it is of great importance to address this gap in the literature. To do so, we present a general framework for set stabilization of (distributed) event-triggered control systems affected by additive measurement noise. It is shown that, under general conditions, Zeno-free static as well as dynamic triggering rules can be designed such that the closed-loop system satisfies an input-to-state practical set stability property. We ensure Zeno-freeness by proving the existence of a uniform strictly positive lower-bound on the minimum inter-event time. The general framework is applied to point stabilization and consensus problems as particular cases, where we show that, under similar assumptions as the original work, existing schemes can be redesigned to robustify them to measurement noise. Consequently, using this framework, noise-robust triggering conditions can be designed both from the ground up and by simple redesign of several important existing schemes. Simulation results are provided that illustrate the strengths of this novel approach. Copyright © 2022, The Authors. All rights reserved.

关键词： Closed loop systems

Gaussian Processes for Advanced Motion control*

学校读者我要写书评

暂无评论

Gaussian Processes for Advanced Motion Control*

American control Conference (ACC)

作者： Maurice Poot Jim Portegies Noud Mooren Max van Haren Max van Meer Tom Oomen Department of Mechanical Engineering Control Systems Technology Section Eindhoven University of Technology Eindhoven The Netherlands Department of Mathematics and Computer Science CASA Eindhoven University of Technology Eindhoven The Netherlands Delft Center for Systems and Control Delft University of Technology Delft The Netherlands

ISBN: (数字)9781665451963

ISBN: (纸本)9781665494809

Machine learning techniques, including Gaussian processes (GPs), are expected to play a significant role in meeting speed, accuracy, and functionality requirements in future data-intensive mechatronic systems. This paper aims to reveal the potential of GPs for motion control applications. Successful applications of GPs for feedforward and learning control, including the identification and learning for noncausal feedforward, position-dependent snap feedforward, nonlinear feedforward, and GP-based spatial repetitive control, are outlined. Experimental results on various systems, including a desktop printer, wirebonder, and substrate carrier, confirmed that data-based learning using GPs can significantly improve the accuracy of mechatronic systems.

关键词： Mechatronics Gaussian processes Machine learning Printers Feedforward systems Motion control Substrates

Accelerating Soft-Constrained MPC for Linear systems Through Online Constraint Removal

学校读者我要写书评

暂无评论

Accelerating Soft-Constrained MPC for Linear Systems Through...

IEEE Conference on Decision and control

作者： S.A.N. Nouwens M.M. Paulides W.P.M.H. Heemels Department of Mechanical Engineering Control Systems Technology section Eindhoven University of Technology Eindhoven The Netherlands Department of Electrical Engineering Electromagnetics for Care & Cure Eindhoven University of Technology Eindhoven The Netherlands Department of Radiotherapy Erasmus University Medical Center Cancer Institute Rotterdam The Netherlands

Optimization-based controllers, such as Model Predictive control (MPC), have attracted significant research interest due to their intuitive concept, constraint handling capabilities, and natural application to multi-input multi-output systems. However, the computational complexity of solving a receding horizon problem at each time step remains a challenge for the deployment of MPC. This is particularly the case for systems constrained by many inequalities. Recently, we introduced the concept of constraint-adaptive MPC (ca-MPC) to address this challenge for linear systems with hard constraints. In ca-MPC, at each time step, a subset of the constraints is removed from the optimization problem, thereby accelerating the optimization procedure, while resulting in identical closed-loop behavior. The present paper extends this framework to soft-constrained MPC by detecting and removing constraints based on sub-optimal predicted input sequences, which is rather easy for soft-constrained MPC due to the receding horizon principle and the inclusion of slack variables. We will translate these new ideas explicitly to an offset-free output tracking problem. The effectiveness of these ideas is demonstrated on a two-dimensional thermal transport model, showing a three order of magnitude improvement in online computational time of the MPC scheme.

关键词：

Nonlinear Model Predictive control Approximation: Applications to Truck-Trailer control System

学校读者我要写书评

暂无评论

Nonlinear Model Predictive Control Approximation: Applicatio...

International Conference on control, Automation and systems ( ICCAS)

作者： Suyong Park Duc Giap Nguyen Yongsik Jin Jinrak Park Dohee Kim Jeong Soo Eo Kyoungseok Han Department of Automotive Engineering Hanyang University Seoul Republic of Korea Department of Mechanical Engineering Kyungpook National University Daegu Republic of Korea Robotics IT Convergence Research Section Electronics and Telecommunications Research Institute (ETRI) Daegu Republic of Korea Department of Electrified Systems Control Research Lab Hyundai Motor Company Hwaseong Republic of Korea

ISBN: (数字)9788993215380

ISBN: (纸本)9798331517939

In this work, we demonstrate the effectiveness of nonlinear model predictive control (NMPC) approximation based on deep neural network (DNN). MPC has been widely adopted in autonomous driving control problems to handle multiple objectives and constraints. We first design the implicit NMPC for the forward and backward motions of a truck-trailer (TT) system, which follows the reference path while maintaining safety between the head truck (HT) and the trailer (TR). However, the computational load in implicit MPC makes it a challenge for real-time implementations. To alleviate the computational burden in implicit NMPC online, an NMPC approximation approach based on DNN is adopted in this study to achieve a parametric function approximation. We conduct a comparative study on the proposed approach and a baseline controller for control performance analysis, and the computational load is evaluated on a hardware-in-the-loop (HIL) experimental system.

关键词： Training Tracking Simulation Training data Artificial neural networks Predictive models control systems Real-time systems Safety Predictive control

Evaluation of the Safety Shell Architecture for Automated Driving in a Realistic Simulator

学校读者我要写书评

暂无评论

Evaluation of the Safety Shell Architecture for Automated Dr...

IEEE Symposium on Intelligent Vehicle

作者： C.A.J. Hanselaar Y. Fu A. Terechko J. Seemann T. Beurskens E. Silvas W.P.M.H Heemels Department of Mechanical Engineering Control Systems Technology Section Eindhoven University of Technology NXP Semiconductors N.V Eindhoven The Netherlands Former NXP Semiconductors N.V. Employee TNO Dutch Organization for Applied Scientific Research Integrated Vehicle Safety Helmond The Netherlands

ISBN: (数字)9798350348811

ISBN: (纸本)9798350348828

The transition from advanced driver assistance systems to highly automated vehicles proves to be difficult, as the driver is no longer a safety fallback for the latter systems. One of the main challenges is formed by edge cases in the encountered driving scenarios that trigger functional insufficiencies in automated driving (AD) systems. Functional insufficiencies, for the sake of understanding, may be viewed as an inappropriate understanding of or response to a scenario in an AD system, which in turn causes dangerous vehicle behavior. Prior research suggests that using an architecture capable of including redundant heterogeneous AD systems as separate channels, such as the Safety Shell, can mitigate some of these functional insufficiencies. However, this benefit has only been evaluated in limited and deterministic simulation environments. To overcome this, our objectives in this paper are to (i) develop an experimental method for extensive testing of such architectures, and (ii) to assess the suitability of the Safety Shell architecture to handle edge cases with this new method. Using the developed experimental setup we observe a significant safety and availability increase of the Safety Shell compared to the included individual AD channels in the tested scenarios. Finally, our study provides insight into the requirements for the evaluated AD channels.

关键词： Actuators Intelligent vehicles Switches Safety Object recognition Advanced driver assistance systems Testing

Position-Dependent Snap Feedforward: A Gaussian Process Framework

学校读者我要写书评

暂无评论

arXiv 2022年

作者： van Haren, Max Poot, Maurice Portegies, Jim Oomen, Tom The Control Systems Technology Section Department of Mechanical Engineering Eindhoven University of Technology Eindhoven Netherlands The Centre for Analysis Scientific Computing and Applications Department of Mathematics and Computer Science Eindhoven University of Technology Eindhoven Netherlands The Delft Center for Systems and Control Delft University of Technology Delft Netherlands

Mechatronic systems have increasingly high performance requirements for motion control. The low-frequency contribution of the flexible dynamics, i.e. the compliance, should be compensated for by means of snap feedforward to achieve high accuracy. Position-dependent compliance, which often occurs in motion systems, requires the snap feedforward parameter to be modeled as a function of position. Position-dependent compliance is compensated for by using a Gaussian process to model the snap feedforward parameter as a continuous function of position. A simulation of a flexible beam shows that a significant performance increase is achieved when using the Gaussian process snap feedforward parameter to compensate for position-dependent compliance. © 2022, CC BY.

关键词： Flexible structures