检索结果-内蒙古大学图书馆

mixed-integer optimal control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

optimal control APPLICATIONS & METHODS 2025年第1期46卷 307-319页

作者： Xu, Jinming Azad, Nasser Lashgarian Lin, Yuan South China Univ Technol Shien Ming Wu Sch Intelligent Engn Guangzhou Guangdong Peoples R China Univ Waterloo Dept Syst Design Engn Waterloo ON Canada

Many optimal control problems require the simultaneous output of discrete and continuous control variables. These problems are typically formulated as mixed-integer optimal control (MIOC) problems, which are challenging to solve due to the complexity of the solution space. Numerical methods such as branch-and-bound are computationally expensive and undesirable for real-time control. This article proposes a novel hybrid-action reinforcement learning (HARL) algorithm, twin delayed deep deterministic actor-Q (TD3AQ), for MIOC problems. TD3AQ leverages actor-critic and Q-learning methods to manage discrete and continuous action spaces simultaneously. The proposed algorithm is evaluated on a plug-in hybrid electric vehicle (PHEV) energy management problem, where real-time control of the discrete variables, clutch engagement/disengagement and gear shift, and continuous variable, engine torque, is essential to maximize fuel economy while satisfying driving constraints. Simulation results show that TD3AQ achieves near-optimal control, with only a 4.69% difference from dynamic programming (DP), and outperforms baseline reinforcement learning algorithms for hybrid action spaces. The sub-millisecond execution time indicates potential applicability in other time-critical scenarios, such as autonomous driving or robotic control. The proposed hybrid-action reinforcement learning algorithm, TD3AQ, demonstrates promising potential in addressing mixed-integer optimal control problems. image

关键词： hybrid action space hybrid electric vehicle energy management mixed-integer optimal control reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Efficient upper and lower bounds for global mixed-integer optimal control

引用

JOURNAL OF GLOBAL OPTIMIZATION 2015年第4期61卷 721-743页

作者： Sager, Sebastian Claeys, Mathieu Messine, Frederic Univ Magdeburg Fac Math D-39106 Magdeburg Germany Univ Cambridge Dept Engn Cambridge CB2 1PZ England Univ Toulouse LAAS Toulouse France Univ Toulouse LAPLACE ENSEEIHT Toulouse France

We present a control problem for an electrical vehicle. Its motor can be operated in two discrete modes, leading either to acceleration and energy consumption, or to a recharging of the battery. Mathematically, this leads to a mixed-integer optimal control problem (MIOCP) with a discrete feasible set for the controls taking into account the electrical and mechanical dynamic equations. The combination of nonlinear dynamics and discrete decisions poses a challenge to established optimization and control methods, especially if global optimality is an issue. Probably for the first time, we present a complete analysis of the optimal solution of such a MIOCP: solution of the integer-relaxed problem both with a direct and an indirect approach, determination of integer controls by means of the sum up rounding strategy, and calculation of global lower bounds by means of the method of moments. As we decrease the control discretization grid and increase the relaxation order, the obtained series of upper and lower bounds converge for the electrical car problem, proving the asymptotic global optimality of the calculated chattering behavior. We stress that these bounds hold for the optimal control problem in function space, and not on an a priori given (typically coarse) control discretization grid, as in other approaches from the literature. This approach is generic and is an alternative to global optimal control based on probabilistic or branch-and-bound based techniques. The main advantage is a drastic reduction of computational time. The disadvantage is that only local solutions and certified lower bounds are provided with no possibility to reduce these gaps. For the instances of the electrical car problem, though, these gaps are very small. The main contribution of the paper is a survey and new combination of state-of-the-art methods for global mixed-integer optimal control and the in-depth analysis of an important, prototypical control problem. Despite the comparatively low

关键词： mixed-integer optimal control Global optimal control Maximum principle Method of moments Energy consumption Electrical car

来源：评论

学校读者我要写书评

暂无评论

Flight trajectory design in the presence of contrails: Application of a multiphase mixed-integer optimal control approach

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2014年第Nov.期48卷 172-194页

作者： Soler, Manuel Zou, Bo Hansen, Mark Univ Carlos III Madrid Dept Bioengn & Aerosp Engn Madrid Spain Univ Illinois Dept Civil & Mat Engn Chicago IL 60680 USA Univ Calif Berkeley Dept Civil & Environm Engn Berkeley CA 94720 USA

In this paper we study the 4D trajectory planning problem in a contrail sensitive environment. We identify the control inputs that steer the aircraft from the initial fix to the final fix following a horizontal route of waypoints while performing step climbs and descents, in order to minimize the overall flying cost of fuel consumption, CO2 emissions, passenger travel time, and persistent contrail formation. The optimal trajectory design problem is formulated as a multiphase mixed integer optimal control problem, which is converted into a mixed integer non-linear program by first making the unknown switching times part of the state, then applying a Hermite-Simpson direct collocation method, and finally introducing binary variables to model the decision making. We solve the mixed-integer nonlinear program using a branch-and-bound algorithm. The numerical results show the effectiveness of the approach. (C) 2014 Elsevier Ltd. All rights reserved.

关键词： Flight 4D trajectory design Persistent contrails Climate impact mixed-integer optimal control mixed-integer non-linear program

来源：评论

学校读者我要写书评

暂无评论

Relaxation methods for hyperbolic PDE mixed-integer optimal control problems

引用

optimal control APPLICATIONS & METHODS 2017年第6期38卷 1103-1110页

作者： Hante, Falk M. Univ Erlangen Nurnberg Dept Math Cauerstr 11 D-91058 Erlangen Germany

The convergence analysis for methods solving partial differential equations constrained optimal control problems containing both discrete and continuous control decisions based on relaxation and rounding strategies is extended to the class of first order semilinear hyperbolic systems in one space dimension. The results are obtained by novel a priori estimates for the size of the relaxation gap based on the characteristic flow, fixed-point arguments, and particular regularity theory for such mixed-integer control problems. Motivated by traffic flow problems, a relaxation model for optimal flux switching control in conservation laws is considered as an application.

关键词： hyperbolic PDEs mixed-integer optimal control PDE-constrained optimization

来源：评论

学校读者我要写书评

暂无评论

A numerical study of transformed mixed-integer optimal control problems

引用

MATHEMATICAL PROGRAMMING COMPUTATION 2024年第4期16卷 561-597页

作者： Sager, Sebastian Tetschke, Manuel Zeile, Clemens Otto Von Guericke Univ Fac Math Magdeburg Germany

Time transformation is a ubiquitous tool in theoretical sciences, especially in physics. It can also be used to transform switched optimal control problems into control problems with a fixed switching order and purely continuous decisions. This approach is known either as enhanced time transformation, time-scaling, or switching time optimization (STO) for mixed-integer optimal control. The approach is well understood and used widely due to its many favorable properties. Recently, several extensions and algorithmic improvements have been proposed. We use an alternative formulation, the partial outer convexification (POC), to study convergence properties of (STO). We introduce the open-source software package ampl_mintoc (Sager et al., czeile/ampl_mintoc: Math programming c release, 2024, https://***/10.5281/zenodo.12520490). It is based on AMPL, designed for the formulation of mixed-integer optimal control problems, and allows to use almost identical implementations for (STO) and (POC). We discuss and explain our main numerical result: (STO) is likely to result in more local minima for each discretization grid than (POC), but the number of local minima is asymptotically identical for both approaches.

关键词： mixed-integer optimal control Switched systems Enhanced time transformation Switching time optimization Optimization software

来源：评论

学校读者我要写书评

暂无评论

Mathematical Programs with Blocks of Vanishing Constraints Arising in Discretized mixed-integer optimal control Problems

引用

SET-VALUED AND VARIATIONAL ANALYSIS 2015年第1期23卷 149-167页

作者： Palagachev, Konstantin Gerdts, Matthias Univ Bundeswehr Munchen Fak Luft & Raumfahrttech Inst Math & Rechneranwendung D-85577 Neubiberg Germany

A time transformation technique is used to discretize an optimal control problem with discrete-valued controls and mixed control-state constraints, which depend on the discrete-valued controls. The discretized optimal control problem leads to a mathematical program with blocks of vanishing constraints (MPBVC). The MPBVC is analyzed in terms of stationarity concepts and constraint qualifications extending previous results on mathematical programs with vanishing constraints to the case of blocks of vanishing constraints. A relaxation technique is employed to regularize the MPBVC and to solve it numerically. The relaxation method is validated for a flight path optimization problem taking into account a discrete number of positions for the flaps of the aircraft in a landing approach.

关键词： mixed-integer optimal control Blocks vanishing constraints Variable time transformation Relaxation technique Flight path optimization

来源：评论

学校读者我要写书评

暂无评论

Numerical mixed-integer optimal control and motorized traveling salesmen problems

引用

Journal Europeen des Systemes Automatises 2001年第4期35卷 519-533页

作者： Von, Stryk, Oskar Glocker, Markus Simulat. and Syst. Optimization Grp. Department of Computer Science Darmstadt University of Technology Alexanderstr. 10 D-64283 Darmstadt Germany

A general approach for the numerical solution of hybrid, mixed-integer optimal control problems is presented. In an outer level iteration a branch-and-bound procedure is applied to search the entire feasible discrete variable space. An inner level iteration contains for each actual value of the discrete variable a continuous nonlinear optimal control problem with its nonlinear dynamics defined in multiple phases and phase transitions occuring at unknown switching points (events) which must be solved numerically subject to nonlinear constraints. For this purpose, a robust and efficient direct collocation method is employed that parameterizes both the continuous state and control variables and exploits the sparse structure in the resulting nonlinearly constrained optimization problems. The proposed approach is successfully applied to two new hybrid optimal control benchmark problems for a motorized traveling salesman and for a team of two cooperating, motorized salesmen.

关键词： Branch-and-bound mixed-integer optimal control Motorized traveling salesman Nonlinear hybrid dynamical systems Sparse direct collocation

来源：评论

学校读者我要写书评

暂无评论

Modeling mixed-integer Constrained optimal control Problems in AMPL

引用

IFAC Proceedings Volumes 2012年第2期45卷 1124-1129页

作者： Christian Kirches Hans Georg Bock Sven Leyffer Interdisciplinary Center for Scientific Computing Ruprecht-Karls-Universität Heidelberg Germany Mathematics & Computer Science Division Argonne National Laboratory Argonne IL U.S.A.

Abstract Modeling languages and systems for simulation and optimization of continuous ODE/DAE systems are commonly available. For the most part, they focus on convenience of user interaction, and are tightly coupled to one or a few selected numerical methods. control problems with discrete and hybrid controls, called mixed-integer optimal control problems (MIOCPs), have recently gained increased attention as the potential for optimization is high. The mixed-integer optimization community however most often considers problems without dynamics, and relies on symbolic modeling languages such as AMPL. Access to many advances MI(N)LP codes is provided by the NEOS Server for Optimization through the AMPL modeling language. Addressing this gap, we describe a set of extensions to the AMPL modeling language to conveniently model mixed-integer optimal control problems for ODE or DAE dynamic processes. These extensions are easily realized and do not require intrusive changes to the AMPL language standard or implementation itself. An example of an optimal control problem solver interfaced with AMPL is the “multiple shooting code for optimal control” MUSCOD-II, a direct and simultaneous method for ODE/DAE-constrained optimal control, and its extension MS-MINTOC for mixed-integer optimal control. As an example, we use the described AMPL extensions to model a heavy duty truck control problem.

关键词： mixed-integer optimal control differential-algebraic equations domain specific languages

来源：评论

学校读者我要写书评

暂无评论

A semi-centralized multi-agent RL framework for efficient irrigation scheduling

引用

control ENGINEERING PRACTICE 2025年 155卷

作者： Agyeman, Bernard T. Decardi-Nelson, Benjamin Liu, Jinfeng Shah, Sirish L. Univ Alberta Dept Chem & Mat Engn Edmonton AB T6G 1H9 Canada Cornell Univ Syst Engn Ithaca NY 14853 USA

Efficient water management in agriculture is essential for addressing the growing freshwater scarcity crisis. Multi-Agent Reinforcement Learning (MARL) has emerged as a promising method for solving daily irrigation scheduling problems in spatially variable fields, where management zones are employed to account for field variability. To enhance the application of MARL to address daily irrigation scheduling in large-scale fields with significant spatial variation, this study proposes a Semi-Centralized MARL (SCMARL) framework. The SCMARL framework adopts a hierarchical structure, decomposing the daily irrigation scheduling problem into two levels of decision-making. At the top level, a centralized coordinator agent determines irrigation timing, which is modeled as a discrete variable, based on field-wide soil moisture data, crop conditions, and weather forecasts. At the lower level, decentralized local agents use local soil moisture, crop, and weather information to determine the appropriate irrigation amounts for each management zone. To address the issue of non-stationarity in this framework, a state augmentation technique is employed, wherein the coordinator's decision is incorporated into the decision-making process of the local agents. The SCMARL framework, which leverages the Proximal Policy Optimization algorithm for training the agents, is evaluated on a large-scale field in Lethbridge, Canada, and compared with an existing MARL irrigation scheduling approach. The results demonstrate improved performance, achieving a 4.0% reduction in water use and a 6.3% increase in irrigation water use efficiency.

关键词： Irrigation scheduling mixed-integer optimal control Multi-Agent Reinforcement Learning Semi-Centralized Multi-Agent Reinforcement Learning

来源：评论

学校读者我要写书评

暂无评论

Compactness and convergence rates in the combinatorial integral approximation decomposition

引用

MATHEMATICAL PROGRAMMING 2021年第2期188卷 569-598页

作者： Kirches, Christian Manns, Paul Ulbrich, Stefan Tech Univ Carolo Wilhelmina Braunschweig Univ Pl 2 D-38106 Braunschweig Germany Tech Univ Darmstadt Dolivostr 15 D-64293 Darmstadt Germany

The combinatorial integral approximation decomposition splits the optimization of a discrete-valued control into two steps: solving a continuous relaxation of the discrete control problem, and computing a discrete-valued approximation of the relaxed control. Different algorithms exist for the second step to construct piecewise constant discrete-valued approximants that are defined on given decompositions of the domain. It is known that the resulting discrete controls can be constructed such that they converge to a relaxed control in the weak* topology of L-infinity if the grid constant of this decomposition is driven to zero. We exploit this insight to formulate a general approximation result for optimization problems, which feature discrete and distributed optimization variables, and which are governed by a compact control-to-state operator. We analyze the topology induced by the grid refinements and prove convergence rates of the control vectors for two problem classes. We use a reconstruction problem from signal processing to demonstrate both the applicability of the method outside the scope of differential equations, the predominant case in the literature, and the effectiveness of the approach.

关键词： mixed-integer optimal control Approximation methods Convergence rates Combinatorial integral decomposition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：