Meta-Reinforcement Learning (MRL) is a promising framework for training agents that can quickly adapt to new environments and tasks. In this work, we study the MRL problem under the policy gradient formulation, where ...
Meta-Reinforcement Learning (MRL) is a promising framework for training agents that can quickly adapt to new environments and tasks. In this work, we study the MRL problem under the policy gradient formulation, where we propose a novel algorithm that uses Moreau envelope surrogate regularizers to jointly learn a meta-policy that is adjustable to the environment of each individual task. Our algorithm, called Moreau Envelope Meta-Reinforcement Learning (MEMRL), learns a meta-policy that can adapt to a distribution of tasks by efficiently updating the policy parameters using a combination of gradient-based optimization and Moreau Envelope regularization. Moreau Envelopes provide a smooth approximation of the policy optimization problem, which enables us to apply standard optimization techniques and converge to an appropriate stationary point. We provide a detailed analysis of the MEMRL algorithm, where we show a sublinear convergence rate to a first-order stationary point for non-convex policy gradient optimization. We finally show the effectiveness of MEMRL on a multi-task 2D-navigation problem.
Surface reconstruction is a classical process in industrial engineering and manufacturing, particularly in reverse engineering, where the goal is to obtain a digital model from a physical object. For that purpose, the...
详细信息
Surface reconstruction is a classical process in industrial engineering and manufacturing, particularly in reverse engineering, where the goal is to obtain a digital model from a physical object. For that purpose, the real object is typically scanned or digitized and the resulting point cloud is then fitted to mathematical surfaces through numerical optimization. The choice of the approximating functions is crucial for the accuracy of the process. Real-world objects such as manufactured workpieces often require complex nonlinear approximating functions, which are not well suited for standard numerical methods. In a previous paper presented at the ISMSI 2023 conference, we addressed this issue by using manually selected approximation functions via optimization through the cuckoo search algorithm with Lévy flights. Building upon that work, this paper presents an enhanced and extended method for surface reconstruction by using height-map surfaces obtained through a combination of exponential, polynomial and logarithmic functions. A feasible approach for this goal is to consider continuous bivariate distribution functions, which ensures consistent models along with good mathematical properties for the output shapes, such as smoothness and integrability. However, this approach leads to a difficult multivariate, constrained, multimodal continuous nonlinear optimization problem. To tackle this issue, we apply particle swarm optimization, a popular swarm intelligence technique for continuous optimization. The method is hybridized with a local search procedure for further improvement of the solutions and applied to a benchmark of 15 illustrative examples of point clouds fitted to different surface models. The performance of the method is analyzed through several computational experiments. The numerical and graphical results show that the method is able to recover the shape of the point clouds accurately and automatically. Furthermore, our approach outperforms other alternati
This paper considers the decentralized (discrete) optimal transport (D-OT) problem. In this setting, a network of agents seeks to design a transportation plan jointly, where the cost function is the sum of privately h...
详细信息
ISBN:
(数字)9798350382655
ISBN:
(纸本)9798350382662
This paper considers the decentralized (discrete) optimal transport (D-OT) problem. In this setting, a network of agents seeks to design a transportation plan jointly, where the cost function is the sum of privately held costs for each agent. We reformulate the D-OT problem as a constraint-coupled optimization problem and propose a single-loop decentralized algorithm with an iteration complexity of
$O(1/\epsilon)$
that matches existing centralized first-order approaches. Moreover, we propose the decentralized equitable optimal transport (DE-OT) problem. In DE-OT, in addition to cooperatively designing a transportation plan that minimizes transportation costs, agents seek to ensure equity in their individual costs. The iteration complexity of the proposed method to solve DE-OT is also
$O(1/\epsilon)$
. This rate improves existing centralized algorithms, where the best iteration complexity obtained is
$O(1/\epsilon^{2})$
.
Existing decentralized algorithms usually require knowledge of problem parameters for updating local iterates. For example, the hyperparameters (such as learning rate) usually require the knowledge of Lipschitz consta...
详细信息
In this paper we consider a stochastic SEIQR (susceptible-exposed-infected-quarantined-recovered) epidemic model with a generalized incidence function. Using the Lyapunov method, we establish the existence and uniquen...
详细信息
One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state sp...
详细信息
ISBN:
(数字)9798350316339
ISBN:
(纸本)9798350316346
One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state space geometry, dynamics, and spectral structure of a system have been used to characterize the well-posedness of this problem, without assumptions on the inputs. However, certain structural assumptions, such as input sparsity, have been shown to translate to practical gains in the performance of inversion algorithms, surpassing classical guarantees. Establishing necessary and sufficient conditions for left invertibility of systems with sparse inputs is therefore a crucial step toward understanding the performance limits of system inversion under structured input assumptions. In this work, we provide the first necessary and sufficient characterizations of left invertibility for linear systems with sparse inputs, echoing classic characterizations for standard linear systems. The key insight in deriving these results is in establishing the existence of two novel geometric invariants unique to the sparseinput setting, the weakly unobservable and strongly reachable subspace arrangements. By means of a concrete example, we demonstrate the utility of these characterizations. We conclude by discussing extensions and applications of this framework to several related problems in sparse control.
In this paper we consider the estimation of unknown parameters in Bayesian inverse problems. In most cases of practical interest, there are several barriers to performing such estimation, This includes a numerical app...
详细信息
The simulation of many complex phenomena in engineering and science requires solving expensive, high-dimensional systems of partial differential equations (PDEs). To circumvent this, reduced-order models (ROMs) have b...
详细信息
Data-driven approaches achieve remarkable results for the modeling of complex dynamics based on collected data. However, these models often neglect basic physical principles which determine the behavior of any real-wo...
详细信息
HfO2- and ZrO2-based ferroelectric thin films have emerged as promising candidates for the gate oxides of next generation electronic devices. Recent work has experimentally demonstrated that a tetragonal/orthorhombic ...
详细信息
暂无评论