检索结果-内蒙古大学图书馆

This Too Shall Pass: Removing Stale Observations in Dynamic Bayesian optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bardou, Anthony Thiran, Patrick Ranieri, Giovanni IC EPFL Lausanne Switzerland

Bayesian optimization (BO) has proven to be very successful at optimizing a static, noisy, costly-to-evaluate black-box function f : S → R. However, optimizing a black-box which is also a function of time (i.e., a dynamic function) f : S×T → R remains a challenge, since a dynamic Bayesian optimization (DBO) algorithm has to keep track of the optimum over time. This changes the nature of the optimization problem in at least three aspects: (i) querying an arbitrary point in S × T is impossible, (ii) past observations become less and less relevant for keeping track of the optimum as time goes by and (iii) the DBO algorithm must have a high sampling frequency so it can collect enough relevant observations to keep track of the optimum through time. In this paper, we design a Wasserstein distance-based criterion able to quantify the relevancy of an observation with respect to future predictions. Then, we leverage this criterion to build W-DBO, a DBO algorithm able to remove irrelevant observations from its dataset on the fly, thus maintaining simultaneously a good predictive performance and a high sampling frequency, even in continuous-time optimization tasks with unknown horizon. Numerical experiments establish the superiority of W-DBO, which outperforms state-of-the-art methods by a comfortable margin. © 2024, CC BY.

关键词： optimization algorithms

Systematic Design of Decentralized algorithms for Consensus optimization

学校读者我要写书评

暂无评论

IEEE CONTROL SYSTEMS LETTERS 2019年第4期3卷 966-971页

作者： Han, Shuo Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA

We propose a separation principle that enables a systematic way of designing decentralized algorithms used in consensus optimization. Specifically, we show that a decentralized optimization algorithm can be constructed by combining a non-decentralized base optimization algorithm and decentralized consensus tracking. The separation principle provides modularity in both the design and analysis of algorithms under an automated convergence analysis framework using integral quadratic constraints (IQCs). We show that consensus tracking can be incorporated into the IQC-based analysis. The workflow is illustrated through the design and analysis of a decentralized algorithm based on the alternating direction method of multipliers.

关键词： optimization algorithms robust control

Parallel IG Algorithm for Matrix Shop AGV Scheduling Problem with Time and Capacity Constraints

学校读者我要写书评

暂无评论

Parallel IG Algorithm for Matrix Shop AGV Scheduling Problem...

2023 International Conference on Automation in Manufacturing, Transportation and Logistics, iCaMaL 2023

作者： Li, Zhongkai Pan, Quanke Wang, Bingtao School of Mechanical Engineering and Automation Shanghai University Shanghai China

ISBN: (纸本)9781839539954

In the current research on scheduling problems, intelligent optimization algorithms have been widely applied by scholars as excellent solutions. The advantage of this method is that it can obtain better solutions for NP-hard problems in a short period of time. The limitation of current optimization methods is that they cannot obtain larger computational space in a short period of time. This paper proposes a parallel iterative greed algorithm (PIGA) that combines intelligent optimization algorithms with parallel computing to achieve larger computational space. In the proposed PIGA, four operators and a restart strategy are used to enhance the algorithm's global and local search capabilities. To verify the performance of the proposed algorithm, PIGA was applied to the matrix workshop automated guided vehicle (AGV) scheduling problem and compared with other optimization algorithms. The experimental results demonstrate the superiority of the proposed algorithm. © The Institution of Engineering & Technology 2023.

关键词： optimization algorithms

Block Coordinate Descent Methods for Structured Nonconvex optimization with Nonseparable Constraints: Optimality Conditions and Global Convergence

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yuan, Zhijie Yuan, Ganzhao Sun, Lei School of Systems Science and Engineering Sun Yat-sen University Guangzhou China Peng Cheng Laboratory Shenzhen China

Coordinate descent algorithms are widely used in machine learning and large-scale data analysis due to their strong optimality guarantees and impressive empirical performance in solving non-convex problems. In this work, we introduce Block Coordinate Descent (BCD) method for structured nonconvex optimization with nonseparable constraints. Unlike traditional large-scale Coordinate Descent (CD) approaches, we do not assume the constraints are separable. Instead, we account for the possibility of nonlinear coupling among them. By leveraging the inherent problem structure, we propose new CD methods to tackle this specific challenge. Under the relatively mild condition of locally bounded non-convexity, we demonstrate that achieving coordinate-wise stationary points offer a stronger optimality criterion compared to standard critical points. Furthermore, under the Luo-Tseng error bound conditions, our BCD methods exhibit Q-linear convergence to coordinate-wise stationary points or critical points. To demonstrate the practical utility of our methods, we apply them to various machine learning and signal processing models. We also provide the geometry analysis for the models. Experiments on real-world data consistently demonstrate the superior objective values of our approaches compared to existing methods. © 2024, CC BY.

关键词： optimization algorithms

Conjugate-Gradient-like Based Adaptive Moment Estimation optimization Algorithm for Deep Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tian, Jiawu Xu, Liwei Zhang, Xiaowei Li, Yongqi School of Mathematical Sciences University of Electronic Science and Technology of China China

Training deep neural networks is a challenging task. In order to speed up training and enhance the performance of deep neural networks, we rectify the vanilla conjugate gradient as conjugate-gradient-like and incorporate it into the generic Adam, and thus propose a new optimization algorithm named CG-like-Adam for deep learning. Specifically, both the first-order and the second-order moment estimation of generic Adam are replaced by the conjugate-gradient-like. Convergence analysis handles the cases where the exponential moving average coefficient of the first-order moment estimation is constant and the first-order moment estimation is unbiased. Numerical experiments show the superiority of the proposed algorithm based on the CIFAR10/100 dataset. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

ON QUASI-CONVEX SMOOTH optimization PROBLEMS BY A COMPARISON ORACLE

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gasnikov, Alexander V. Alkousa, Mohammad S. Lobanov, Alexander V. Dorn, Yuriy V. Stonyakin, Fedor S. Kuruzov, Ilya A. Singh, Sanjeev R. Innopolis University Moscow Institute of Physics and Technology Caucasus Mathematical Center Steklov Mathematical Institute Russian Academy of Sciences Innopolis University Moscow Institute of Physics and Technology Moscow Institute of Physics and Technology Skolkovo Institute of Science and Technology ISP RAS Research Center for Trusted Artificial Intelligence Institute for Artificial Intelligence Lomonosov Moscow State University Moscow Institute of Physics and Technology Moscow Institute of Physics and Technology V. I. Vernadsky Crimean Federal University Innopolis University Research Center for Artificial Intelligence Innopolis University Times School of Media Bennett University

Frequently, when dealing with many machine learning models, optimization problems appear to be challenging due to a limited understanding of the constructions and characterizations of the objective functions in these problems. Therefore, major complications arise when dealing with first-order algorithms, in which gradient computations are challenging or even impossible in various scenarios. For this reason, we resort to derivative-free methods (zeroth-order methods). This paper is devoted to an approach to minimizing quasi-convex functions using a recently proposed (in [56]) comparison oracle only. This oracle compares function values at two points and tells which is larger, thus by the proposed approach, the comparisons are all we need to solve the optimization problem under consideration. The proposed algorithm to solve the considered problem is based on the technique of comparison-based gradient direction estimation and the comparison-based approximation normalized gradient descent. The normalized gradient descent algorithm is an adaptation of gradient descent, which updates according to the direction of the gradients, rather than the gradients themselves. We proved the convergence rate of the proposed algorithm when the objective function is smooth and strictly quasi-convex in Rn, this algorithm needs O ((nD2/Ε2) log (nD/Ε)) comparison queries to find an Ε-approximate of the optimal solution, where D is an upper bound of the distance between all generated iteration points and an optimal solution. © 2024, CC BY.

关键词： optimization algorithms

Simulation-based optimization as a method for dealing with complexity in energy system modeling

学校读者我要写书评

暂无评论

Research Square

Research Square 2024年

作者： Jürgens, Patrick Müller, Paul Brandhuber, Fritz Kost, Christoph Fraunhofer Institute for Solar Energy Systems ISE Heidenhofstraße 2 Freiburg79110 Germany Albert-Ludwigs Universität Freiburg Department of Sustainable Systems Engineering INATECH Emmy-Noether-Straße 2 Freiburg79110 Germany

To reflect the complexity of the energy transition, a current challenge in modeling national energy transition pathways is to combine high resolution in time, space, techno-economic and sector coupling details in a single model. To address this challenge, the paper discusses the simulation-based optimization approach used in the energy system model REMod as an alternative to linear optimization. The REMod model uses a simulation of the energy system coupled with a black-box optimization algorithm to optimize the transformation path. To limit the computational complexity, taking into account the hourly operation of the energy system along the whole transformation path, various aspects have to be considered: performance of the simulation, choice of the optimization algorithm and selection of the termination criterion and population size. The model employs a novel method of endogenous interpolation to incorporate the entire transformation path into the objective function, and it can be evaluated in parallel. This allows to increase the complexity both in technological details and in geographical resolution, leading to a model with high resolution both in sector coupling and technological details, as well as in time and space. © 2024, CC BY.

关键词： optimization algorithms

A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ye, Feiyang Lin, Baijiong Cao, Xiaofeng Zhang, Yu Tsang, Ivor W. Department of Computer Science and Engineering Southern University of Science and Technology China Australian Artificial Intelligence Institute University of Technology Sydney Australia China School of Artificial Intelligence Jilin University China Shanghai Artificial Intelligence Laboratory China Centre for Frontier AI Research Agency for Science Technology and Research Singapore Institute of High Performance Computing Agency for Science Technology and Research School of Computer Science and Engineering Nanyang Technological University Singapore

In this paper, we study the Multi-Objective Bi-Level optimization (MOBLO) problem, where the upper-level subproblem is a multi-objective optimization problem and the lower-level subproblem is for scalar optimization. Existing gradient-based MOBLO algorithms need to compute the Hessian matrix, causing the computational inefficient problem. To address this, we propose an efficient first-order multi-gradient method for MOBLO, called FORUM. Specifically, we reformulate MOBLO problems as a constrained multi-objective optimization (MOO) problem via the value-function approach. Then we propose a novel multi-gradient aggregation method to solve the challenging constrained MOO problem. Theoretically, we provide the complexity analysis to show the efficiency of the proposed method and a non-asymptotic convergence result. Empirically, extensive experiments demonstrate the effectiveness and efficiency of the proposed FORUM method in different learning problems. In particular, it achieves state-of-the-art performance on three multi-task learning benchmark datasets. The code is available at https://***/Baijiong-Lin/FORUM. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

An Elite-Inspired Evolutionary Algorithm for Large-Scale Multiobjective optimization

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Cheng, Du Xu, Zhiguo Yu, Fanhua Li, Qingliang Zhou, Ning Department of Artificial Intelligence Jilin University Jilin Province China Department of Mathematics Jilin University Jilin Province China Department of Computer Science Beihua University Jilin Province China Department of Computer Science Changchun Normal University Jilin Province China Department of Mathematics Changchun University of Technology Jilin Province China College of Computer Science and Technology

In large-scale multi-objective optimization problems, the search space increases exponentially with the dimensionality of decision variables. This vast search space often contains multiple local optima, making it particularly challenging to identify the global optimum. Existing large-scale algorithms based on problem transformation may result in the loss of problem-specific information, while those relying on decision variable grouping are heavily dependent on the grouping method. Moreover, these methods often require the pre-setting of several key sensitive parameters, which can adversely affect the algorithm's generalization capability. To address these issues, we propose an elite-inspired large-scale multi-objective evolutionary algorithm (EIEA) that balances exploration and exploitation through the learning mechanisms of elite and ordinary individuals. In this algorithm, elite individuals enhance exploitation through mutual learning and improve exploration through self-learning, aiding in escaping local optima. Ordinary individuals improve convergence by learning from elite individuals and further enhance exploration through mutation. Additionally, a reference vector-based environmental selection mechanism is employed to increase the algorithm's diversity. Experiments on large-scale multi-objective optimization problems with up to 2000 decision variables demonstrate that our proposed algorithm is highly competitive compared to six state-of-the-art large-scale multi-objective evolutionary algorithms. © 2024, The Authors. All rights reserved.

关键词： optimization algorithms