检索结果-内蒙古大学图书馆

How Free is Parameter-Free Stochastic optimization?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Attia, Amit Koren, Tomer Blavatnik School of Computer Science Tel Aviv University Blavatnik School of Computer Science Tel Aviv University Google Research Tel Aviv

We study the problem of parameter-free stochastic optimization, inquiring whether, and under what conditions, do fully parameter-free methods exist: these are methods that achieve convergence rates competitive with optimally tuned methods, without requiring significant knowledge of the true problem parameters. Existing parameter-free methods can only be considered "partially" parameter-free, as they require some non-trivial knowledge of the true problem parameters, such as a bound on the stochastic gradient norms, a bound on the distance to a minimizer, etc. In the non-convex setting, we demonstrate that a simple hyperparameter search technique results in a fully parameter-free method that outperforms more sophisticated state-of-the-art algorithms. We also provide a similar result in the convex setting with access to noisy function values under mild noise assumptions. Finally, assuming only access to stochastic gradients, we establish a lower bound that renders fully parameter-free stochastic convex optimization infeasible, and provide a method which is (partially) parameter-free up to the limit indicated by our lower bound. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Vanilla Bayesian optimization Performs Great in High Dimensions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hvarfner, Carl Hellsten, Erik Orm Nardi, Luigi Lund University Lund Sweden DBtune Malmö Sweden

High-dimensional problems have long been considered the Achilles’ heel of Bayesian optimization. Spurred by the curse of dimensionality, a large collection of algorithms aim to make it more performant in this setting, commonly by imposing various simplifying assumptions on the objective. In this paper, we identify the degeneracies that make vanilla Bayesian optimization poorly suited to high-dimensional tasks, and further show how existing algorithms address these degeneracies through the lens of lowering the model complexity. Moreover, we propose an enhancement to the prior assumptions that are typical to vanilla Bayesian optimization, which reduces the complexity to manageable levels without imposing structural restrictions on the objective. Our modification - a simple scaling of the Gaussian process lengthscale prior with the dimensionality - reveals that standard Bayesian optimization works drastically better than previously thought in high dimensions, clearly outperforming existing state-of-the-art algorithms on multiple commonly considered real-world high-dimensional tasks. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Bayesian optimization of Bilevel Problems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ekmekcioglu, Omer Aydin, Nursen Branke, Juergen Warwick Business School University of Warwick United Kingdom

Bilevel optimization, a hierarchical mathematical framework where one optimization problem is nested within another, has emerged as a powerful tool for modeling complex decision-making processes in various fields such as economics, engineering, and machine learning. This paper focuses on bilevel optimization where both upper-level and lower-level functions are black boxes and expensive to evaluate. We propose a Bayesian optimization framework that models the upper and lower-level functions as Gaussian processes over the combined space of upper and lower-level decisions, allowing us to exploit knowledge transfer between different sub-problems. Additionally, we propose a novel acquisition function for this model. Our experimental results demonstrate that the proposed algorithm is highly sample-efficient and outperforms existing methods in finding high-quality solutions. © 2024, CC BY-NC-ND.

关键词： optimization algorithms

Stopping Bayesian optimization with Probabilistic Regret Bounds

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wilson, James T. Morgan Stanley New York United States

Bayesian optimization is a popular framework for efficiently tackling black-box search problems. As a rule, these algorithms operate by iteratively choosing what to evaluate next until some predefined budget has been exhausted. We investigate replacing this de facto stopping rule with criteria based on the probability that a point satisfies a given set of conditions. We focus on the prototypical example of an (ϵ, δ)-criterion: stop when a solution has been found whose value is within ϵ > 0 of the optimum with probability at least 1 − δ under the model. For Gaussian process priors, we show that Bayesian optimization satisfies this criterion under mild technical assumptions. Further, we give a practical algorithm for evaluating Monte Carlo stopping rules in a manner that is both sample efficient and robust to estimation error. These findings are accompanied by empirical results which demonstrate the strengths and weaknesses of the proposed approach. © 2024, CC BY.

关键词： optimization algorithms

Continuous and discrete-time accelerated methods for an inequality constrained convex optimization problem

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Juan Huang, Nan-Jing Long, Xian-Jun Li, Xue-Song Department of Mathematics Sichuan University Sichuan Chengdu610064 China College of Mathematics and Statistics Chongqing Technology and Business University Chongqing400067 China

This paper is devoted to the study of acceleration methods for an inequality constrained convex optimization problem by using Lyapunov functions. We first approximate such a problem as an unconstrained optimization problem by employing the logarithmic barrier function. Using the Hamiltonian principle, we propose a continuous-time dynamical system associated with a Bregman Lagrangian for solving the unconstrained optimization problem. Under certain conditions, we demonstrate that this continuous-time dynamical system exponentially converges to the optimal solution of the inequality constrained convex optimization problem. Moreover, we derive several discrete-time algorithms from this continuous-time framework and obtain their optimal convergence rates. Finally, we present numerical experiments to validate the effectiveness of the proposed algorithms. © 2024, CC BY.

关键词： optimization algorithms

SPINEX-optimization: Similarity-based Predictions with Explainable Neighbors Exploration for Single, Multiple, and Many Objectives optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Naser, M.Z. Naser, Ahmed Z. Clemson University United States Clemson University United States Department of Mechanical Engineering University of Manitoba Canada

This article introduces an expansion within SPINEX (Similarity-based Predictions with Explainable Neighbors Exploration) suite, now extended to single, multiple, and many objective optimization problems. The newly developed SPINEX-optimization algorithm incorporates a nuanced approach to optimization in low and high dimensions by accounting for similarity across various solutions. We conducted extensive benchmarking tests comparing SPINEX-optimization against ten single and eight multi/many optimization algorithms over 55 mathematical benchmarking functions and realistic scenarios. Then, we evaluated the performance of the proposed algorithm in terms of scalability and computational efficiency across low and high dimensions, number of objectives, and population sizes. The results indicate that SPINEX-optimization consistently outperforms most algorithms and excels in managing complex scenarios – especially in high dimensions. The algorithm's capabilities in explainability, Pareto efficiency, and moderate complexity are highlighted through in-depth experiments and visualization methods. © 2024, CC BY.

关键词： optimization algorithms

A meta-heuristic optimization-based approach for 3D simplified parametric analysis of embedded soil-foundation systems undergoing coupled horizontal-rocking vibrations

学校读者我要写书评

暂无评论

COMPUTERS AND GEOTECHNICS 2023年第1期155卷

作者： Chen, Shi-Shuenn Shi, Jun-Yang Doan, Minh -Tam Natl Taiwan Univ Sci & Technol Dept Civil & Construct Engn Taipei Taiwan Natl Univ Kaohsiung Dept Civil & Environm Engn Kaohsiung Taiwan

This paper presents a novel modeling approach based on meta-heuristic optimization for constructing adaptive lumped-parameter models to approximate the dynamic response of a shallow foundation in layered soil considering coupled horizontal and rocking vibrations. The developed model is adaptive to complexities of impedance functions for different types of layered soil deposits. The optimal parameters of the proposed model are computed using three optimization algorithms, with the forensic-based investigation outperforming the others and being chosen as the best optimization strategy for numerical investigations. A parametric study of foundation characteristics and soil properties on frequency response curves, including magnification factors and phase angle, is performed to comprehensively verify the proposed model's appropriateness. The findings reveal that dynamic responses of the proposed simplified systems are perfectly consistent with those computed using existing impedances and worldwide-recognized software in both frequency and time domains. It shall be concluded that the proposed model is compatible with different soil profiles and more effective than existing models in approximating the unbounded behavior of layered soil. Furthermore, it offers great potential to implement the time-history analysis of interacting systems subjected to forced vibrations and seismic motions in commercial software based on the proposed model for engineering practice.

关键词： Soil-structure interaction Lumped -parameter models Layered soil optimization algorithms

Exploring Monotone Priority Queues for Dijkstra optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Costa, Jonas Castro, Lucas de Freitas, Rosiane PPGI - Instituto de Computação Universidade Federal do Amazonas Manaus Brazil

This paper presents a comprehensive overview of monotone priority queues, focusing on their evolution and application in shortest path algorithms. Monotone priority queues are characterized by the property that their minimum key does not decrease over time, making them particularly effective for label-setting algorithms like Dijkstra’s. Some key data structures within this category are explored, emphasizing those derived directly from Dial’s algorithm, including variations of multi-level bucket structures and radix heaps. Theoretical complexities and practical considerations of these structures are discussed, with insights into their development and refinement provided through a historical timeline. © 2024, CC BY.

关键词： optimization algorithms

Randomized directional search for nonconvex optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yuxuan Xing, Wenxun Department of Mathematical Sciences Tsinghua University Beijing100084 China

Direct search methods are a class of popular global optimization algorithms for general nonconvex programs. In this paper, we propose a randomized directional search algorithm (RDSA) for globally solving nonconvex optimization problems. The convergence of RDSA to a global optimizer is established and its computational complexity is derived. Additionally, IRDSA is proposed as an improved algorithm of RDSA. We then apply both RDSA and IRDSA to nonconvex programs with multiple local optima. Results demonstrate that RDSA and IRDSA exhibit remarkable efficacy to identify the global optimum and outperform other global optimization algorithms in terms of accuracy, efficiency, and *** Codes 90C26, 90C56, 90C60 © 2024, CC BY.

关键词： optimization algorithms