检索结果-内蒙古大学图书馆

Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ma, Yuhan Sun, Dan Gao, Erdi Sang, Ningjing Li, Iris Huang, Guanming Johns Hopkins University Baltimore United States Washington University in St. Louis St. Louis United States New York University New York United States Columbia University New York United States University of Chicago Chicago United States

optimization theory serves as a pivotal scientific instrument for achieving optimal system performance, with its origins in economic applications to identify the best investment strategies for maximizing benefits. Over the centuries, from the geometric inquiries of ancient Greece to the calculus contributions by Newton and Leibniz, optimization theory has significantly advanced. The persistent work of scientists like Lagrange, Cauchy, and von Neumann has fortified its progress. The modern era has seen an unprecedented expansion of optimization theory applications, particularly with the growth of computer science, enabling more sophisticated computational practices and widespread utilization across engineering, decision analysis, and operations research. This paper delves into the profound relationship between optimization theory and deep learning, highlighting the omnipresence of optimization problems in the latter. We explore the gradient descent algorithm and its variants, which are the cornerstone of optimizing neural networks. The chapter introduces an enhancement to the SGD optimizer, drawing inspiration from numerical optimization methods, aiming to enhance interpretability and accuracy. Our experiments on diverse deep learning tasks substantiate the improved algorithm's efficacy. The paper concludes by emphasizing the continuous development of optimization theory and its expanding role in solving intricate problems, enhancing computational capabilities, and informing better policy decisions. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

An Adaptive Genetic Algorithm for determining optimal structures for atomic clusters

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Willnecker, Brandon Moodley, Mervlyn School of Chemistry and Physics University of KwaZulu-Natal Westville Campus Private Bag X54001 Durban4000 South Africa

The implementation of adaptive genetic algorithms (AGA) for optimization problems has proven to be superior than many other methods due to its nature of producing more robust and high quality solutions. Considering the complexity involved in many-body simulations, a novel AGA is proposed for applications to such systems and is specifically used to determine the lowest energy structures of various sized atomic clusters. For demonstrative purposes, we apply our method to various sized Lennard-Jones clusters and show that our results are more accurate than those found in the literature employing different methods. © 2024, CC BY.

关键词： optimization algorithms

Constraint Preconditioning and Parameter Selection for a First-Order Primal-Dual Method applied to Model Predictive Control

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chari, Govind M. Yu, Yue Açıkmeşe, Behçet Department of Aeronautics & Astronautics the University of Washington SeattleWA98195 United States Department of Aerospace Engineering and Mechanics the University of Minnesota Twin Cities MinneapolisMN55455 United States Department of Aeronautics & Astronautics the University of Washington SeattleWA98195 United States

Many techniques for real-time trajectory optimization and control require the solution of optimization problems at high frequencies. However, ill-conditioning in the optimization problem can significantly reduce the speed of first-order primal-dual optimization algorithms. We introduce a preconditioning technique and step-size heuristic for Proportional-Integral Projected Gradient (PIPG), a first-order primal-dual algorithm. The preconditioning technique, based on the QR factorization, aims to reduce the condition number of the KKT matrix associated with the optimization problem. Our step-size selection heuristic chooses step-sizes to minimize the upper bound on the convergence of the primal-dual gap for the optimization problem. These algorithms are tested on two model predictive control problem examples and show a solve-time reduction of at least 3.6x. © 2024, CC BY-NC-ND.

关键词： optimization algorithms

Optimal Scheduling Model of Power System Based on Multi-Objective Evolutionary Algorithm

学校读者我要写书评

暂无评论

Engineering Intelligent Systems

Engineering Intelligent Systems 2024年第5期32卷 543-552页

作者： Zhang, Zhenyuan School of Electronic and Electrical Engineering Zibo Vocational Institute Shandong Zibo255300 China

In today’s increasingly prosperous society, almost every industry requires electricity to support its operations, and whether the power system can operate stably and safely has been closely related to the steady development of the economy. Optimal scheduling research plays a decisive role in power system operation and control. In order to ensure reliable power supply and power quality, it is important to optimize the operational efficiency of the power system, so that the system can provide greater economic benefits. Traditional multi-objective optimization methods have major shortcomings, and multi-objective optimization methods based on traditional mathematical planning principles usually have certain vulnerabilities in terms of practical engineering optimization problems, so there is a need for in-depth research on efficient and practical multi-objective optimization algorithms and theories. In this paper, the modeling technology of the power information-physical system is studied and the multi-objective scheduling optimization of power system is explored using a multi-objective evolutionary algorithm. The algorithm analysis results showed that compared with the traditional dispatching model, the proposed multi-objective evolutionary algorithm not only improved the optimization effect of the power system by about 8.95%, but also offered decision makers guidance on dispatching, with good convergence speed and accuracy compared with previous optimization methods and solutions. © 2024 CRL Publishing Ltd.

关键词： optimization algorithms

OPTIMIZING THE CGMS UPPER BOUND ON RAMSEY NUMBERS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gupta, Parth Ndiaye, Ndiamé Norin, Sergey Wei, Louis Department of Mathematics and Statistics McGill University MontréalQC Canada

In a recent breakthrough Campos, Griffiths, Morris and Sahasrabudhe obtained the first exponential improvement of the upper bound on the diagonal Ramsey numbers since 1935. We shorten their proof, replacing the underlying book algorithm with a simple inductive statement. This modification allows us to give a very short proof of an improved upper bound on the off-diagonal Ramsey numbers, which extends to the multicolor setting, clarify the dependence of the bounds on underlying parameters and optimize these parameters, obtaining, in particular, an upper bound R(k, k) ≤ (3.8)k+o(k)on the diagonal Ramsey numbers. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Accelerated Bregman gradient methods for relatively smooth and relatively Lipschitz continuous minimization problems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Savchuk, O.S. Alkousa, M.S. Shushko, A.S. Vyguzov, A.A. Stonyakin, F.S. Pasechnyuk, D.A. Gasnikov, A.V. V. I. Vernadsky Crimean Federal University Vernadsky Avenue 4 Simferopol Russia Adyghe State University 208 Pervomayskaya st. Maykop Russia Innopolis University Universitetskaya Str. 1 Innopolis420500 Russia Moscow Institute of Physics and Technology 9 Institutsky lane Dolgoprudny141701 Russia Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi United Arab Emirates Ivannikov Institute for System Programming The Russian Academy of Sciences Russia

In this paper, we propose some accelerated methods for solving optimization problems under the condition of relatively smooth and relatively Lipschitz continuous functions with an inexact oracle. We consider the problem of minimizing the convex differentiable and relatively smooth function concerning a reference convex function. The first proposed method is based on a similar triangles method with an inexact oracle, which uses a special triangular scaling property for the used Bregman divergence. The other proposed methods are non-adaptive and adaptive (tuning to the relative smoothness parameter) accelerated Bregman proximal gradient methods with an inexact oracle. These methods are universal in the sense that they are applicable not only to relatively smooth but also to relatively Lipschitz continuous optimization problems. We also introduced an adaptive intermediate Bregman method which interpolates between slower but more robust algorithms non-accelerated and faster, but less robust accelerated algorithms. We conclude the paper with the results of numerical experiments demonstrating the advantages of the proposed algorithms for the Poisson inverse problem. © 2024, CC BY.

关键词： optimization algorithms

Fast Algorithm for Fuel-Optimal Impulsive Control of Linear Systems With Time-Varying Cost

学校读者我要写书评

暂无评论

IEEE TRANSACTIONS ON AUTOMATIC CONTROL 2021年第9期66卷 4029-4042页

作者： Koenig, Adam W. D'Amico, Simone Stanford Univ Dept Aeronaut & Astronaut Stanford CA 94305 USA

This article presents a new fast and robust algorithm that provides fuel-optimal impulsive control input sequences that drive a linear time-variant system to a desired state at a specified time. This algorithm is applicable to a broad class of problems where the cost is expressed as a time-varying norm-like function of the control input, enabling inclusion of complex operational constraints in the control planning problem. First, it is shown that the reachable sets for this problem have identical properties to those in prior works using constant cost functions, enabling use of existing algorithms in conjunction with newly derived contact and support functions. By reformulating the optimal control problem as a semi-infinite convex program, it is also demonstrated that the semi-infinite component of the commonly studied primer vector is an outward normal vector to the reachable set at the target state. Using this formulation, a fast and robust algorithm that provides globally optimal impulsive control input sequences is proposed. The algorithm iteratively refines estimates of an outward normal vector to the reachable set at the target state and a minimal set of control input times until the optimality criteria are satisfied to within a user-specified tolerance. Next, optimal control inputs are computed by solving a quadratic program. The algorithm is validated through simulations of challenging example problems based on the recently proposed miniaturized distributed occulter/telescope small satellite mission, which demonstrate that the proposed algorithm converges several times faster than comparable algorithms in the literature.

关键词： Space vehicles Cost function Optimal control Heuristic algorithms Robustness Computational modeling Computational methods linear systems optimization algorithms time-varying systems

Mfwoa: Multifactorial Whale Optimisation Algorithm

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Ye, Lei Xiang, Benhua Ding, Hangqi Wu, Yue Gong, Maoguo Test center National University of Defense Technology Xi’an710100 China School of Computer Science and Technology Xidian University Xi’an710071 China School of Electronic Engineering Xidian University Xi’an710071 China Key Laboratory of Collaborative Intelligence Systems Ministry of Education Xi’an710071 China College of Mathematics Science Inner Mongolia Normal University Hohhot010028 China

Multi-task optimisation is a new research topic in the area of evolutionary computing, and it has received extensive attention from researchers since it was proposed. It can exploit the potential synergistic facilitation between tasks to solve multiple optimisation problems simultaneously and efficiently. Whale optimisation algorithm (WOA) is an emerging swarm intelligence algorithm, which has the advantages of rapid convergence and strong global exploration ability. Therefore, this article is the first attempt to use WOA as a search mechanism to implement multi-task optimisation, and proposes the multifactorial whale optimisation algorithm (MFWOA). Among them, the inter-task knowledge transfer strategy which plays a key role is designed in detail. This strategy is implemented in two ways. The first way is to exchange the search experience by adding distance terms from other tasks. Another way is to generate a new random individual or optimal individual through crossover and mutation operators, and then use them to guide position updates. By combining the two ways, it can explore a wider area. In addition, to better balance the useful information transfer between tasks and within tasks, a random mating probability parameter self-adaptive strategy is designed. Finally, we provide an exhaustive analysis of the overall optimisation ability of the proposed algorithm. The experimental findings demonstrate that the algorithm achieves effective and efficient knowledge transfer and can achieve better convergence speed and accuracy, showing that the algorithm is a promising emerging multi-task optimisation algorithm. © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Prediction of lining upward movement during shield tunneling using machine learning algorithms and field monitoring data

学校读者我要写书评

暂无评论

TRANSPORTATION GEOTECHNICS 2023年 41卷

作者： Ye, Xiao-Wei Zhang, Xiao-Long Zhang, Hong-Qian Ding, Yang Chen, Yun-Min Zhejiang Univ MOE Key Lab Soft Soils & Geoenvironm Engn Hangzhou 310058 Peoples R China Zhejiang Univ Dept Civil Engn Hangzhou 310058 Peoples R China

During the construction of shield tunnels, the tunnel lining often has the problem of local or overall upward movement in soft soil areas. Excessive upward movement will lead to lining dislocation, cracks, damage, and even axis deviation. This paper elaborates on how to predict the process of tunnel lining upward movement using machine learning algorithms and field monitoring data systematically. First, fourteen input variables including shield operational parameters, tunnel geometry, geological conditions and anomalous condition are considered to predict the upward displacement of twelve output variables that represent the process of the upward move-ment of the tunnel lining. In addition, 80% field monitoring data (81 datasets) are selected randomly as the training set, and the remaining 20% (20 datasets) are the test set. Then, the average of 5-fold cross validation mean absolute error is regarded as the fitness function of optimization algorithms to find the optimal hyper-parameters. Finally, the prediction performance of four machine learning (ML) algorithms back-propagation neural network (BPNN), general regression neural network (GRNN), extreme learning machine (ELM), and support vector machine (SVM) optimized by particle swarm optimization (PSO) and genetic algorithm (GA) were compared. All ML algorithms except BPNN predicted successfully the trend of upward movement of tunnel lining. In particular, PSO-GRNN accurately captures the evolution of upward displacement in different periods of each ring with the lowest errors and the largest correlation coefficient values.

关键词： Structural health monitoring Shield tunneling construction Upward movement prediction Lining displacement Machine learning (ML) optimization algorithms