检索结果-内蒙古大学图书馆

consensus, error estimates and applications of first- and second-order consensus-based optimization algorithms

MATHEMATICAL MODELS & METHODS IN APPLIED SCIENCES 2025年第2期35卷 345-401页

作者： Byeon, Junhyeok Ha, Seung-Yeal Hwang, Gyuyoung Ko, Dongnam Yoon, Jaeyoung Seoul Natl Univ Dept Stat Seoul 08826 South Korea Seoul Natl Univ Res Inst Basic Sci Seoul 08826 South Korea Seoul Natl Univ Dept Math Sci Seoul 08826 South Korea Seoul Natl Univ Res Inst Math Seoul 08826 South Korea Inst for Basic Sci Korea Pioneer Res Ctr Math & Computat Sci Biomed Math Grp Daejeon 34126 South Korea Catholic Univ Korea Dept Math Bucheon 14662 South Korea Tech Univ Munich Sch Computat Informat & Technol Dept Math Boltzmannstr 3 D-85748 Garching Germany

Swarm-intelligence has received a lot of attention in relation to the modeling of cooperative control of drones, unmanned vehicles and robots from engineering, and meta-heuristic optimizers are often used by researchers from diverse disciplines from sociology to engineering. To name a few, ant-colony algorithm, artificial bee colony algorithm, genetic algorithm and particle swarm optimization correspond to such meta-heuristic optimizers driven by biology. Despite their handy implementations and broad coverage to diverse optimization problems, their rigorous theoretical analysis such as convergence, stability and error estimates are not well understood. Recently, consensus-based optimization (CBO) algorithm was proposed in the realm of applied mathematics and its theoretical analysis has been done, e.g. consensus and error estimates of CBO. In this paper, we provide a survey on the state-of-the-art results on the quantitative estimates and applications of first-order and second-order CBOs which are derivative-free optimization method that employs a multi-point aggregation model. Although it was motivated by the particle swarm optimization algorithm, CBO admits theoretical analysis such as convergence and error estimates under suitable frameworks. This paper offers a comprehensive summary of recent achievement concerning consensus, convergence, and performance results in first- and second-order time-discrete CBO algorithms. We provide numerical experiments, illustrating operations on various random networks and examining the effects on computation time and minimization performance under different parameters. Furthermore, we also discuss several applications in finance and real-world optimization problems.

关键词： consensus-based optimization global optimization mean-field limit particle swarm optimization

来源：评论

学校读者我要写书评

暂无评论

A PDE framework of consensus-based optimization for objectives with multiple global minimizers

引用

COMMUNICATIONS IN PARTIAL DIFFERENTIAL EQUATIONS 2025年第4期50卷 493-541页

作者： Fornasier, Massimo Sun, Lukang Tech Univ Munich Sch Computat Informat & Technol Dept Math Munich Germany Munich Ctr Machine Learning Munich Germany Munich Data Sci Inst Munich Germany King Abdullah Univ Sci & Technol Thuwal Saudi Arabia KAUST AI Initiat Thuwal Saudi Arabia

consensus-based optimization (CBO) is an agent-based derivative-free method for non-smooth global optimization that has been introduced in 2017, leveraging a surprising interplay between stochastic exploration and Laplace's principle. In addition to its versatility and effectiveness in handling high-dimensional, non-convex, and non-smooth optimization problems, this approach lends itself well to theoretical analysis. Indeed, its dynamics is governed by a degenerate nonlinear Fokker-Planck equation, whose large time behavior explains the convergence of the method. Recent results provide guarantees of convergence under the restrictive assumption of unique global minimizer for the objective function. In this work, we propose a novel and simple variation of CBO to tackle non-convex optimization problems with multiple global minimizers. Despite the simplicity of this new model, its analysis is particularly challenging because of its nonlinearity and nonlocal nature. We prove existence of solutions of the corresponding nonlinear Fokker-Planck equation and we show exponential concentration in time to the set of minimizers made of multiple smooth, convex, and compact components. Our proofs require combining several ingredients, such as delicate geometrical arguments, new variants of a quantitative Laplace principle, ad hoc regularizations and approximations, and regularity theory for parabolic equations. Ultimately, this result suggests that the corresponding CBO algorithm, formulated as an Euler-Maruyama discretization of the underlying empirical stochastic process, tends to converge to multiple global minimizers.

关键词： global optimization derivative-free optimization consensus-based optimization stochastic differential equations nonlinear Fokker-Planck equations well-posedness large time asymptotics

来源：评论

学校读者我要写书评

暂无评论

Polarized consensus-based dynamics for optimization and sampling

引用

MATHEMATICAL PROGRAMMING 2025年第1-2期211卷 125-155页

作者： Bungert, Leon Roith, Tim Wacker, Philipp Univ Wurzburg Inst Math WurzburgEmil F Str 40 D-97074 Wurzburg Germany Deutsch Elektronen Synchrotron DESY Computat Imaging Helmholtz Imaging Notkestr 85 D-22607 Hamburg Germany Univ Canterbury Sch Math & Stat Sci Rd Christchurch 8140 New Zealand

In this paper we propose polarized consensus-based dynamics in order to make consensus-based optimization (CBO) and sampling (CBS) applicable for objective functions with several global minima or distributions with many modes, respectively. For this, we "polarize" the dynamics with a localizing kernel and the resulting model can be viewed as a bounded confidence model for opinion formation in the presence of common objective. Instead of being attracted to a common weighted mean as in the original consensus-based methods, which prevents the detection of more than one minimum or mode, in our method every particle is attracted to a weighted mean which gives more weight to nearby particles. We prove that in the mean-field regime the polarized CBS dynamics are unbiased for Gaussian targets. We also prove that in the zero temperature limit and for sufficiently well-behaved strongly convex objectives the solution of the Fokker-Planck equation converges in the Wasserstein-2 distance to a Dirac measure at the minimizer. Finally, we propose a computationally more efficient generalization which works with a predefined number of clusters and improves upon our polarized baseline method for high-dimensional optimization.

关键词： Global optimization consensus-based optimization Polarization Sampling

来源：评论

学校读者我要写书评

暂无评论

consensus-based optimization METHODS CONVERGE GLOBALLY\ast

引用

SIAM JOURNAL ON optimization 2024年第3期34卷 2973-3004页

作者： Fornasier, Massimo Klock, Timo Riedl, Konstantin Tech Univ Munich Sch Computat Informat & Technol Dept Math Munich Germany Munich Ctr Machine Learning Munich Germany Simula Res Lab Dept Numer Anal & Sci Comp Oslo Norway

In this paper we study consensus-based optimization (CBO), which is a multiagent metaheuristic derivative-free optimization method that can globally minimize nonconvex nonsmooth functions and is amenable to theoretical analysis. based on an experimentally supported intuition that, on average, CBO performs a gradient descent of the squared Euclidean distance to the global minimizer, we devise a novel technique for proving the convergence to the global minimizer in mean- field law for a rich class of objective functions. The result unveils internal mechanisms of CBO that are responsible for the success of the method. In particular, we prove that CBO performs a convexification of a large class of optimization problems as the number of optimizing agents goes to infinity. Furthermore, we improve prior analyses by requiring mild assumptions about the initialization of the method and by covering objectives that are merely locally Lipschitz continuous. As a core component of this analysis, we establish a quantitative nonasymptotic Laplace principle, which may be of independent interest. From the result of CBO convergence in mean-field law, it becomes apparent that the hardness of any global optimization problem is necessarily encoded in the rate of the mean-field approximation, for which we provide a novel probabilistic quantitative estimate. The combination of these results allows us to obtain probabilistic global convergence guarantees of the numerical CBO method.

关键词： global optimization derivative-free optimization nonsmoothness nonconvexity metaheuristics consensus-based optimization mean-field limit Fokker--Plank equations

来源：评论

学校读者我要写书评

暂无评论

consensus-based optimization on hypersurfaces: Well-posedness and mean-field limit

引用

MATHEMATICAL MODELS & METHODS IN APPLIED SCIENCES 2020年第14期30卷 2725-2751页

作者： Fornasier, Massimo Huang, Hui Pareschi, Lorenzo Sunnen, Philippe Tech Univ Munich Dept Math Boltzmannstrae 3 D-85748 Munich Germany Univ Ferrara Dept Math & Comp Sci Via Machiavelli 30 I-44121 Ferrara Italy

We introduce a new stochastic differential model for global optimization of nonconvex functions on compact hypersurfaces. The model is inspired by the stochastic Kuramoto-Vicsek system and belongs to the class of consensus-based optimization methods. In fact, particles move on the hypersurface driven by a drift towards an instantaneous consensus point, computed as a convex combination of the particle locations weighted by the cost function according to Laplace's principle. The consensus point represents an approximation to a global minimizer. The dynamics is further perturbed by a random vector field to favor exploration, whose variance is a function of the distance of the particles to the consensus point. In particular, as soon as the consensus is reached, then the stochastic component vanishes. In this paper, we study the well-posedness of the model and we derive rigorously its mean-field approximation for large particle limit.

关键词： consensus-based optimization optimization over manifolds stochastic Kuramoto-Vicsek model well-posedness mean-field limit

来源：评论

学校读者我要写书评

暂无评论

consensus-based optimization on the Sphere: Convergence to Global Minimizers and Machine Learning

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2021年第1期22卷 1-55页

作者： Fornasier, Massimo Huang, Hui Pareschi, Lorenzo Suennen, Philippe Tech Univ Munich Dept Math Boltzmannstr 3 D-85748 Munich Germany Univ Ferrara Dept Math & Comp Sci Via Machiavelli 30 I-44121 Ferrara Italy

We investigate the implementation of a new stochastic Kuramoto-Vicsek-type model for global optimization of nonconvex functions on the sphere. This model belongs to the class of consensus-based optimization. In fact, particles move on the sphere driven by a drift towards an instantaneous consensus point, which is computed as a convex combination of particle locations, weighted by the cost function according to Laplace's principle, and it represents an approximation to a global minimizer. The dynamics is further perturbed by a random vector field to favor exploration, whose variance is a function of the distance of the particles to the consensus point. In particular, as soon as the consensus is reached the stochastic component vanishes. The main results of this paper are about the proof of convergence of the numerical scheme to global minimizers provided conditions of well-preparation of the initial datum. The proof combines previous results of mean-field limit with a novel asymptotic analysis, and classical convergence results of numerical methods for SDE. We present several numerical experiments, which show that the algorithm proposed in the present paper scales well with the dimension and is extremely versatile. To quantify the performances of the new approach, we show that the algorithm is able to perform essentially as good as ad hoc state of the art methods in challenging problems in signal processing and machine learning, namely the phase retrieval problem and the robust subspace detection.

关键词： global optimization consensus-based optimization asymptotic convergence analysis stochastic Kuramoto-Vicsek model mean-field limit numerical methods for SDE

来源：评论

学校读者我要写书评

暂无评论

consensus-based optimization FOR SADDLE POINT PROBLEMS

引用

SIAM JOURNAL ON CONTROL AND optimization 2024年第2期62卷 1093-1121页

作者： Huang, Hui Qiu, Jinniao Riedl, Konstantin Karl Franzens Univ Graz Dept Math & Sci Comp A-8010 Graz Austria Univ Calgary Dept Math & Stat Calgary AB T2N 1N4 Canada Tech Univ Munich Sch Computat Informat & Technol Dept Math D-80333 Munich Germany Munich Ctr Machine Learning Munich Germany

In this paper, we propose consensus -based optimization for saddle point problems (CBO-SP), a novel multi -particle metaheuristic derivative -free optimization method capable of provably finding global Nash equilibria. Following the idea of swarm intelligence, the method employs two groups of interacting particles, one which performs a minimization over one variable while the other performs a maximization over the other variable. The two groups constantly exchange information through a suitably weighted average. This paradigm permits a passage to the mean -field limit, which makes the method amenable to theoretical analysis, and it allows to obtain rigorous convergence guarantees under reasonable assumptions about the initialization and the objective function, which most notably include nonconvex-nonconcave objectives. We further provide numerical evidence for the success of the algorithm.

关键词： saddle point problems Nash equilibria nonconvex-nonconcave derivative-free optimization metaheuristics consensus-based optimization Fokker-Planck equations

来源：评论

学校读者我要写书评

暂无评论

Time-discrete momentum consensus-based optimization algorithm and its application to Lyapunov function approximation

引用

MATHEMATICAL MODELS & METHODS IN APPLIED SCIENCES 2024年第6期34卷 1153-1204页

作者： Ha, Seung-Yeal Hwang, Gyuyoung Kim, Sungyoon Seoul Natl Univ Res Inst Math Dept Math Sci Seoul 08826 South Korea Seoul Natl Univ Dept Math Sci Seoul 08826 South Korea Stanford Univ Dept Elect Engn Stanford CA 94305 USA

In this paper, we study a discrete momentum consensus-based optimization (Momentum-CBO) algorithm which corresponds to a second-order generalization of the discrete first-order CBO [S.-Y. Ha, S. Jin and D. Kim, Convergence of a first-order consensus-based global optimization algorithm, Math. Models Methods Appl. Sci. 30 (2020) 2417-2444]. The proposed algorithm can be understood as the modification of ADAM-CBO, replacing the normalization term by unity. For the proposed Momentum-CBO, we provide a sufficient framework which guarantees the convergence of algorithm toward a global minimum of the objective function. Moreover, we present several experimental results showing that Momentum-CBO has an improved success rate of finding the global minimum compared to vanilla-CBO and show the stability of Momentum-CBO under different initialization schemes. We also show that Momentum-CBO can be used as the alternative of ADAM-CBO which does not have a proper convergence analysis. Finally, we give an application of Momentum-CBO for Lyapunov function approximation using symbolic regression techniques.

关键词： Collective dynamics consensus-based optimization machine learning symbolic regression Lyapunov function

来源：评论

学校读者我要写书评

暂无评论

On the mean-field limit for the consensus-based optimization

引用

MATHEMATICAL METHODS IN THE APPLIED SCIENCES 2022年第12期45卷 7814-7831页

作者： Huang, Hui Qiu, Jinniao Univ Calgary Dept Math & Stat 2500 Univ Dr NW Calgary AB T2N 1N4 Canada

This paper is concerned with the large particle limit for the consensus-based optimization (CBO), which was postulated in the pioneering works by Carrillo, Pinnau, Totzeck and many others. In order to solve this open problem, we adapt a compactness argument by first proving the tightness of the empirical measures {mu N}N >= 2$$ {\left\{{\mu} circumflex N\right\}}_{N\ge 2} $$ associated to the particle system and then verifying that the time marginal of the limit measure mu$$ \mu $$ is the unique weak solution to the mean-field CBO equation. Such results are further extended to the model of particle swarm optimization (PSO).

关键词： consensus-based optimization particle swarm optimization propagation of chaos tightness weak convergence

来源：评论

学校读者我要写书评

暂无评论

ANISOTROPIC DIFFUSION IN consensus-based optimization ON THE SPHERE

引用

SIAM JOURNAL ON optimization 2022年第3期32卷 1958-1986页

作者： Fornasier, Massimo Huang, Hui Pareschi, Lorenzo Sunnen, Philippe Tech Univ Munich Dept Math D-85748 Garching Germany Tech Univ Munich Munich Data Sci Inst D-85748 Garching Germany Univ Calgary Dept Math & Stat Calgary AB T2N 1N4 Canada Univ Ferrara Dept Math I-44121 Ferrara Italy Univ Ferrara Dept Comp Sci I-44121 Ferrara Italy

In this paper, we are concerned with the global minimization of a possibly non-smooth and nonconvex objective function constrained on the unit hypersphere by means of a multi-agent derivative-free method. The proposed algorithm falls into the class of the recently introduced consensus-based optimization. In fact, agents move on the sphere driven by a drift towards an instantaneous consensus point, which is computed as a convex combination of agent locations, weighted by the cost function according to Laplace's principle, and it represents an approximation to a global minimizer. The dynamics is further perturbed by an anisotropic random vector field to favor exploration. The main results of this paper are about the proof of convergence of the numerical scheme to global minimizers provided conditions of well-preparation of the initial datum. The proof of convergence combines a mean-field limit result with a novel asymptotic analysis and classical convergence results of numerical methods for stochastic differential equations. The main innovation with respect to previous work is the introduction of an anisotropic stochastic term, which allows us to ensure the independence of the parameters of the algorithm from the dimension and to scale the method to work in very high dimension. We present several numerical experiments, which show that the algorithm proposed in the present paper is extremely versatile and outperforms previous formulations with isotropic stochastic noise.

关键词： high-dimensional optimization derivative-free optimization geometric optimization consensus-based optimization anisotropic stochastic Kuramoto-Vicsek model Fokker-Planck equations signal processing machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：