检索结果-内蒙古大学图书馆

Stochastic analysis of an adaptive cubic regularization method under inexact gradient evaluations and dynamic Hessian accuracy

引用

OPTIMIZATION 2022年第1期71卷 227-261页

作者： Bellavia, Stefania Gurioli, Gianmarco Univ Firenze Dipartimento Ingn Ind Florence Italy Univ Firenze Dipartimento Matemat & Informat Ulisse Dini Florence Italy

We here adapt an extended version of the adaptive cubic regularization method with dynamic inexact Hessian information for nonconvex optimization in Bellavia et al. [adaptive cubic regularization methods with dynamic inexact hessian information and applications to finite-sum minimization. IMA Journal of Numerical Analysis. 2021;41(1):764-799] to the stochastic optimization setting. While exact function evaluations are still considered, this novel variant inherits the innovative use of adaptive accuracy requirements for Hessian approximations introduced in the just quoted paper and additionally employs inexact computations of the gradient. Without restrictions on the variance of the errors, we assume that these approximations are available within a sufficiently large, but fixed, probability and we extend, in the spirit of Cartis and Scheinberg [Global convergence rate analysis of unconstrained optimization methods based on probabilistic models. Math Program Ser A. 2018;159(2):337-375], the deterministic analysis of the framework to its stochastic counterpart, showing that the expected number of iterations to reach a first-order stationary point matches the well-known worst-case optimal complexity. This is, in fact, still given by O(epsilon(-3/2)), with respect to the first-order epsilon tolerance. Finally, numerical tests on nonconvex finite-sum minimization confirm that using inexact first- and second-order derivatives can be beneficial in terms of the computational savings.

关键词： adaptive cubic regularization methods inexact derivatives evaluations stochastic nonconvex optimization worst-case complexity analysis finite-sum minimization

来源：评论

学校读者我要写书评

暂无评论

A unified efficient implementation of trust-region type algorithms for unconstrained optimization

引用

INFOR 2020年第2期58卷 290-309页

作者： Dussault, Jean-Pierre Univ Sherbrooke Dept Informat Sherbrooke PQ J1K 2R1 Canada

adaptive cubic regularization (ARC) and trust-region (TR) methods use modified linear systems to compute their steps. The modified systems consist in adding some multiple of the identity matrix (or a well-chosen positive definite matrix) to the Hessian to obtain a sufficiently positive definite linear system, the so called shifted system. This type of system was first proposed by Levenberg and Marquardt. Some trial and error is often involved to obtain a specified value for this shift parameter. We provide an efficient unified implementation to track the shift parameter;our implementation encompasses many ARC and TR variants.

关键词： Nonlinear optimization unconstrained optimization trust-region algorithms adaptive cubic regularization methods Julia programming language

来源：评论

学校读者我要写书评

暂无评论

ARC_q: a new adaptive regularization by cubics

引用

OPTIMIZATION methods & SOFTWARE 2018年第2期33卷 322-335页

作者： Dussault, Jean-Pierre Univ Sherbrooke Dept Informat Sherbrooke PQ J1K 2R1 Canada

We present a new adaptive regularization by cubics (ARC) algorithm, which we name ARC(q) and which is very close in spirit to trust region methods, closer than the original ARC is. We prove global convergence to second-order critical points. We also obtain as a corollary the convergence of the original ARC method. We prove the optimal complexity property for the ARC(q) and identify the key elements which allow it. We end by proposing an efficient implementation using a Cholesky like factorization. Limited preliminary experimentation suggests that ARC(q) may be more robust than its trust region counterpart and that our implementation is efficient.

关键词： Nonlinear optimization unconstrained optimization Trust region algorithms adaptive cubic regularization methods

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：