检索结果-内蒙古大学图书馆

A neurodynamic optimization approach to distributed nonconvex optimization based on an HP augmented Lagrangian function

引用

NEURAL NETWORKS 2025年 181卷 106791页

作者： Guan, Huimin Liu, Yang Kou, Kit Ian Gui, Weihua Zhejiang Normal Univ Sch Math Sci Jinhua 321004 Peoples R China Zhejiang Normal Univ Key Lab Intelligent Educ Technol & Applicat Zhejia Jinhua 321004 Zhejiang Peoples R China Linyi Univ Intelligent Comp Univ Shandong Sch Automat & Elect Engn Linyi 276005 Peoples R China Univ Macau Fac Sci & Technol Dept Math Macau Peoples R China Cent South Univ Sch Automat Changsha 410083 Peoples R China

This paper develops a neurodynamic model for distributed nonconvex-constrained optimization. In the distributed constrained optimization model, the objective function and inequality constraints do not need to be convex, and equality constraints do not need to be affine. A Hestenes-Powell augmented Lagrangian function for handling the nonconvexity is established, and a neurodynamic system is developed based on this. It is proved that it is stable at a local optimal solution of the optimization model. Two illustrative examples are provided to evaluate the enhanced stability and optimality of the developed neurodynamic systems.

关键词： Distributed optimization nonconvex optimization Neurodynamic optimization Neurodynamic systems Hestenes-Powell augmented Lagrangian function

来源：评论

学校读者我要写书评

暂无评论

Low-Rank Matrix Recovery Via nonconvex optimization Methods with Application to Errors-in-Variables Matrix Regression

引用

JOURNAL OF optimization THEORY AND APPLICATIONS 2025年第3期205卷 1-27页

作者： Li, Xin Wu, Dongya Northwest Univ Sch Math Xian Peoples R China Northwest Univ Sch Informat Sci & Technol Xian Peoples R China

We consider the nonconvex regularized method for low-rank matrix recovery problems. Under suitable regularity conditions on the nonconvex loss function and the regularizer, we provide the recovery bound for any stationary point of the nonconvex method via separating singular values of the parameter matrix into larger and smaller ones. In this way, the established recovery bound can be much tighter than that of the convex nuclear norm regularized method when some of the singular values are larger than a threshold defined by the nonconvex regularizer. In addition, we consider the errors-in-variables matrix regression as an application of the nonconvex method. Probabilistic consequences and the advantage of the nonconvex method are demonstrated through verifying the regularity conditions for specific models with additive noise and missing data.

关键词： Low-rank matrix recovery nonconvex optimization nonconvex regularization Recovery bounds Errors-in-variables matrix regression

来源：评论

学校读者我要写书评

暂无评论

Compressed gradient tracking algorithms for distributed nonconvex optimization

引用

AUTOMATICA 2025年 177卷

作者： Xu, Lei Yi, Xinlei Wen, Guanghui Shi, Yang Johansson, Karl H. Yang, Tao Northeastern Univ State Key Lab Synthet Automat Proc Ind Shenyang 110819 Peoples R China Univ Victoria Dept Mech Engn Victoria BC V8W 2Y2 Canada Tongji Univ Coll Elect & Informat Engn Shanghai 201804 Peoples R China MIT Lab Informat & Decis Syst Cambridge MA 02139 USA Southeast Univ Sch Math Dept Syst Sci Nanjing 210096 Peoples R China KTH Royal Inst Technol Sch Elect Engn & Comp Sci S-10044 Stockholm Sweden

In this paper, we study the distributed nonconvex optimization problem, aiming to minimize the average value of the local nonconvex cost functions using local information exchange. To reduce the communication overhead, we introduce three general classes of compressors, i.e., compressors with bounded relative compression error, compressors with globally bounded absolute compression error, and compressors with locally bounded absolute compression error. By integrating them, respectively, with the distributed gradient tracking algorithm, we then propose three corresponding compressed distributed nonconvex optimization algorithms. Motivated by the state-of-the-art BEER algorithm proposed in Zhao et al. (2022), which is an efficient compressed algorithm integrating gradient tracking with biased and contractive compressors, our first proposed algorithm extends this algorithm to accommodate both biased and non-contractive compressors For each algorithm, we design a novel Lyapunov function to demonstrate its sublinear convergence to a stationary point if the local cost functions are smooth. Furthermore, when the global cost function satisfies the Polyak-& Lstrok;ojasiewicz (P-& Lstrok;) condition, we show that our proposed algorithms linearly converge to a global optimal point. It is worth noting that, for compressors with bounded relative compression error and globally bounded absolute compression error, our proposed algorithms' parameters do not require prior knowledge of the P-& Lstrok;constant. (c) 2025 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.

关键词： Communication compression Gradient tracking algorithm Linear convergence nonconvex optimization Polyak-& Lstrok ojasiewicz condition Sublinear convergence

来源：评论

学校读者我要写书评

暂无评论

The Boosted Double-proximal Subgradient Algorithm for nonconvex optimization

引用

MATHEMATICAL PROGRAMMING 2025年 1-47页

作者： Aragon-Artacho, Francisco J. Perez-Aros, Pedro Torregrosa-Belen, David Univ Alicante Dept Math Alicante Spain Univ Chile Ctr Modelamiento Matemat Dept Ingn Matemat CNRS IRL2807 Beauchef 851 Santiago Chile Univ Chile Ctr Modelamiento Matemat CNRS IRL2807 Beauchef 851 Santiago Chile

In this paper we introduce the Boosted Double-proximal Subgradient Algorithm (BDSA), a novel splitting algorithm designed to address general structured nonsmooth and nonconvex mathematical programs expressed as sums and differences of composite functions. BDSA exploits the combined nature of subgradients from the data and proximal steps, and integrates a linesearch procedure to enhance its performance. While BDSA encompasses existing schemes proposed in the literature, it extends its applicability to more diverse problem domains. We establish the convergence of BDSA under the Kurdyka-& Lstrok;ojasiewicz property and provide an analysis of its convergence rate. To evaluate the effectiveness of BDSA, we introduce two novel test functions with an abundance of critical points. We conduct comparative evaluations, including algorithms with inertial terms, that illustrate its ability to effectively escape non-optimal critical points. Additionally, we present two practical applications of BDSA for testing its efficacy, namely, a constrained minimum-sum-of-squares clustering problem and a nonconvex generalization of Heron's problem.

关键词： nonconvex optimization Difference programming Kurdyka-& Lstrok ojasiewicz property Descent lemma Minimum-sum of squares clustering Heron's problem

来源：评论

学校读者我要写书评

暂无评论

Regularized methods via cubic model subspace minimization for nonconvex optimization

引用

COMPUTATIONAL optimization AND APPLICATIONS 2025年第3期90卷 801-837页

作者： Bellavia, Stefania Palitta, Davide Porcelli, Margherita Simoncini, Valeria Univ Firenze Dipartimento Ingn Ind Viale Morgagni 40-44 I-50134 Florence Italy Alma Mater Studiorum Univ Bologna Dipartimento Matemat AM2 Piazza Porta San Donato 5 I-40126 Bologna Italy CNR ISTI Via Moruzzi 1 Pisa Italy CNR IMATI Via Ferrata 5-A Pavia Italy

Adaptive cubic regularization methods for solving nonconvex problems need the efficient computation of the trial step, involving the minimization of a cubic model. We propose a new approach in which this model is minimized in a low dimensional subspace that, in contrast to classic approaches, is reused for a number of iterations. Whenever the trial step produced by the low-dimensional minimization process is unsatisfactory, we employ a regularized Newton step whose regularization parameter is a by-product of the model minimization over the low-dimensional subspace. We show that the worst-case complexity of classic cubic regularized methods is preserved, despite the possible regularized Newton steps. We focus on the large class of problems for which (sparse) direct linear system solvers are available and provide several experimental results showing the very large gains of our new approach when compared to standard implementations of adaptive cubic regularization methods based on direct linear solvers. Our first choice as projection space for the low-dimensional model minimization is the polynomial Krylov subspace;nonetheless, we also explore the use of rational Krylov subspaces in case where the polynomial ones lead to less competitive numerical results.

关键词： Adaptive regularization methods nonconvex optimization Secant methods Krylov subspaces Worst-case iteration complexity

来源：评论

学校读者我要写书评

暂无评论

Distributed nonconvex optimization for Control of Water Networks with Time-coupling Constraints

引用

WATER RESOURCES MANAGEMENT 2025年第1期39卷 523-546页

作者： Jenks, Bradley Ulusoy, Aly-Joy Pecci, Filippo Stoianov, Ivan Imperial Coll London Dept Civil & Environm Engn London SW7 2BU England Princeton Univ Andlinger Ctr Energy & Environm Princeton NJ 08544 USA

In this paper, we present a new control model for optimizing pressure and water quality operations in water distribution networks. Our formulation imposes a set of time-coupling constraints to manage temporal pressure variations, which are exacerbated by the transition between pressure and water quality controls. The resulting optimization problem is a nonconvex, nonlinear program with nonseparable structure across time steps. This problem proves challenging for state-of-the-art nonlinear solvers, often precluding their direct use for near real-time control in large-scale networks. To overcome this computational burden, we investigate a distributed optimization approach based on the alternating direction method of multipliers (ADMM). In particular, we implement and evaluate two algorithms: a standard ADMM scheme and a two-level variant that provides theoretical convergence guarantees for our nonconvex problem. We use a benchmarking water network and a large-scale operational network in the UK for our numerical experiments. The results demonstrate good convergence behavior across all problem instances for the two-level algorithm, whereas the standard ADMM approach struggles to converge in some instances. With an appropriately tuned penalty parameter, however, both distributed algorithms yield good quality solutions and computational times compatible with near real-time (e.g. hourly) control requirements for large-scale water networks.

关键词： Distributed optimization nonconvex optimization Alternating direction method of multipliers Water distribution networks

来源：评论

学校读者我要写书评

暂无评论

Yet another fast variant of Newton's method for nonconvex optimization

引用

IMA JOURNAL OF NUMERICAL ANALYSIS 2025年第2期45卷 971-1008页

作者： Gratton, Serge Jerad, Sadok Toint, Philippe L. Univ Toulouse IRIT INP F-31000 Toulouse France Univ Toulouse IRIT ANITI INP F-31000 Toulouse France Univ Namur NAXYS B-5000 Namur Belgium

A class of second-order algorithms is proposed for minimizing smooth nonconvex functions that alternates between regularized Newton and negative curvature steps in an iteration-dependent subspace. In most cases, the Hessian matrix is regularized with the square root of the current gradient and an additional term taking moderate negative curvature into account, a negative curvature step being taken only exceptionally. Practical variants are detailed where the subspaces are chosen to be the full space, or Krylov subspaces. In the first case, the proposed method only requires the solution of a single linear system at nearly all iterations. We establish that at most $\mathcal{O}\big ( |\!\log \epsilon |\,\epsilon <^>{-3/2}\big )$ evaluations of the problem's objective function and derivatives are needed for algorithms in the new class to obtain an $\epsilon $-approximate first-order minimizer, and at most $\mathcal{O}\big (|\!\log \epsilon |\,\epsilon <^>{-3}\big )$ to obtain a second-order one. Encouraging initial numerical experiments with two full-space and two Krylov-subspaces variants are finally presented.

关键词： Newton's method nonconvex optimization negative curvature adaptive regularization methods evaluation complexity

来源：评论

学校读者我要写书评

暂无评论

BREGMAN ADMM: A NEW ALGORITHM FOR nonconvex optimization WITH LINEAR CONSTRAINTS

JOURNAL OF NONLINEAR AND VARIATIONAL ANALYSIS

引用

JOURNAL OF NONLINEAR AND VARIATIONAL ANALYSIS 2025年第2期9卷 179-196页

作者： Tan, Li Guo, Ke China West Normal Univ China West Normal Univ Sichuan Prov Sch Math & Informat Key Lab Optimizat Theory & Applicat Nanchong 637009 Peoples R China

Alternating direction method of multipliers (ADMM) is a widely used algorithm for solving two-block separable problems with linear constraints. However, its applicability in various fields is limited by the need to assume the global Lipschitz continuity of the gradient of differentiable functions, which is often infeasible in nonconvex optimization problems. To address this limitation, we propose a new version of the Bregman ADMM that can return to the ADMM while avoiding the need for global Lipschitz continuity of the gradient. The Bregman ADMM relaxes the classical ADMM's requirement for global Lipschitz continuous gradient, enriching its applications. We prove that when the associated function satisfies the Kurdyka-Lojasiewicz inequality and certain assumptions, the iterative sequence generated by our algorithm converges to a critical point of the problem. Additionally, we analyze the rate of convergence of the algorithm.

关键词： Alternating direction method of multipliers Bregman ADMM Kurdyka-Lojasiewicz in-equality Lipschitz-like convexity condition nonconvex optimization

来源：评论

学校读者我要写书评

暂无评论

Learning-Rate-Free Momentum SGD with Reshuffling Converges in Nonsmooth nonconvex optimization

引用

JOURNAL OF SCIENTIFIC COMPUTING 2025年第3期102卷 1-31页

作者： Hu, Xiaoyin Xiao, Nachuan Liu, Xin Toh, Kim-Chuan Hangzhou City Univ Sch Comp & Comp Sci Hangzhou 310015 Peoples R China Hangzhou City Univ Acad Edge Intelligence Hangzhou Peoples R China Natl Univ Singapore Inst Operat Res & Analyt Singapore Singapore Chinese Acad Sci Acad Math & Syst Sci State Key Lab Sci & Engn Comp Beijing Peoples R China Univ Chinese Acad Sci Beijing Peoples R China Natl Univ Singapore Dept Math Singapore 119076 Singapore Natl Univ Singapore Inst Operat Res & Analyt Singapore 119076 Singapore

In this paper, we propose a generalized framework for developing learning-rate-free momentum stochastic gradient descent (SGD) methods in the minimization of nonsmooth nonconvex functions, especially in training nonsmooth neural networks. Our framework adaptively generates learning rates based on the historical data of stochastic subgradients and iterates. Under mild conditions, we prove that our proposed framework enjoys global convergence to the stationary points of the objective function in the sense of the conservative field, hence providing convergence guarantees for training nonsmooth neural networks. Based on our proposed framework, we propose a novel learning-rate-free momentum SGD method (LFM). Preliminary numerical experiments reveal that LFM performs comparably to the state-of-the-art learning-rate-free methods (which have not been shown theoretically to be convergent) across well-known neural network training benchmarks.

关键词： Nonsmooth optimization Stochastic subgradient methods nonconvex optimization Learning-rate free Differential inclusion

来源：评论

学校读者我要写书评

暂无评论

Variance-reduced reshuffling gradient descent for nonconvex optimization: Centralized and distributed algorithms

引用

AUTOMATICA 2025年 171卷

作者： Jiang, Xia Zeng, Xianlin Xie, Lihua Sun, Jian Chen, Jie Chinese Univ Hong Kong Dept Syst Engn & Engn Management Hong Kong 999077 Peoples R China Beijing Inst Technol Natl Key Lab Autonomous Intelligent Unmanned Syst Beijing 100081 Peoples R China Nanyang Technol Univ Sch Elect & Elect Engn Singapore 639798 Singapore Beijing Inst Technol Chongqing Innovat Ctr Chongqing 401120 Peoples R China Tongji Univ Sch Elect & Informat Engn Shanghai 200082 Peoples R China

nonconvex finite-sum optimization plays a crucial role in signal processing and machine learning, fueling the development of numerous centralized and distributed stochastic algorithms. However, existing stochastic optimization algorithms often suffer from high stochastic gradient variance due to the use of random sampling with replacement. To address this issue, this paper introduces an explicit variance-reduction step and proposes variance-reduced reshuffling gradient algorithms with a sampling-without-replacement scheme. Specifically, this paper proves that the proposed centralized variance-reduced reshuffling gradient algorithm (VR-RG) with constant step sizes converges to a stationary point for nonconvex optimization under the Kurdyka-& Lstrok;ojasiewicz condition. Moreover, for nonconvex optimization over connected multi-agent networks, the proposed distributed variance- reduced reshuffling gradient algorithm (DVR-RG) converges to a neighborhood of stationary points, where the neighborhood can be made arbitrarily small under mild conditions. Notably, the proposed DVR-RG requires only one communication round at each epoch. Finally, numerical simulations demonstrate the efficiency of the proposed algorithms. (c) 2024 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.

关键词： Reshuffling gradient descent Variance reduction nonconvex optimization Sampling without replacement Multi-agent networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：