检索结果-内蒙古大学图书馆

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

COMPUTATIONAL optimization AND APPLICATIONS 2024年第2期88卷 603-641页

作者： Liu, Ruyu Pan, Shaohua Wu, Yuqia Yang, Xiaoqi South China Univ Technol Sch Math Guangzhou Peoples R China Hong Kong Polytech Univ Dept Appl Math Hong Kong Peoples R China

This paper focuses on the minimization of a sum of a twice continuously differentiable function f and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by an approximation to the Hessian off involving the Qth power of the KKT residual. For Q=0, we justify the global convergence of the iterate sequence for the KL objective function and its R-linear convergence rate for the KL objective function of exponent 1/2. For Q is an element of(0,1), by assuming that cluster points satisfy a locally Holderian error bound of order q on a second-order stationary point set and a local error bound of order q>1+Q on the common stationary pointset, respectively, we establish the global convergence of the iterate sequence and its superlinear convergence rate with order depending on q and Q. A dual semismooth Newton augmented Lagrangian method is also developed for seeking an inexact minimizer of subproblems. Numerical comparisons with two state-of-the-art methods on l(1)-regularized Student's t-regressions, group penalized Student's t-regressions, and nonconvex image restoration confirm the efficiency of the proposed method

关键词： nonconvex and nonsmooth optimization Regularized proximal Newton method Global convergence Convergence rate KL function Metric q-subregularity

来源：评论

学校读者我要写书评

暂无评论

A Hybrid and Inexact Algorithm for nonconvex and nonsmooth optimization

引用

JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY 2024年 1-21页

作者： Wang, Yiyang Song, Xiaoliang Dalian Maritime Univ Coll Artificial Intelligence Dalian 116026 Peoples R China Dalian Univ Technol Sch Math Sci Dalian 116026 Peoples R China

The problem of nonconvex and nonsmooth optimization (NNO) has been extensively studied in the machine learning community, leading to the development of numerous fast and convergent numerical algorithms. Existing algorithms typically employ unified iteration schemes and require explicit solutions to subproblems for ensuring convergence. However, these inflexible iteration schemes overlook task-specific details and may encounter difficulties in providing explicit solutions to subproblems. In contrast, there is evidence suggesting that practical applications can benefit from approximately solving subproblems;however, many existing works fail to establish the theoretical validity of such approximations. In this paper, the authors propose a hybrid inexact proximal alternating method (hiPAM), which addresses a general NNO problem with coupled terms while overcoming all aforementioned challenges. The proposed hiPAM algorithm offers a flexible yet highly efficient approach by seamlessly integrating any efficient methods for approximate subproblem solving that cater to specificities. Additionally, the authors have devised a simple yet implementable stopping criterion that generates a Cauchy sequence and ultimately converges to a critical point of the original NNO problem. The proposed numerical experiments using both simulated and real data have demonstrated that hiPAM represents an exceedingly efficient and robust approach to NNO problems.

关键词： Hybrid inexact proximal alternating method inexact minimization criteria machine learning nonconvex and nonsmooth optimization

来源：评论

学校读者我要写书评

暂无评论

An ADMM Approach of a nonconvex and nonsmooth optimization Model for Low-Light or Inhomogeneous Image Segmentation

引用

ASIA-PACIFIC JOURNAL OF OPERATIONAL RESEARCH 2024年第3期41卷 2350021-2350021页

作者： Xing, Zheyuan Wu, Tingting Yue, Junhong Taiyuan Univ Technol Coll Math Taiyuan 030024 Peoples R China Nanjing Univ Posts & Telecommun Sch Sci Nanjing 210023 Peoples R China Taiyuan Univ Technol Coll Date Sci Taiyuan 030024 Peoples R China

In this paper, we propose a novel nonconvex and nonsmooth optimization model for low-light or inhomogeneous image segmentation which is a hybrid of Mumford-Shah energy functional and Retinex theory. The given image is decomposed into the reflectance component and the illumination component by solving Retinex-based Mumford-Shah model with L-1 - ??L-2 regularizer. Indeed, the existence of the L-1 - ??L-2 regularizer means the nonsmooth term in the model is nonconvex. Thus, it is difficult to solve the proposed model directly. An alternating direction method of multipliers (ADMM) algorithm is developed to solve the proposed nonconvex and nonsmooth model. We apply a novel splitting technique in our algorithm to ensure that all subproblems admit closed-form solutions. Theoretically, we prove that the sequence generated by our proposed algorithm converges to a stationary point under mild conditions. Next, once the reflectance is obtained, the K-means clustering method is utilized to complete the segmentation. We compare the proposed Retinex-based method with other state-of-the-art segmentation methods under special lighting conditions. Experimental results show that the proposed method has better performance for both gray-scale images and color images efficiently in terms of the quantitative and qualitative results.

关键词： nonconvex and nonsmooth optimization alternating direction method of multipliers (ADMM) image segmentation intensity inhomogeneity Retinex

来源：评论

学校读者我要写书评

暂无评论

Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization

引用

JOURNAL OF GLOBAL optimization 2022年第4期84卷 913-939页

作者： Yashtini, Maryam Georgetown Univ Dept Math & Stat 327A St Marys Hall 37th & O St NW Washington DC 20057 USA

In this paper, we consider a proximal linearized alternating direction method of multipliers, or PL-ADMM, for solving linearly constrained nonconvex and possibly nonsmooth optimization problems. The algorithm is generalized by using variable metric proximal terms in the primal updates and an over-relaxation step in the multiplier update. Extended results based on the augmented Lagrangian including subgradient band, limiting continuity, descent and monotonicity properties are established. We prove that the PL-ADMM sequence is bounded. Under the powerful Kurdyka-Lojasiewicz inequality we show that the PL-ADMM sequence has a finite length thus converges, and we drive its convergence rates.

关键词： nonconvex and nonsmooth optimization Kurdyka-Lojasiewicz inequality Alternating direction method of multiplier (ADMM) Convergence convergence rate Variable metric proximal terms

来源：评论

学校读者我要写书评

暂无评论

Two inertial proximal coordinate algorithms for a family of nonsmooth and nonconvex optimization problems

引用

AUTOMATICA 2025年 171卷

作者： Dang, Ya Zheng Sun, Jie Teo, Kok Lay Univ Shanghai Sci & Technol Sch Management Shanghai Peoples R China Natl Univ Singapore Sch Business Singapore Singapore Curtin Univ Sch EECMS Perth Australia Sunway Univ Sch Math Sci Selangor Malaysia

The inertial proximal method is extended to minimize the sum of a series of separable nonconvex and possibly nonsmooth objective functions and a smooth nonseparable function (possibly nonconvex). Here, we propose two new algorithms. The first one is an inertial proximal coordinate subgradient algorithm, which updates the variables by employing the proximal subgradients of each separable function at the current point. The second one is an inertial proximal block coordinate method, which updates the variables by using the subgradients of the separable functions at the partially updated points. Global convergence is guaranteed under the Kurdyka-& Lstrok;ojasiewicz (K & Lstrok;) property and some additional mild assumptions. Convergence rate is derived based on the & Lstrok;ojasiewicz exponent. Two numerical examples are given to illustrate the effectiveness of the algorithms. (c) 2024 Published by Elsevier Ltd.

关键词： nonconvex and nonsmooth optimization Inertial proximal algorithm Subgradient Global convergence Convergence rate

来源：评论

学校读者我要写书评

暂无评论

Stochastic subgradient algorithm for nonsmooth nonconvex optimization

引用

JOURNAL OF APPLIED MATHEMATICS AND COMPUTING 2024年第1期70卷 317-334页

作者： Yalcin, Gulcin Dinc Eskisehir Tech Univ Dept Ind Engn TR-26555 Eskisehir Turkiye

In this paper, we study on a stochastic subgradient algorithm for the finite-sum optimization problems where the functions are not necessarily convex and smooth and we use a weak subgradient of only one function at each iteration. We then analyze the convergence properties of the stochastic weak subgradient algorithm (SWSA). In addition, we focus on the problem that arises in the semi-supervised machine learning (SSML) problem where all the functions in the problem are not smooth and convex, and we propose an algorithm called WS-SSML to compute a weak subgradient of these functions in SSML. Finally, we solve the SSML problem using the data sets from the literature and we compare our results. We can conclude that SWSA with WS-SSML works well in practice.

关键词： Stochastic algorithms Finite-sum optimization nonconvex and nonsmooth optimization Semi-supervised machine learning

来源：评论

学校读者我要写书评

暂无评论

An accelerated stochastic ADMM for nonconvex and nonsmooth finite-sum optimization

引用

AUTOMATICA 2024年 163卷

作者： Zeng, Yuxuan Wang, Zhiguo Bai, Jianchao Shen, Xiaojing Sichuan Univ Coll Math Chengdu 610064 Peoples R China Northwestern Polytech Univ Shenzhen Res & Dev Inst Shenzhen 518057 Peoples R China Northwestern Polytech Univ Sch Math & Stat Xian 710129 Peoples R China

The nonconvex and nonsmooth finite -sum optimization problem with linear constraint has attracted much attention in the fields of artificial intelligence, computer, and mathematics, due to its wide applications in machine learning and the lack of efficient algorithms with convincing convergence theories. A popular approach to solve it is the stochastic Alternating Direction Method of Multipliers (ADMM), but most stochastic ADMM-type methods focus on convex models. In addition, the variance reduction (VR) and acceleration techniques are useful tools in the development of stochastic methods due to their simplicity and practicability in providing acceleration characteristics of various machine learning models. However, it remains unclear whether accelerated SVRG-ADMM algorithm (ASVRGADMM), which extends SVRG-ADMM by incorporating momentum techniques, exhibits a comparable acceleration characteristic or convergence rate in the nonconvex setting. To fill this gap, we consider a general nonconvex nonsmooth optimization problem and study the convergence of ASVRG-ADMM. By utilizing a well-defined potential energy function, we establish its sublinear convergence rate O(1/T), where T denotes the iteration number. Furthermore, under the additional Kurdyka-Lojasiewicz (KL) property which is less stringent than the frequently used conditions for showcasing linear convergence rates, such as strong convexity, we show that the ASVRG-ADMM sequence almost surely has a finite length and converges to a stationary solution with a linear convergence rate. Several experiments on solving the graph -guided fused lasso problem and regularized logistic regression problem validate that the proposed ASVRG-ADMM performs better than the state-of-the-art methods. (c) 2024 Elsevier Ltd. All rights reserved.

关键词： nonconvex and nonsmooth optimization Acceleration technique Stochastic ADMM Linear convergence

来源：评论

学校读者我要写书评

暂无评论

Hard thresholding pursuit with continuation for ⁰-regularized minimizations

引用

MATHEMATICAL METHODS IN THE APPLIED SCIENCES 2018年第16期41卷 6195-6209页

作者： Sun, Tao Jiang, Hao Cheng, Lizhi Natl Univ Def Technol Dept Math Changsha 410073 Hunan Peoples R China Natl Univ Def Technol Coll Comp Changsha 410073 Hunan Peoples R China Natl Univ Def Technol State Key Lab High Performance Computat Changsha 410073 Hunan Peoples R China

The (0) regularization has found many applications in imaging science and machine learning research for its good performance. Although the nonconvex proximal splitting method provides a way to solve it, the algorithm performs quite slowly because of that the regularization parameter is always small in applications. In view of this, in this paper, we propose a hard thresholding pursuit algorithm with continuation for the (0)-regularized problem. Such an algorithm has 2 steps in each iteration: in the first one, we choose a subspace by the proximal splitting;and then, we minimize the function over the subspace in the second one. We prove the convergence of this algorithm and apply it to the (0)-regularized least-squares problem. Numerical results demonstrate the efficiency of this algorithm.

关键词： hard thresholding nonconvex and nonsmooth optimization subspace pursuit (0) regularization

来源：评论

学校读者我要写书评

暂无评论

Robust Low-rank subspace segmentation with finite mixture noise

引用

PATTERN RECOGNITION 2019年 93卷 55-67页

作者： Guo, Xianglin Xie, Xingyu Liu, Guangcan Wei, Mingqiang Wang, Jun Nanjing Univ Aeronaut & Astronaut Coll Mech & Elect Engn Nanjing 210016 Peoples R China Nanjing Univ Informat & Technol Dept Informat & Control B DAT Lab Nanjing 210014 Peoples R China Nanjing Univ Aeronaut & Astronaut Coll Comp Sci & Technol Nanjing 210016 Peoples R China

Subspace segmentation or clustering remains a challenge of interest in computer vision when handling complex noise existing in high-dimensional data. Most of the current sparse representation or minimum rank based techniques are constructed on l(1)-norm or l(2)-norm losses, which is sensitive to outliers. Finite mixture model, as a class of powerful and flexible tools for modeling complex noise, becomes a must. Among all the choices, exponential family mixture is extremely useful in practice due to its universal approximation ability for any continuous distribution and hence covers a broader scope of characteristics of noise distribution. Equipped with such a modeling idea, this paper focuses on the complex noise contaminated subspace clustering problem by using finite mixture of exponential power (MoEP) distributions. We then harness a penalized likelihood function to perform automatic model selection and hence avoid over-fitting. Moreover, we introduce a novel prior on the singular values of representation matrix, which leads to a novel penalty in our nonconvex and nonsmooth optimization. The parameters of the MoEP model can be estimated with a Maximum A Posteriori (MAP) method. Meanwhile, the subspace is computed with joint weighted l(p)-norm and Schatten-q quasi-norm minimization. Both theoretical and experimental results show the effectiveness of our method. (C) 2019 Elsevier Ltd. All rights reserved.

关键词： Subspace clustering Noises modelling Finite mixture model nonconvex and nonsmooth optimization

来源：评论

学校读者我要写书评

暂无评论

A Stochastic Proximal Alternating Minimization for nonsmooth and nonconvex

引用

SIAM JOURNAL ON IMAGING SCIENCES 2021年第4期14卷 1932-1970页

作者： Driggs, Derek Tang, Junqi Liang, Jingwei Davies, Mike Schonlieb, Carola-Bibiane Univ Cambridge Dept Appl Math & Theoret Phys Cambridge CB3 0WA England Univ Edinburgh Sch Engn Edinburgh EH9 3JL Midlothian Scotland Shanghai Jiao Tong Univ Inst Nat Sci Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ Sch Math Sci Shanghai 200240 Peoples R China

In this work, we introduce a novel stochastic proximal alternating linearized minimization algorithm [J. Bolte, S. Sabach, and M. Teboulle, Math. Program., 146 (2014), pp. 459--494] for solving a class of nonsmooth and nonconvex optimization problems. Large-scale imaging problems are becoming increasingly prevalent due to the advances in data acquisition and computational capabilities. Motivated by the success of stochastic optimization methods, we propose a stochastic variant of proximal alternating linearized minimization. We provide global convergence guarantees, demonstrating that our proposed method with variance-reduced stochastic gradient estimators, such as SAGA [A. Defazio, F. Bach, and S. Lacoste-Julien, Advances in Neural Information Processing Systems, 2014, pp. 1646--1654] and SARAH [L. M. Nguyen, J. Liu, K. Scheinberg, and M. Taka'\c<^>\, Proceedings of the 34th International Conference on Machine Learning, PMLR 70, 2017, pp. 2613-2621], achieves stateof-the-art oracle complexities. We also demonstrate the efficacy of our algorithm via several numerical examples including sparse nonnegative matrix factorization, sparse principal component analysis, and blind image-deconvolution.

关键词： nonconvex and nonsmooth optimization stochastic optimization variance reduction Kurdyka--Lojasiewicz inequality stochastic PALM

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：