检索结果-内蒙古大学图书馆

Inexact proximal gradient algorithm with random reshuffling for nonsmooth optimization

Science China(Information Sciences) 2025年第1期68卷 219-237页

作者： Xia JIANG Yanyan FANG Xianlin ZENG Jian SUN Jie CHEN National Key Lab of Autonomous Intelligent Unmanned Systems School of AutomationBeijing Institute of Technology Beijing Institute of Technology Chongqing Innovation Center School of Electronic and Information Engineering Tongji University

Proximal gradient algorithms are popularly implemented to achieve convex optimization with nonsmooth regularization. Obtaining the exact solution of the proximal operator for nonsmooth regularization is challenging because errors exist in the computation of the gradient; consequently, the design and application of inexact proximal gradient algorithms have attracted considerable attention from researchers. This paper proposes computationally efficient basic and inexact proximal gradient descent algorithms with random reshuffling. The proposed stochastic algorithms take randomly reshuffled data to perform successive gradient descents and implement only one proximal operator after all data pass through. We prove the convergence results of the proposed proximal gradient algorithms under the sampling-without-replacement reshuffling *** computational errors exist in gradients and proximal operations, the proposed inexact proximal gradient algorithms can converge to an optimal solution neighborhood. Finally, we apply the proposed algorithms to compressed sensing and compare their efficiency with some popular algorithms.

关键词： proximal operator random reshuffling inexact computation compressed sensing nonsmooth optimization

来源：评论

学校读者我要写书评

暂无评论

Solution Existence and Compactness Analysis for nonsmooth optimization Problems

引用

JOURNAL OF optimization THEORY AND APPLICATIONS 2025年第1期205卷 1-25页

作者： Hung, Nguyen Canh Chuong, Thai Doan Anh, Nguyen Le Hoang Univ Sci Ho Chi Minh City Fac Math & Comp Sci Ho Chi Minh City Vietnam Vietnam Natl Univ Ho Chi Minh City Vietnam Brunel Univ London Dept Math London England

This paper is concerned with the analysis of geometrical properties and behaviors of the optimal value and global optimal solutions for a class of nonsmooth optimization problems. We provide conditions under which the solution set of a nonsmooth and nonconvex optimization problem is non-empty and/or compact. We also examine related properties such as the compactness of the sublevel sets, the boundedness from below and the coercivity of the objective function to characterize the non-emptiness and the compactness of the solution set of the underlying optimization problem under the unboundedness of its associated feasible set.

关键词： Solution existence Limiting subdifferential nonsmooth optimization Coercivity Compactness

来源：评论

学校读者我要写书评

暂无评论

An inexact ADMM for separable nonconvex and nonsmooth optimization

引用

COMPUTATIONAL optimization AND APPLICATIONS 2025年第2期90卷 445-479页

作者： Bai, Jianchao Zhang, Miao Zhang, Hongchao Northwestern Polytech Univ Shenzhen Res & Dev Inst Shenzhen 518057 Peoples R China Northwestern Polytech Univ Sch Math & Stat Xian 710072 Peoples R China Louisiana State Univ Dept Math Baton Rouge LA 70803 USA

An inexact alternating direction method of multiplies (I-ADMM) with an expansion linesearch step was developed for solving a family of separable minimization problems subject to linear constraints, where the objective function is the sum of a smooth but possibly nonconvex function and a possibly nonsmooth nonconvex function. Global convergence and linear convergence rate of the I-ADMM were established under proper conditions while inexact relative error criterion was used for solving the subproblems. In addition, a unified proximal gradient (UPG) method with momentum acceleration was proposed for solving the smooth but possibly nonconvex subproblem. This UPG method guarantees global convergence and will automatically reduce to an optimal accelerated gradient method when the smooth function in the objective is convex. Our numerical experiments on solving nonconvex quadratic programming problems and sparse optimization problems from statistical learning show that the proposed I-ADMM is very effective compared with other state-of-the-art algorithms in the literature.

关键词： Nonconvex optimization nonsmooth optimization Separable structure Lipschitz continuous Inexact ADMM Accelerated gradient method Global convergence Linear convergence rate

来源：评论

学校读者我要写书评

暂无评论

On a globally convergent semismooth* Newton method in nonsmooth nonconvex optimization

引用

COMPUTATIONAL optimization AND APPLICATIONS 2025年第1期91卷 67-124页

作者： Gfrerer, Helmut Johannes Kepler Univ Linz Inst Computat Math A-4040 Linz Austria Austrian Acad Sci OAW Johann Radon Inst Computat & Appl Math RICAM A-4040 Linz Austria

In this paper we present GSSN, a globalized SCD semismooth* Newton method for solving nonsmooth nonconvex optimization problems. The global convergence properties of the method are ensured by the proximal gradient method, whereas locally superlinear convergence is established via the SCD semismooth* Newton method under quite weak assumptions. The Newton direction is based on the SC (subspace containing) derivative of the subdifferential mapping and can be computed by the (approximate) solution of an equality-constrained quadratic program. Special attention is given to the efficient numerical implementation of the overall method.

关键词： nonsmooth optimization Nonconvex Proximal gradient method Global convergence Superlinear convergence nonsmooth optimization Nonconvex Semismooth(& lowast ) Newton method Proximal gradient method Global convergence Superlinear convergence

来源：评论

学校读者我要写书评

暂无评论

Learning-Rate-Free Momentum SGD with Reshuffling Converges in nonsmooth Nonconvex optimization

引用

JOURNAL OF SCIENTIFIC COMPUTING 2025年第3期102卷 1-31页

作者： Hu, Xiaoyin Xiao, Nachuan Liu, Xin Toh, Kim-Chuan Hangzhou City Univ Sch Comp & Comp Sci Hangzhou 310015 Peoples R China Hangzhou City Univ Acad Edge Intelligence Hangzhou Peoples R China Natl Univ Singapore Inst Operat Res & Analyt Singapore Singapore Chinese Acad Sci Acad Math & Syst Sci State Key Lab Sci & Engn Comp Beijing Peoples R China Univ Chinese Acad Sci Beijing Peoples R China Natl Univ Singapore Dept Math Singapore 119076 Singapore Natl Univ Singapore Inst Operat Res & Analyt Singapore 119076 Singapore

In this paper, we propose a generalized framework for developing learning-rate-free momentum stochastic gradient descent (SGD) methods in the minimization of nonsmooth nonconvex functions, especially in training nonsmooth neural networks. Our framework adaptively generates learning rates based on the historical data of stochastic subgradients and iterates. Under mild conditions, we prove that our proposed framework enjoys global convergence to the stationary points of the objective function in the sense of the conservative field, hence providing convergence guarantees for training nonsmooth neural networks. Based on our proposed framework, we propose a novel learning-rate-free momentum SGD method (LFM). Preliminary numerical experiments reveal that LFM performs comparably to the state-of-the-art learning-rate-free methods (which have not been shown theoretically to be convergent) across well-known neural network training benchmarks.

关键词： nonsmooth optimization Stochastic subgradient methods Nonconvex optimization Learning-rate free Differential inclusion

来源：评论

学校读者我要写书评

暂无评论

nonsmooth optimization techniques for semisupervised classification

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2007年第12期29卷 2135-2142页

作者： Astorino, Annabella Fuduli, Antonio Univ Calabria Dipartimento Elettron Informat & Sistemist I-87036 Arcavacata Di Rende CS Italy Univ Calabria Dipartimento Matemat I-87036 Arcavacata Di Rende CS Italy

We apply nonsmooth optimization techniques to classification problems, with particular reference to the Transductive Support Vector Machine ( TSVM) approach, where the considered decision function is nonconvex and nondifferentiable, hence difficult to minimize. We present some numerical results obtained by running the proposed method on some standard test problems drawn from the binary classification literature.

关键词： semisupervised learning nonsmooth optimization bundle methods

来源：评论

学校读者我要写书评

暂无评论

nonsmooth optimization through mesh adaptive direct search and variable neighborhood search

引用

JOURNAL OF GLOBAL optimization 2008年第2期41卷 299-318页

作者： Audet, Charles Bechard, Vincent Le Digabel, Sebastien Ecole Polytech Dept Math & Genie Ind Montreal PQ H3C 3A7 Canada Gerad Montreal PQ H3C 3A7 Canada

This paper proposes a way to combine the Mesh Adaptive Direct Search (MADS) algorithm, which extends the Generalized Pattern Search (GPS) algorithm, with the Variable Neighborhood Search (VNS) metaheuristic, for nonsmooth constrained optimization. The resulting algorithm retains the convergence properties of MADS, and allows the far reaching exploration features of VNS to move away from local solutions. The paper also proposes a generic way to use surrogate functions in the VNS search. Numerical results illustrate advantages and limitations of this method.

关键词： nonsmooth optimization mesh adaptive direct search generalized pattern search variable neighborhood search

来源：评论

学校读者我要写书评

暂无评论

nonsmooth optimization for multiband frequency domain control design

引用

AUTOMATICA 2007年第4期43卷 724-731页

作者： Apkarian, Pierre Noll, Dominikus ONERA CERT Ctr Etud & Rech Toulouse Control Syst Dept F-31055 Toulouse France Univ Toulouse 3 Inst Math F-31062 Toulouse France

Multiband frequency domain synthesis consists in the minimization of a finite family of closed-loop transfer functions on prescribed frequency intervals. This is an algorithmically difficult problem due to its inherent nonsmoothness and nonconvexity. We extend our previous work on nonsmooth H-infinity synthesis to develop a nonsmooth optimization technique to compute local solutions to multiband synthesis problems. The proposed method is shown to perform well on illustrative examples. (C) 2007 Elsevier Ltd. All rights reserved.

关键词： H-infinity-synthesis multidisk problems structured controller design nonsmooth optimization

来源：评论

学校读者我要写书评

暂无评论

nonsmooth optimization-Based Model and Algorithm for Semisupervised Clustering

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023年第9期34卷 5517-5530页

作者： Bagirov, Adil M. Taheri, Sona Bai, Fusheng Zheng, Fangying Federat Univ Australia Sch Engn Informat Technol & Phys Sci Ballarat Vic 3353 Australia RMIT Univ Sch Sci Melbourne Vic 3000 Australia Chongqing Normal Univ Natl Ctr Appl Math Chongqing Chongqing 401331 Peoples R China Zhejiang Sci Tech Univ Dept Math Sci Hangzhou 310018 Peoples R China

Using a nonconvex nonsmooth optimization approach, we introduce a model for semisupervised clustering (SSC) with pairwise constraints. In this model, the objective function is represented as a sum of three terms: the first term reflects the clustering error for unlabeled data points, the second term expresses the error for data points with must-link (ML) constraints, and the third term represents the error for data points with cannot-link (CL) constraints. This function is nonconvex and nonsmooth. To find its optimal solutions, we introduce an adaptive SSC (A-SSC) algorithm. This algorithm is based on the combination of the nonsmooth optimization method and an incremental approach, which involves the auxiliary SSC problem. The algorithm constructs clusters incrementally starting from one cluster and gradually adding one cluster center at each iteration. The solutions to the auxiliary SSC problem are utilized as starting points for solving the nonconvex SSC problem. The discrete gradient method (DGM) of nonsmooth optimization is applied to solve the underlying nonsmooth optimization problems. This method does not require subgradient evaluations and uses only function values. The performance of the A-SSC algorithm is evaluated and compared with four benchmarking SSC algorithms on one synthetic and 12 real-world datasets. Results demonstrate that the proposed algorithm outperforms the other four algorithms in identifying compact and well-separated clusters while satisfying most constraints.

关键词： Clustering algorithms Partitioning algorithms optimization Measurement Costs Semisupervised learning Linear programming Cluster analysis discrete gradient method (DGM) nonsmooth optimization pairwise constraints semisupervised clustering (SSC)

来源：评论

学校读者我要写书评

暂无评论

nonsmooth optimization for Efficient Beamforming in Cognitive Radio Multicast Transmission

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2012年第6期60卷 2941-2951页

作者： Anh Huy Phan Hoang Duong Tuan Ha Hoang Kha Duy Trong Ngo Univ New S Wales Sch Elect Engn & Telecommun Unsw Sydney NSW 2052 Australia Univ Technol Sydney Fac Engn & Informat Technol Broadway NSW 2007 Australia McGill Univ Dept Elect & Comp Engn Montreal PQ HA3 2A7 Canada

It is known that the design of optimal transmit beamforming vectors for cognitive radio multicast transmission can be formulated as indefinite quadratic optimization programs. Given the challenges of such nonconvex problems, the conventional approach in literature is to recast them as convex semidefinite programs (SDPs) together with rank-one constraints. Then, these nonconvex and discontinuous constraints are dropped allowing for the realization of a pool of relaxed candidate solutions, from which various randomization techniques are utilized with the hope to recover the optimal solutions. However, it has been shown that such approach fails to deliver satisfactory outcomes in many practical settings, wherein the determined solutions are found to be unacceptably far from the actual optimality. On the contrary, we in this contribution tackle the aforementioned optimal beamforming problems differently by representing them as SDPs with additional reverse convex (but continuous) constraints. Non-smooth optimization algorithms are then proposed to locate the optimal solutions of such design problems in an efficient manner. Our thorough numerical examples verify that the proposed algorithms offer almost global optimality whilst requiring relatively low computational load.

关键词： Beamforming cognitive radio multicast transmission nonsmooth optimization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：