检索结果-内蒙古大学图书馆

4th International Conference on Big Data and Computing (ICBDC)

作者： Wu, Zhongming Wang, Kai Zhou, Zhangjin Southeast Univ Sch Econ & Management Nanjing Peoples R China Nanjing Univ Sci & Technol Sch Sci Nanjing Peoples R China

ISBN: (纸本)9781450362788

In this paper, we propose an accelerated proximal gradient method with line search for solving the large-scale nonconvex penalty problems. Compared with the classic proximal gradient method, the new method does not need to evaluate the Lipschitz constant, parameters are initialized by some aggressive values and updated adaptively at each iteration. Under some common assumptions, the global subsequential convergence is established. Moreover, some numerical results on large-scale SCAD and MCP nonconvex penalty problems are reported to indicate the efficiency and superiority of the proposed method.

关键词： large-scale accelerated proximal gradient method line search convergence nonconvex penalty

来源：评论

学校读者我要写书评

暂无评论

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

引用

JOURNAL OF GLOBAL OPTIMIZATION 2024年第4期89卷 863-897页

作者： Liu, Hongwei Wang, Ting Liu, Zexian Xidian Univ Sch Math & Stat Xian 710126 Peoples R China Xian Univ Posts & Telecommun Sch Sci Xian 710121 Peoples R China Guizhou Univ Sch Math & Stat Guiyang 550025 Peoples R China

In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately;otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka-Lojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

关键词： Nonconvex Nonsmooth accelerated proximal gradient method Variable stepsize strategy Kurdyka-Lojasiewicz property Convergence

来源：评论

学校读者我要写书评

暂无评论

AN accelerated RANDOMIZED proximal COORDINATE gradient method AND ITS APPLICATION TO REGULARIZED EMPIRICAL RISK MINIMIZATION

引用

SIAM JOURNAL ON OPTIMIZATION 2015年第4期25卷 2244-2273页

作者： Lin, Qihang Lu, Zhaosong Xiao, Lin Univ Iowa Tippie Coll Business Iowa City IA 52242 USA Simon Fraser Univ Dept Math Burnaby BC V5A 1S6 Canada Microsoft Res Machine Learning Grp Redmond WA 98052 USA

We consider the problem of minimizing the sum of two convex functions: one is smooth and given by a gradient oracle, and the other is separable over blocks of coordinates and has a simple known structure over each block. We develop an accelerated randomized proximal coordinate gradient (APCG) method for minimizing such convex composite functions. For strongly convex functions, our method achieves faster linear convergence rates than existing randomized proximal coordinate gradient methods. Without strong convexity, our method enjoys accelerated sublinear convergence rates. We show how to apply the APCG method to solve the regularized empirical risk minimization (ERM) problem and devise efficient implementations that avoid full-dimensional vector operations. For ill-conditioned ERM problems, our method obtains improved convergence rates than the state-of-the-art stochastic dual coordinate ascent method.

关键词： convex optimization coordinate descent method randomized algorithm accelerated proximal gradient method empirical risk minimization

来源：评论

学校读者我要写书评

暂无评论

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

引用

NUMERICAL ALGORITHMS 2024年第1期95卷 207-241页

作者： Wang, Ting Liu, Hongwei Xian Univ Posts & Telecommun Sch Sci Xian 710121 Peoples R China Xidian Univ Sch Math & Stat Xian 710126 Peoples R China

Extrapolation, restart and stepsize are very powerful strategies for accelerating the convergence rates of first-order algorithms. In this paper, we propose a modified accelerated proximal gradient algorithm (modAPG), which incorporates the adaptive nonmonotone stepsize strategy, extrapolation and a modified rule of setting the extrapolation parameter to zero at some iterations, for solving a minimization problem composed of a smooth (possibly nonconvex) with Lipschitz continuous gradient and nonsmooth (possibly nonconvex) function. Moreover, the proposed algorithm has a simpler form for this problem with convex setting. We show that any cluster point of the sequence generated by modAPG is a critical point of the problem, and analyze the convergence rates of function values and iterates under the assumption that objective function has the Kurdyka-Lojasiewicz property. Finally, we conduct some preliminary numerical experiments for solving the convex problems, such as image deblurring problem and sparse logistic regression problem;and the nonconvex regularization problem, such as the L-1/2 penalty and the smoothly clipped absolute deviation (SCAD) penalty problem. Numerical experiments demonstrate the promising performance of the proposed method.

关键词： accelerated proximal gradient method Kurdyka-Lojasiewicz property Convergence

来源：评论

学校读者我要写书评

暂无评论

The augmented Lagrangian method based on the APG strategy for an inverse damped gyroscopic eigenvalue problem

引用

COMPUTATIONAL OPTIMIZATION AND APPLICATIONS 2015年第3期62卷 815-850页

作者： Lu, Yue Zhang, Liwei Dalian Univ Technol Sch Math Sci Inst Operat Res & Control Theory Dalian 116024 Peoples R China

In this paper, we propose an augmented Lagrangian method based on the accelerated proximal gradient (APG) strategy for an inverse damped gyroscopic eigenvalue problem (IDGEP), which is a special case of the classical inverse quadratic eigenvalue problem. Under mild conditions, we show that the whole sequence of iterations generated by the proposed algorithm converges to the unique solution of the IDGEP. In view of the iteration-complexity, the proposed algorithm requires at most outer iterations and at most APG calls to obtain an -feasible and -optimal solution of the IDGEP. Numerical results indicate that the proposed algorithm can solve the test problems efficiently.

关键词： Inverse damped gyroscopic eigenvalue problem Augmented Lagrangian method accelerated proximal gradient method Iteration-complexity

来源：评论

学校读者我要写书评

暂无评论

accelerated stochastic gradient descent with step size selection rules

引用

SIGNAL PROCESSING 2019年 159卷 171-186页

作者： Yang, Zhuang Wang, Cheng Zhang, Zhemin Li, Jonathan Xiamen Univ Sch Informat Sci & Engn Fujian Key Lab Sensing & Comp Smart Cities Xiamen 361005 FJ Peoples R China Sun Yat Sen Univ Sch Elect & Commun Engn Guangzhou 510275 Guangdong Peoples R China Univ Waterloo Dept Geog & Environm Management Waterloo ON N2L 3G1 Canada

accelerated stochastic gradient descent (ASGD) methods, which incorporate accelerated proximal gradient (APG) and stochastic gradient (SG), have received considerable attention recently for solving regularized risk minimization problems in signal/image processing, statistics and machine learning. However, there has been a paucity of practical guidance proposed for resolving one of the major issues in ASGD: how to choose an appropriate step size. To solve this problem, we propose to use the Barzilai-Borwein (BB) method to automatically compute step size for the accelerated mini-batch Prox-SVRG (Acc-Prox-SVRG) method (the state of the art ASGD method), thereby obtaining a new accelerated method: Acc-Prox-SVRGBB. We prove the convergence of Acc-Prox-SVRG-BB and show that its complexity is comparable with the best known stochastic gradient methods. In addition, we incorporate Beck and Teboulle's APG (FISTA) and Prox-SVRG in a mini-batch setting and obtain another new accelerated gradient descent method, FISTA-Prox-SVRG, which requires the selection of fewer unknown parameters than those required in Acc-Prox-SVRG. Finally, we introduce the BB method into FISTA-Prox-SVRG to further show the efficacy of the BB method. Numerical results demonstrate the advantage of our algorithms. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Empirical risk minimization accelerated proximal gradient method Mini-batches Barzilai-Borwein method Variance reduction Convex optimization

来源：评论

学校读者我要写书评

暂无评论

MAGMA: Multilevel accelerated gradient Mirror Descent Algorithm for Large-Scale Convex Composite Minimization

引用

SIAM JOURNAL ON IMAGING SCIENCES 2016年第4期9卷 1829-1857页

作者： Hovhannisyan, Vahan Parpas, Panos Zafeiriou, Stefanos Imperial Coll London Dept Comp London SW7 2AZ England

Composite convex optimization models arise in several applications and are especially prevalent in inverse problems with a sparsity inducing norm and in general convex optimization with simple constraints. The most widely used algorithms for convex composite models are accelerated first order methods;however, they can take a large number of iterations to compute an acceptable solution for large-scale problems. In this paper we propose speeding up first order methods by taking advantage of the structure present in many applications and in image processing in particular. Our method is based on multilevel optimization methods and exploits the fact that many applications that give rise to large-scale models can be modeled using varying degrees of fidelity. We use Nesterov's acceleration techniques together with the multilevel approach to achieve an O(1/root epsilon) convergence rate, where epsilon denotes the desired accuracy. The proposed method has a better convergence rate than any other existing multilevel method for convex problems and in addition has the same rate as accelerated methods, which is known to be optimal for first order methods. Moreover, as our numerical experiments show, on large-scale face recognition problems our algorithm is several times faster than the state of the art.

关键词： convex optimization multilevel optimization linear inverse problem optimal gradient methods face recognition accelerated proximal gradient method

来源：评论

学校读者我要写书评

暂无评论

Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2013年第9期35卷 2117-2130页

作者： Hu, Yao Zhang, Debing Ye, Jieping Li, Xuelong He, Xiaofei Zhejiang Univ Coll Comp Sci State Key Lab CAD&CG Hangzhou 310058 Zhejiang Peoples R China Arizona State Univ Dept Comp Sci & Engn Tempe AZ 85287 USA Arizona State Univ Ctr Evolutionary Med & Informat Biodesign Inst Tempe AZ 85287 USA Chinese Acad Sci Xian Inst Opt & Precis Mech State Key Lab Transicent Opt & Photon Ctr Opt IMagery Anal & Learning OPTIMAL Xian 710119 Shaanxi Peoples R China

Recovering a large matrix from a small subset of its entries is a challenging problem arising in many real applications, such as image inpainting and recommender systems. Many existing approaches formulate this problem as a general low-rank matrix approximation problem. Since the rank operator is nonconvex and discontinuous, most of the recent theoretical studies use the nuclear norm as a convex relaxation. One major limitation of the existing approaches based on nuclear norm minimization is that all the singular values are simultaneously minimized, and thus the rank may not be well approximated in practice. In this paper, we propose to achieve a better approximation to the rank of matrix by truncated nuclear norm, which is given by the nuclear norm subtracted by the sum of the largest few singular values. In addition, we develop a novel matrix completion algorithm by minimizing the Truncated Nuclear Norm. We further develop three efficient iterative procedures, TNNR-ADMM, TNNR-APGL, and TNNR-ADMMAP, to solve the optimization problem. TNNR-ADMM utilizes the alternating direction method of multipliers (ADMM), while TNNR-AGPL applies the accelerated proximal gradient line search method (APGL) for the final optimization. For TNNR-ADMMAP, we make use of an adaptive penalty according to a novel update rule for ADMM to achieve a faster convergence rate. Our empirical study shows encouraging results of the proposed algorithms in comparison to the state-of-the-art matrix completion algorithms on both synthetic and real visual datasets.

关键词： Matrix completion nuclear norm minimization alternating direction method of multipliers accelerated proximal gradient method

来源：评论

学校读者我要写书评

暂无评论

An implementable proximal point algorithmic framework for nuclear norm minimization

引用

MATHEMATICAL PROGRAMMING 2012年第1-2期133卷 399-436页

作者： Liu, Yong-Jin Sun, Defeng Toh, Kim-Chuan Natl Univ Singapore Dept Math Singapore 119076 Singapore Natl Univ Singapore Risk Management Inst Singapore 119076 Singapore Shenyang Aerosp Univ Fac Sci Shenyang 110136 Peoples R China Singapore MIT Alliance Singapore 117576 Singapore

The nuclear norm minimization problem is to find a matrix with the minimum nuclear norm subject to linear and second order cone constraints. Such a problem often arises from the convex relaxation of a rank minimization problem with noisy data, and arises in many fields of engineering and science. In this paper, we study inexact proximal point algorithms in the primal, dual and primal-dual forms for solving the nuclear norm minimization with linear equality and second order cone constraints. We design efficient implementations of these algorithms and present comprehensive convergence results. In particular, we investigate the performance of our proposed algorithms in which the inner sub-problems are approximately solved by the gradient projection method or the accelerated proximal gradient method. Our numerical results for solving randomly generated matrix completion problems and real matrix completion problems show that our algorithms perform favorably in comparison to several recently proposed state-of-the-art algorithms. Interestingly, our proposed algorithms are connected with other algorithms that have been studied in the literature.

关键词： Nuclear norm minimization proximal point method Rank minimization gradient projection method accelerated proximal gradient method

来源：评论

学校读者我要写书评

暂无评论

Imputation and low-rank estimation with Missing Not At Random data

引用

STATISTICS AND COMPUTING 2020年第6期30卷 1629-1643页

作者： Sportisse, Aude Boyer, Claire Josse, Julie Sorbonne Univ Lab Probabilites Stat & Modelisat Paris France Ecole Polytech Ctr Math Appl Palaiseau France Ecole Normale Super Dept Math & Applicat Paris France INRIA Saclay XPOP Palaiseau France

Missing values challenge data analysis because many supervised and unsupervised learning methods cannot be applied directly to incomplete data. Matrix completion based on low-rank assumptions are very powerful solution for dealing with missing values. However, existing methods do not consider the case of informative missing values which are widely encountered in practice. This paper proposes matrix completion methods to recover Missing Not At Random (MNAR) data. Our first contribution is to suggest a model-based estimation strategy by modelling the missing mechanism distribution. An EM algorithm is then implemented, involving a Fast Iterative Soft-Thresholding Algorithm (FISTA). Our second contribution is to suggest a computationally efficient surrogate estimation by implicitly taking into account the joint distribution of the data and the missing mechanism: the data matrix is concatenated with the mask coding for the missing values;a low-rank structure for exponential family is assumed on this new matrix, in order to encode links between variables and missing mechanisms. The methodology that has the great advantage of handling different missing value mechanisms is robust to model specification errors. The performances of our methods are assessed on the real data collected from a trauma registry (TraumaBase (R)) containing clinical information about over twenty thousand severely traumatized patients in France. The aim is then to predict if the doctors should administrate tranexomic acid to patients with traumatic brain injury, that would limit excessive bleeding.

关键词： Informative missing values Denoising Matrix completion accelerated proximal gradient method EM algorithm Nuclear norm penalty

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：