检索结果-内蒙古大学图书馆

Asymptotic convergence of an inertial proximal method for unconstrained quasiconvex minimization

JOURNAL OF GLOBAL OPTIMIZATION 2009年第4期45卷 631-644页

作者： Mainge, Paul-Emile Département Scientifique Interfacultaire GRIMAAG Université des Antilles et de la Guyane Campus de Schoelcher 97230 Cedex Martinique (F.W.I.) France

This paper deals with the convergence analysis of a second order proximal method for approaching critical points of a smooth and quasiconvex objective function defined on a real Hilbert space. The considered method, well-known in the convex case, unifies proximal method, relaxation and inertial-type extrapolation. The convergence theorems established in this new setting improve recent ones.

关键词： Quasiconvex minimization proximal algorithm Inertial algorithm Over-relaxation

来源：评论

学校读者我要写书评

暂无评论

Splitting of Composite Neural Networks via proximal Operator With Information Bottleneck

引用

IEEE ACCESS 2024年 12卷 157-167页

作者： Han, Sang-Il Nakamura, Kensuke Hong, Byung-Woo Chung Ang Univ Dept Artificial Intelligence Seoul 06974 South Korea

Deep learning has achieved efficient success in the field of machine learning, made possible by the emergence of efficient optimization methods such as Stochastic Gradient Descent (SGD) and its variants. Simultaneously, the Information Bottleneck theory (IB) has been studied to train neural networks, aiming to enhance the performance of optimization methods. However, previous works have focused on their specific tasks, and the effect of the IB theory on general deep learning tasks is still unclear. In this study, we introduce a new method inspired by the proximal operator, which sequentially updates the neural network parameters based on the defined bottleneck features between the forward and backward networks. Unlike the conventional proximal-based methods, we consider the second-order gradients of the objective function to achieve better updates for the forward networks. In contrast to SGD-based methods, our approach involves accessing the network's black box, and incorporating the bottleneck feature update process into the parameter update process. This way, from the perspective of the IB theory, the data is well compressed up to the bottleneck feature, ensuring that the compressed information maintains sufficient mutual information up to the final output. To demonstrate the performance of the proposed approach, we applied the method to various optimizers with several tasks and analyzed the results by training on both the MNIST dataset and CIFAR-10 dataset. We also conducted several ablation studies by modifying the components of the proposed algorithm to further validate its performance.

关键词： Linear programming Deep learning Task analysis Mutual information Training Optimization methods Biological neural networks information bottleneck stochastic gradient descent proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

On a primal-proximal heuristic in discrete optimization

引用

MATHEMATICAL PROGRAMMING 2005年第1期104卷 105-128页

作者： Daniilidis, A Lemaréchal, C INRIA F-38334 Saint Ismier France

Lagrangian relaxation is useful to bound the optimal value of a given optimization problem, and also to obtain relaxed solutions. To obtain primal solutions, it is conceivable to use a convexification procedure suggested by D.P. Bertsekas in 1979, based on the proximal algorithm in the primal space. The present paper studies the theory assessing the approach in the framework of combinatorial optimization. Our results indicate that very little can be expected in theory, even though fairly good practical results have been obtained for the unit-commitment problem.

关键词： proximal algorithm Lagrangian relaxation primal-dual heuristics

来源：评论

学校读者我要写书评

暂无评论

Automated Regularization Parameter Selection Using Continuation Based proximal Method for Compressed Sensing MRI

引用

IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING 2020年 6卷 1309-1319页

作者： Mathew, Raji Susan Paul, Joseph Suresh Indian Inst Informat Technol Med Image Comp & Signal Proc Lab Trivandrum 695581 Kerala India

For compressed sensing magnetic resonance imaging (CS-MRI) that utilize sparse representations, the regularization parameter establishes a trade-off between sparsity and data fidelity. While convergence to the desired solution is slow for mean squared error (MSE) optimal constant regularization, continuation using decreasing parameter values enables faster convergence. To derive an explicit rule for continuation, we propose an intermediate step optimization that involves maximization of the l(2)-norm of the gradient descent update. This is achieved by inclusion of an extra prior to the CS-MRI cost function. The solution is obtained using an alternating minimization approach in which the first sub-problem deals with the sparse regularization using the previously computed parameter value, and the second sub-problem aims at estimation of the parameter value to be used in the succeeding iteration. The solution to the second sub-problem is computed using standard root finding methods. Irrespective of the initial choice of the regularization parameter, we show that application of this continuation based proximal approach enables faster convergence to the desired solution.

关键词： Convergence Image reconstruction Minimization Magnetic resonance imaging Optimization Transforms Continuation scheme proximal algorithm regularization parameter scale factor sparse reconstruction

来源：评论

学校读者我要写书评

暂无评论

Accelerated proximal Subsampled Newton Method

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021年第10期32卷 4374-4388页

作者： Ye, Haishan Luo, Luo Zhang, Zhihua Chinese Univ Hong Kong Shenzhen Res Inst Big Data Shenzhen 518172 Peoples R China Hong Kong Univ Sci & Technol Dept Math Hong Kong Peoples R China Peking Univ Dept Math Beijing 100871 Peoples R China

Composite function optimization problem often arises in machine learning known as regularized empirical minimization. We introduce the acceleration technique to the Newton-type proximal method and propose a novel algorithm called accelerated proximal subsampled Newton method (APSSN). APSSN only subsamples a small subset of samples to construct an approximate Hessian that achieves computational efficiency. At the same time, APSSN still keeps a fast convergence rate. Furthermore, we obtain the scaled proximal mapping by solving its dual problem using the semismooth Newton method instead of resorting to the first-order methods. Due to our sampling strategy and the fast convergence rate of the semismooth Newton method, we can get the scaled proximal mapping efficiently. Both our theoretical analysis and empirical study show that APSSN is an effective and computationally efficient algorithm for composite function optimization problems.

关键词： Convergence Acceleration Approximation algorithms Newton method Optimization Machine learning Stochastic processes Approximate Newton Nesterov's acceleration nonsmooth optimization proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Stochastic proximal subgradient descent oscillates in the vicinity of its accumulation set

引用

OPTIMIZATION LETTERS 2023年第1期17卷 177-190页

作者： Schechtman, S. Univ Gustave Eiffel ESIEE Paris LIGM F-77454 Marne La Vallee France

We analyze the stochastic proximal subgradient descent in the case where the objective functions are path differentiable and verify a Sard-type condition. While the accumulation set may not be reduced to unique point, we show that the time spent by the iterates to move from one accumulation point to another goes to infinity. An oscillation-type behavior of the drift is established. These results show a strong stability property of the proximal subgradient descent. Using the theory of closed measures, Bolte et al. (2020) established this type of behavior for the deterministic subgradient descent. Our technique of proof relies on the classical works on stochastic approximation of differential inclusions, which allows us to extend results in the deterministic case to a stochastic and proximal setting, as well as to treat these different cases in a unified manner.

关键词： Subgradient descent proximal algorithm Non convex and non smooth optimization Stochastic approximation Tame optimization Differential inclusions

来源：评论

学校读者我要写书评

暂无评论

An Off-Grid DOA Estimation Method Using proximal Splitting and Successive Nonconvex Sparsity Approximation

引用

IEEE ACCESS 2019年 7卷 66764-66773页

作者： Zhang, Xiaowei Jiang, Tao Li, Yingsong Liu, Xiaoguang Harbin Engn Univ Coll Informat & Commun Engn Harbin 150001 Heilongjiang Peoples R China Chinese Acad Sci Natl Space Sci Ctr Key Lab Microwave Remote Sensing Beijing 100190 Peoples R China Univ Calif Davis Elect & Comp Engn Dept Davis CA 95616 USA

Direction-of-arrival (DOA) estimation is a fundamental problem in many signal processing. Recently, a variety of sparsity-aware methods have been proposed for DOA estimation. The discrimination of grid in these methods often suffers from grid mismatch problem, which degrades the performance of DOA estimation, when the true unknown DOAs are not on the potential angular grids. To deal with grid mismatch problem in conventional sparsity-based methods, a novel off-grid model is proposed via Taylor interpolation that jointly estimates the sparse signals and grid offset parameters, using a family of successive nonconvex sparsity approximation penalties on the sparse signals. Then an iterative alternating optimization strategy is utilized to tackle this nonconvex problem with respect to sparse signals and grid offset parameters, and the proposed algorithm is solved by the proximal splitting algorithm in each iteration, where the optimization problem is split into two parts: sparsity and projection. The proposed framework in this paper is universal and also can be extended to other off-grid sparsity-based algorithms. The extensive simulations are conducted to verify that the proposed method achieves superior resolution and more accurate DOA estimation performance than other conventional sparsity-based method and the state-of-the-art off-grid methods in many cases of practical interest.

关键词： Direction-of-arrival (DOA) estimation off-grid model sparsity approximation proximal algorithm iterative approach

来源：评论

学校读者我要写书评

暂无评论

DUALITY RESULTS AND proximal SOLUTIONS OF THE HUBER M-ESTIMATOR PROBLEM

引用

APPLIED MATHEMATICS AND OPTIMIZATION 1994年第2期30卷 203-221页

作者： MICHELOT, C BOUGEARD, ML UNIV BOURGOGNE F-21004 DIJONFRANCE OBSERV PARIS CNRSURA 1125F-75014 PARISFRANCE

We investigate the interest of solving the Huber M-estimator problem by a proximal approach combined with duality theory. Three different duality schemes are developed. The first one which only deals with estimator determination yields useful information on the geometrical structure of the set of optimal solutions. The second scheme links together estimator determination and outliers detection while the third one only focuses on outliers separation. We show that these three duality schemes can be solved by the partial inverse method, i.e., a special instance of the basic proximal point algorithm, which leads to very simple updating rules. This method which is always globally convergent enjoys nice stability properties and permits parallel computations.

关键词： ROBUSTNESS HUBER M-ESTIMATOR OUTLIERS IDENTIFICATION DUALITY proximal algorithm PARTIAL INVERSE METHOD

来源：评论

学校读者我要写书评

暂无评论

A primal-proximal heuristic applied to the French Unit-commitment problem

引用

MATHEMATICAL PROGRAMMING 2005年第1期104卷 129-151页

作者： Dubost, L Gonzalez, R Lemaréchal, C EDF R&D F-92141 Clamart France EDF F-78005 Versailles France INRIA F-38334 Saint Ismier France

This paper is devoted to the numerical resolution of unit-commitment problems, with emphasis on the French model optimizing the daily production of electricity. The solution process has two phases. First a Lagrangian relaxation solves the dual to find a lower bound;it also gives a primal relaxed solution. We then propose to use the latter in the second phase, for a heuristic resolution based on a primal proximal algorithm. This second step comes as an alternative to an earlier approach, based on augmented Lagrangian (i.e. a dual proximal algorithm). We illustrate the method with some real-life numerical results. A companion paper is devoted to a theoretical study of the heuristic in the second phase.

关键词： unit-commitment problem proximal algorithm Lagrangian relaxation primal-dual heuristics combinatorial optimization

来源：评论

学校读者我要写书评

暂无评论

Online proximal Learning Over Jointly Sparse Multitask Networks With l∞,1 Regularization

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2020年 68卷 6319-6335页

作者： Jin, Danqi Chen, Jie Richard, Cedric Chen, Jingdong Northwestern Polytech Univ Ctr Intelligent Acoust & Immers Commun Sch Marine Sci & Technol Xian 710072 Peoples R China Minist Ind & Informat Technol Key Lab Ocean Acoust & Sensing Xian 710072 Peoples R China Univ Cote Azur CNRS F-06100 Nice France

Modeling relations between local optimum parameter vectors to estimate in multitask networks has attracted much attention over the last years. This work considers a distributed optimization problem with jointly sparse structure among nodes, that is, the local solutions have the same sparse support set. Several mixed norm have been proposed to address the jointly sparse structure in the literature. Among several candidates, the (reweighted) l(infinity,1)-norm is element-wise separable, it is more convenient to evaluate their approximate proximal operators. Thus by introducing a (reweighted) l(infinity,1)-norm penalty term at each node, and using a proximal gradient method to minimize the regularized cost, we devise a proximal multitask diffusion LMS algorithm which can promote joint-sparsity. Analyses are provided to characterize the algorithm behavior in the mean and mean-square sense. Simulation results are presented to show its effectiveness, as well as the accuracy of the theoretical findings.

关键词： Distributed optimization diffusion strategy joint sparsity proximal algorithm stochastic performance (reweighted) l(infinity,1-)norm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：