检索结果-内蒙古大学图书馆

On Acceleration for Convex Composite Minimization with Noise-Corrupted Gradients and approximate proximal mapping

JOURNAL OF MACHINE LEARNING RESEARCH 2022年第1期23卷 1-59页

作者： Zhou, Qiang Pan, Sinno Jialin Southeast Univ Sch Cyber Sci & Engn Nanjing 211189 Jiangsu Peoples R China Purple Mt Labs Nanjing 211111 Jiangsu Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore 639798 Singapore

The accelerated proximal methods (APM) have become one of the most important opti-mization tools for large-scale convex composite minimization problems, due to their wide range of applications and the optimal convergence rate in first-order algorithms. However, most existing theoretical results of APM are obtained by assuming that the gradient or-acle is exact and the proximal mapping must be exactly solved, which may not hold in practice. This work presents a theoretical study of APM by allowing to use inexact gradi-ent oracle and approximate proximal mapping. Specifically, we analyze inexact APM by improving the approximate duality gap technique (ADGT) which was originally designed for convergence analysis for first-order methods with both exact gradient oracle and prox-imal mapping. Our approach has several advantages: 1) we provide a unified convergence analysis that allows both inexact gradient oracle and approximate proximal mapping;2) our proof is generic that naturally recovers the convergence rates of both accelerated and non-accelerated proximal methods, on top of which the advantages and the disadvantages of acceleration can be easily derived;3) we derive the same convergence bound as previous methods in terms of inexact gradient oracle, but a tighter convergence bound in terms of approximate proximal mapping.

关键词： Convex Composite Minimization Accelerated proximal Methods Noisy Gradients approximate proximal mapping Bregman Divergence

来源：评论

学校读者我要写书评

暂无评论

On acceleration for convex composite minimization with noise-corrupted gradients and approximate proximal mapping

引用

The Journal of Machine Learning Research 2022年第1期23卷 10098-10156页

作者： Qiang Zhou Sinno Jialin Pan School of Cyber Science and Engineering Southeast University Nanjing Jiangsu China and Purple Mountain Laboratories Nanjing Jiangsu China School of Computer Science and Engineering Nanyang Technological University Singapore

The accelerated proximal methods (APM) have become one of the most important optimization tools for large-scale convex composite minimization problems, due to their wide range of applications and the optimal convergence rate in first-order algorithms. However, most existing theoretical results of APM are obtained by assuming that the gradient oracle is exact and the proximal mapping must be exactly solved, which may not hold in practice. This work presents a theoretical study of APM by allowing to use inexact gradient oracle and approximate proximal mapping. Specifically, we analyze inexact APM by improving the approximate duality gap technique (ADGT) which was originally designed for convergence analysis for first-order methods with both exact gradient oracle and proximal mapping. Our approach has several advantages: 1) we provide a unified convergence analysis that allows both inexact gradient oracle and approximate proximal mapping; 2) our proof is generic that naturally recovers the convergence rates of both accelerated and non-accelerated proximal methods, on top of which the advantages and the disadvantages of acceleration can be easily derived; 3) we derive the same convergence bound as previous methods in terms of inexact gradient oracle, but a tighter convergence bound in terms of approximate proximal mapping.

关键词： convex composite minimization accelerated proximal methods noisy gradients approximate proximal mapping Bregman divergence

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：