检索结果-内蒙古大学图书馆

Large-Scale Low-Rank Matrix Learning with Nonconvex Regularizers

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2019年第11期41卷 2628-2643页

作者： Yao, Quanming Kwok, James T. Wang, Taifeng Liu, Tie-Yan 4Paradigm Inc Beijing 100089 Peoples R China Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Clear Water Bay Hong Kong Peoples R China Microsoft Res Asia Machine Learning Grp Beijing 100010 Peoples R China Microsoft Res Asia Beijing 100010 Peoples R China

Low-rank modeling has many important applications in computer vision and machine learning. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better empirical performance. However, the resulting optimization problem is much more challenging. Recent state-of-the-art requires an expensive full SVD in each iteration. In this paper, we show that for many commonly-used nonconvex low-rank regularizers, the singular values obtained from the proximal operator can be automatically threshold. This allows the proximal operator to be efficiently approximated by the power method. We then develop a fast proximal algorithm and its accelerated variant with inexact proximal step. It can be guaranteed that the squared distance between consecutive iterates converges at a rate of O(1/T), where T is the number of iterations. Furthermore, we show the proposed algorithm can be parallelized, and the resultant algorithm achieves nearly linear speedup w.r.t. the number of threads. Extensive experiments are performed on matrix completion and robust principal component analysis. Significant speedup over the state-of-the-art is observed.

关键词： Low-rank matrix learning nonconvex regularization proximal algorithm parallel algorithm matrix completion

来源：评论

学校读者我要写书评

暂无评论

Tensor Robust Principal Component Analysis From Multilevel Quantized Observations

引用

IEEE TRANSACTIONS ON INFORMATION THEORY 2023年第1期69卷 383-406页

作者： Wang, Jianjun Hou, Jingyao Eldar, Yonina C. C. Southwest Univ Sch Math & Stat Chongqing 400715 Peoples R China China West Normal Univ Coll Math & Informat Nanchong 637002 Sichuan Peoples R China Weizmann Inst Sci Fac Math & Comp Sci IL-7610001 Rehovot Israel

We consider Quantized Tensor Robust Principal Component Analysis (Q-TRPCA), which aims to recover a low-rank tensor and a sparse tensor from noisy, quantized, and sparsely corrupted measurements. A nonconvex constrained maximum likelihood (ML) estimation method is proposed for Q-TRPCA. We provide an upper bound on the Frobenius norm of tensor estimation error under this method. Making use of tools in information theory, we derive a theoretical lower bound on the best achievable estimation error from unquantized measurements. Compared with the lower bound, the upper bound on the estimation error is nearly order-optimal. We further develop an efficient convex ML estimation scheme for Q-TRPCA based on the tensor nuclear norm (TNN) constraint. This method is more robust to sparse noises than the latter nonconvex ML estimation approach. Conducting experiments on both synthetic data and real-world data, we show the effectiveness of the proposed methods.

关键词： Tensors Sparse matrices Quantization (signal) Principal component analysis Matrix decomposition Optimization Maximum likelihood estimation Low-tubal-rank tensor maximum likelihood estimation multi-level quantization proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Monotone operator theory in convex optimization

引用

MATHEMATICAL PROGRAMMING 2018年第1期170卷 177-206页

作者： Combettes, Patrick L. North Carolina State Univ Dept Math Raleigh NC 27695 USA

Several aspects of the interplay between monotone operator theory and convex optimization are presented. The crucial role played by monotone operators in the analysis and the numerical solution of convex minimization problems is emphasized. We review the properties of subdifferentials as maximally monotone operators and, in tandem, investigate those of proximity operators as resolvents. In particular, we study new transformations which map proximity operators to proximity operators, and establish connections with self-dual classes of firmly nonexpansive operators. In addition, new insights and developments are proposed on the algorithmic front.

关键词： Firmly nonexpansive operator Monotone operator Operator splitting proximal algorithm Proximity operator Proximity-preserving transformation Self-dual class Subdifferential

来源：评论

学校读者我要写书评

暂无评论

Accelerating Large-Scale Statistical Computation With the GOEM algorithm

引用

TECHNOMETRICS 2017年第4期59卷 416-425页

作者： Nie, Xiao Huling, Jared Qian, Peter Z. G. Univ Wisconsin Madison Dept Stat Madison WI 53706 USA

Large-scale data analysis problems have become increasingly common across many disciplines. While large volume of data offers more statistical power, it also brings computational challenges. The orthogonalizing expectation-maximization (EM) algorithm by Xiong et al. is an efficient method to deal with large-scale least-square problems from a design point of view. In this article, we propose a reformulation and generalization of the orthogonalizing EM algorithm. Computational complexity and convergence guarantees are established. The reformulation of the orthogonalizing EM algorithm leads to a reduction in computational complexity for least-square problems and penalized least-square problems. The reformulation, named the GOEM (generalized orthogonalizing EM) algorithm, can incorporate a wide variety of convex and nonconvex penalties, including the lasso, group lasso, and minimax concave penalty penalties. The GOEM algorithm is further extended to a wider class of models including generalized linear models and Cox's proportional hazards model. Synthetic and real data examples are included to illustrate its use and efficiency compared with standard techniques. Supplementary materials for this article are available online.

关键词： Big data Computational statistics OEM Optimization proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Energy Efficient Spectrum Allocation and Mode Selection for D2D Communications in Heterogeneous Networks

引用

IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS 2020年 6卷 382-393页

作者： Galanopoulos, Apostolos Foukalas, Fotis Khattab, Tamer Trinity Coll Dublin Dublin D02 PN40 Ireland Qatar Univ Coll Engn Elect Engn Doha 2713 Qatar

In this paper, we consider a heterogeneous network consisting of both macro Base Station (MBS) and pico Base Stations (PBSs) in order to provide a spectrum allocation and mode selection in device-to-device (D2D) communications. A number of Component Carriers (CC) are considered available for allocation to the MBS and PBSs that are being utilized through carrier aggregation (CA) while mode selection decisions are made by each BS in order to balance between power consumption minimization and UE data rate requirements. A power minimization (energy-efficient) problem is formulated in order to provide a joint spectrum allocation and mode selection solution. This problem is solved using a state of the art optimization method known as proximal algorithm. First, a non-cooperative (centralized) solution is provided and second, a cooperative (distributed) employing distributed proximal algorithm is devised reducing the induced complexity. The cooperative solution is achieved by implementing distributed alternating direction method of multipliers (D-ADMM). Simulation results are carried out for all cases that reveal the energy efficient spectrum allocation and mode selection according under certain channels' conditions that can balance between achieving high data rate requirements and power minimization. Finally, useful insights are presented such as complexity, convergence, delay and actual implementation of such a solution for the future wireless networks.

关键词： Device-to-device communication Resource management Interference Power demand Minimization Wireless sensor networks Receivers Device-to-device energy efficiency spectrum allocation mode selection heterogeneous networks proximal algorithm distributed alternating method of multipliers

来源：评论

学校读者我要写书评

暂无评论

GLOBALLY CONVERGENT JACOBI-TYPE algorithmS FOR SIMULTANEOUS ORTHOGONAL SYMMETRIC TENSOR DIAGONALIZATION

引用

SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS 2018年第1期39卷 1-22页

作者： Li, Jianze Usevich, Konstantin Comon, Pierre Tianjin Univ Sch Math Tianjin 300072 Peoples R China Univ Lorraine Campus SciBP 70239 F-54506 Vandoeuvre Les Nancy France CNRS CRAN UMR 7039 Campus SciBP 70239 F-54506 Vandoeuvre Les Nancy France Univ Grenoble Alpes CNRS Grenoble INP GIPSA Lab F-38000 Grenoble France

In this paper, we consider a family of Jacobi-type algorithms for a simultaneous orthogonal diagonalization problem of symmetric tensors. For the Jacobi-based algorithm of [M. Ishteva, P.-A. Absil, and P. Van Dooren, SIAM T. Matrix Anal. Appl., 34 (2013), pp. 651-672], we prove its global convergence for simultaneous orthogonal diagonalization of symmetric matrices and 3rd-order tensors. We also propose a new Jacobi-based algorithm in the general setting and prove its global convergence for sufficiently smooth functions.

关键词： orthogonal tensor diagonalization Jacobi rotation global convergence Lojasiewicz gradient inequality proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2022年第1期23卷 1-60页

作者： Yao, Quanming Wang, Yaqing Han, Bo Kwok, James T. Tsinghua Univ Dept Elect Engn Beijing Peoples R China Baidu Inc Baidu Res Beijing Peoples R China Hong Kong Baptist Univ Dept Comp Sci Hong Kong Peoples R China Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Hong Kong Peoples R China

Nonconvex regularization has been popularly used in low-rank matrix learning. However, extending it for low-rank tensor learning is still computationally expensive. To address this problem, we develop an efficient solver for use with a nonconvex extension of the overlapped nuclear norm regularizer. Based on the proximal average algorithm, the proposed algorithm can avoid expensive tensor folding/unfolding operations. A special "sparse plus low-rank" structure is maintained throughout the iterations, and allows fast computation of the individual proximal steps. Empirical convergence is further improved with the use of adaptive momentum. We provide convergence guarantees to critical points on smooth losses and also on objectives satisfying the KurdykaLojasiewicz condition. While the optimization problem is nonconvex and nonsmooth, we show that its critical points still have good statistical performance on the tensor completion problem. Experiments on various synthetic and real-world data sets show that the proposed algorithm is efficient in both time and space and more accurate than the existing state-of-the-art.

关键词： Low-rank tensor proximal algorithm proximal average algorithm Nonconvex regu-larization Overlapped nuclear norm

来源：评论

学校读者我要写书评

暂无评论

High-dimensional M-estimation for Byzantine-robust decentralized learning

引用

INFORMATION SCIENCES 2024年 653卷

作者： Zhang, Xudong Wang, Lei Nankai Univ Sch Stat & Data Sci KLMDASR LEBPS Tianjin 300071 Peoples R China Nankai Univ Sch Stat & Data Sci KLMDASR LPMC Tianjin 300071 Peoples R China

In this paper, we focus on robust sparse M-estimation over decentralized networks in the presence of Byzantine attacks. In particular, a decentralized network is modeled as an undirected graph without a central node, while a small fraction of nodes usually behave arbitrarily and send erroneous information due to system breakdowns, cyber attacks and so on. To address the Byzantine issue, some pre-determined robust aggregation rules are applied. Moreover, the gradient tracking and proximal algorithm are combined to ensure convergence and sparsity simultaneously. Theoretically, our proposed algorithms are provably robust against Byzantine attacks and achieve linear convergence rates. The finite-sample performance is studied through numerical experiments under various settings and an application to Communities and Crime Data is also presented.

关键词： Byzantine attacks Square loss transformation Gradient tracking proximal algorithm Linear convergence

来源：评论

学校读者我要写书评

暂无评论

A descent method with linear programming subproblems for nondifferentiable convex optimization

引用

MATHEMATICAL PROGRAMMING 1995年第1期71卷 17-28页

作者： Kim, SH Chang, KN Lee, JY Department of Management Science Korea Advanced Institute of Science and Technology Taejon South Korea

Most of the descent methods developed so far suffer from the computational burden due to a sequence of constrained quadratic subproblems which are needed to obtain a descent direction. In this paper we present a class of proximal-type descent methods with a new direction-finding subproblem. Especially, two of them have a linear programming subproblem instead of a quadratic subproblem. Computational experience of these two methods has been performed on two well-known test problems. The results show that these methods are another very promising approach for nondifferentiable convex optimization.

关键词： descent method proximal algorithm direction-finding subproblem

来源：评论

学校读者我要写书评

暂无评论

FedPT-V2G: Security enhanced federated transformer learning for real-time V2G dispatch with non-IID data

引用

APPLIED ENERGY 2024年 358卷

作者： Shang, Yitong Li, Sen Hong Kong Univ Sci & Technol Dept Civil & Environm Engn Hong Kong 999077 Peoples R China

The rising popularity of electric vehicles (EVs) underscores the potential of vehicle-to-grid (V2G) technology to contribute to load peak-shaving, valley-filling, and photovoltaic (PV) self-consumption. Effective V2G control strategies can be obtained by data-driven techniques, which is able to leverage historical and current data to inform future decision-making amidst uncertainties. However, the centralized collection and sharing of data among charging stations face challenges due to data asset concerns. Furthermore, even if data sharing hurdles are overcome, the non-independent and non-identically distributed (Non-IID) nature of data across charging stations can still negatively impact performance. In this study, we introduce FedPT-V2G, a security-enhanced federated transformer learning approach for real-time V2G dispatch that addresses Non-IID data. We employ deep learning models trained on historical and current data to enable real-time decision-making, facilitating both load shifting and PV self-consumption. Additionally, we utilize federated learning to jointly train a global model across all charging stations without collecting or sharing any local private data. We pioneer the application of the proximal algorithm and Transformer model to tackle data distribution discrepancies within the V2G scheduling prediction task. The proximal algorithm employs regularization techniques to align local models at each charging station more closely with the global model during updates. Concurrently, the multi-head attention mechanism within the Transformer model allows learned feature vectors to diverge, enabling better exploitation of variations across the entire feature space. Finally, we validate the proposed FedPT-V2G approach through extensive numerical simulations, demonstrating comparable performance to centralized learning on both balanced (98.93% vs 98.65%) and imbalanced (92.15% vs 92.20% in label skew) datasets.

关键词： Vehicle-to-grid Digital asset security Federated learning Non-IID data Transformer model proximal algorithm High-efficient

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：