检索结果-内蒙古大学图书馆

The Journal of Machine Learning Research 2024年第1期25卷 10882-10934页

作者： Emilie Chouzenoux Víctor Elvira Center for Visual Computing Inria University Paris Saclay Gif-sur-Yvette France School of Mathematics University of Edinburgh Edinburgh UK

Time-series datasets are central in machine learning with applications in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Learning the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on various synthetic data showcases the effectiveness of the proposed model and inference algorithm. This work will significantly contribute to the understanding and utilization of time-series data in diverse scientific and engineering applications where incorporating a graphical approach is essential to perform the i

关键词： state-space models graph inference sparsity graphical lasso majorization-minimization proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Distributed decision-coupled constrained optimization via proximal-Tracking

引用

AUTOMATICA 2022年 135卷 109938-109938页

作者： Falsone, Alessandro Prandini, Maria Politecn Milan Dipartimento Elettron Informaz & Bioingn Via Ponzio 34-5 I-20133 Milan Italy

In this paper we deal with decision-coupled problems involving multiple agents over a network. Each agent has its own local objective function and local constraints, and all agents aim at finding the value of a common decision vector that minimizes the sum of all agents' cost functions and satisfies all local constraints. To this purpose, we introduce a proximal-Tracking distributed optimization algorithm that integrates dynamic average consensus within the proximal minimization method. Convergence to an optimal consensus solution is guaranteed for any value of a constant penalty parameter, under a convexity assumption only, without requiring differentiability, Lipschitz continuity, or smoothness of the local objective functions. Numerical simulations show the effectiveness of the proposed scheme. (C) 2021 Elsevier Ltd. All rights reserved.

关键词： \ Distributed optimization Decision-coupled optimization proximal algorithm Gradient-tracking

来源：评论

学校读者我要写书评

暂无评论

Low-complexity proximal Gauss-Newton algorithm for Nonnegative Matrix Factorization 7

Low-complexity Proximal Gauss-Newton Algorithm for Nonnegati...

引用

7th IEEE Global Conference on Signal and Information Processing (IEEE GlobalSIP)

作者： Huang, Kejun Fu, Xiao Univ Florida Dept CISE Gainesville FL 32611 USA Oregon State Univ Sch EECS Corvallis OR 97331 USA

ISBN: (纸本)9781728127231

In this paper we propose a quasi-Newton algorithm for the celebrated nonnegative matrix factorization (NMF) problem. The proposed algorithm falls into the general framework of Gauss-Newton and Levenberg-Marquardt methods. However, these methods were not able to handle constraints, which is present in NME One of the key contributions in this paper is to apply alternating direction method of multipliers (ADMM) to obtain the iterative update from this Gauss-Newton-like algorithm. Furthermore, we carefully study the structure of the Jacobian Gramian matrix given by the Gauss-Newton updates, and designed a way of exactly inverting the matrix with complexity (O(mnk), which is a significant reduction compared to the naive implementation of complexity (O((m + n)(3)k(3)). The resulting algorithm, which we call NLS-ADMM, enjoys fast convergence rate brought by the quasi-Newton algorithmic framework, while maintaining low per-iteration complexity similar to that of alternating algorithms. Numerical experiments on synthetic data confirms the efficiency of our proposed algorithm.

关键词： nonnegative matrix factorization Gauss-Newton proximal algorithm alternating direction method of multipliers low-complexity

来源：评论

学校读者我要写书评

暂无评论

Factorization Machines with Regularization for Sparse Feature Interactions

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2021年第1期22卷 1-50页

作者： Atarashi, Kyohei Oyama, Satoshi Kurihara, Masahito Hokkaido Univ Grad Sch Informat Sci & Technol Kita Ku Kita 14Nishi 9 Sapporo Hokkaido 0600814 Japan Hokkaido Univ Fac Informat Sci & Technol Kita Ku Kita 14Nishi 9 Sapporo Hokkaido 0600814 Japan

Factorization machines (FMs) are machine learning predictive models based on second-order fea-ture interactions and FMs with sparse regularization are called sparse FMs. Such regularizations enable feature selection, which selects the most relevant features for accurate prediction, and there-fore they can contribute to the improvement of the model accuracy and interpretability. However, because FMs use second-order feature interactions, the selection of features often causes the loss of many relevant feature interactions in the resultant models. In such cases, FMs with regularization specially designed for feature interaction selection trying to achieve interaction-level sparsity may be preferred instead of those just for feature selection trying to achieve feature-level sparsity. In this paper, we present a new regularization scheme for feature interaction selection in FMs. For feature interaction selection, our proposed regularizer makes the feature interaction matrix sparse without a restriction on sparsity patterns imposed by the existing methods. We also describe efficient proximal algorithms for the proposed FMs and how our ideas can be applied or extended to feature selection and other related models such as higher-order FMs and the all-subsets model. The analysis and experimental results on synthetic and real-world datasets show the effectiveness of the proposed methods.

关键词： factorization machines sparse regularization feature interaction selection feature selection proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Automated Regularization Parameter Selection Using Continuation Based proximal Method for Compressed Sensing MRI

引用

IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING 2020年 6卷 1309-1319页

作者： Mathew, Raji Susan Paul, Joseph Suresh Indian Inst Informat Technol Med Image Comp & Signal Proc Lab Trivandrum 695581 Kerala India

For compressed sensing magnetic resonance imaging (CS-MRI) that utilize sparse representations, the regularization parameter establishes a trade-off between sparsity and data fidelity. While convergence to the desired solution is slow for mean squared error (MSE) optimal constant regularization, continuation using decreasing parameter values enables faster convergence. To derive an explicit rule for continuation, we propose an intermediate step optimization that involves maximization of the l(2)-norm of the gradient descent update. This is achieved by inclusion of an extra prior to the CS-MRI cost function. The solution is obtained using an alternating minimization approach in which the first sub-problem deals with the sparse regularization using the previously computed parameter value, and the second sub-problem aims at estimation of the parameter value to be used in the succeeding iteration. The solution to the second sub-problem is computed using standard root finding methods. Irrespective of the initial choice of the regularization parameter, we show that application of this continuation based proximal approach enables faster convergence to the desired solution.

关键词： Convergence Image reconstruction Minimization Magnetic resonance imaging Optimization Transforms Continuation scheme proximal algorithm regularization parameter scale factor sparse reconstruction

来源：评论

学校读者我要写书评

暂无评论

Product of Resolvents on Hadamard Manifolds

引用

MEDITERRANEAN JOURNAL OF MATHEMATICS 2024年第3期21卷 79-79页

作者： Ahmadi, Fatemeh Ahmadi, Parviz Khatibzadeh, Hadi Univ Zanjan Dept Math Univ Blvd Zanjan Iran

The aim of this paper is to study the product of resolvents of a finite number of monotone vector fields on a Hadamard manifold to approximate both the singular points of their sum and a common singular point among them. For the sum of any finitely many maximal monotone vector fields, with some suitable assumptions, it is proved that the obtained sequence of the iterative method is convergent. The paper ends with some examples and applications.

关键词： Hadamard manifold proximal algorithm monotone vector field convergence backward-backward splitting method

来源：评论

学校读者我要写书评

暂无评论

Elaborated-Structure Awareness SAR Imagery Using Hessian-Enhanced TV Regularization

引用

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2023年 61卷 1页

作者： Yang, Lei Gai, Minghui Wang, Tengteng Xing, Mengdao Civil Aviat Univ China Tianjin Key Lab Adv Signal Proc Tianjin 300300 Peoples R China Xidian Univ Natl Key Lab Radar Signal Proc Xian 710071 Peoples R China

Due to the sparse feature enhancement only concentrating on strong scatterers of target of interest, the conventional sparsity-driven synthetic aperture radar (SAR) imagery often encounters the loss of elaborated-structure features, where weak scatterers would be overlapped by the sidelobes of strong scatterers. In this article, an elaborated-structure awareness SAR (ESA-SAR) imaging algorithm is proposed based on Hessian-enhanced total variation (HETV) regularization and cooperation. By encoding the Hessian operator onto the prior of the interested target, the high-order information connected with elaborated-structure features of interests can be captured. Different from the conventional high-order formulation that is projected onto Euclidean norm balls, the proposed algorithm uses the Schatten norm balls as the projection space, where the high-order structure tensor is established, and the elaborated-structure feature can be extracted under the intended convex regularizer. More specifically, the HETV regularizer is analytically solved under the proximal algorithm considering its nondifferentiability. An eigen-soft-thresholding (E-ST) operator is derived, so that a closed-form solution for the elaborated-structure feature can be obtained. Moreover, a synergistic multitask learning framework embedded with the sparse feature enhancement is introduced, in which the elaborated-structure feature can be solved in a cooperative manner. The cooperative learning is guaranteed in terms of both theoretical and practical aspects. Finally, both simulated and raw SAR data are processed to validate the effectiveness of the ESA-SAR algorithm. Comparisons with conventional algorithms examine the superiority of the proposed algorithm.

关键词： Hessian matrix proximal algorithm Schatten norm synthetic aperture radar (SAR) total variation (TV)

来源：评论

学校读者我要写书评

暂无评论

Communication-efficient and Byzantine-robust distributed learning with statistical guarantee

引用

PATTERN RECOGNITION 2023年第1期137卷

作者： Zhou, Xingcai Chang, Le Xu, Pengfei Lv, Shaogao Nanjing Audit Univ Sch Stat & Data Sci Nanjing 211815 Jiangsu Peoples R China

Communication efficiency and robustness are two major issues in modern distributed learning frame-works. This is due to the practical situations where some computing nodes may have limited commu-nication power or may behave adversarial behaviors. To address the two issues simultaneously, this pa-per develops two communication-efficient and robust distributed learning algorithms for convex prob-lems. Our motivation is based on surrogate likelihood framework and the median and trimmed mean operations. Particularly, the proposed algorithms are provably robust against Byzantine failures, and also achieve optimal statistical rates for strong convex losses and convex (non-smooth) penalties. For typical statistical models such as generalized linear models, our results show that statistical errors dominate op-timization errors in finite iterations. Simulated and real data experiments are conducted to demonstrate the numerical performance of our algorithms.(c) 2023 Elsevier Ltd. All rights reserved.

关键词： Distributed learning Byzantine failure Communication efficiency Surrogate likelihood proximal algorithm

来源：评论

学校读者我要写书评

暂无评论

Augmented Lagrangian Tracking for distributed optimization with equality and inequality coupling constraints

引用

AUTOMATICA 2023年第1期157卷

作者： Falsone, Alessandro Prandini, Maria Politecn Milan Dipartimento Elettron Informaz & Bioingn Via Ponzio 34-5 I-20133 Milan Italy

In this paper we propose a novel Augmented Lagrangian Tracking distributed optimization algorithm for solving multi-agent optimization problems where each agent has its own decision variables, cost function and constraint set, and the goal is to minimize the sum of the agents' cost functions subject to local constraints plus some additional coupling constraint involving the decision variables of all the agents. In contrast to alternative approaches available in the literature, the proposed algorithm jointly features a constant penalty parameter, the ability to cope with unbounded local constraint sets, and the ability to handle both affine equality and nonlinear inequality coupling constraints, while requiring convexity only. The effectiveness of the approach is shown first on an artificial example with complexity features that make other state-of-the-art algorithms not applicable and then on a realistic example involving the optimization of the charging schedule of a fleet of electric vehicles.& COPY;2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://***/licenses/by-nc-nd/4.0/).

关键词： Distributed optimization Constraint-coupled optimization proximal algorithm Augmented Lagrangian ADMM

来源：评论

学校读者我要写书评

暂无评论

Online proximal Learning Over Jointly Sparse Multitask Networks With l∞,1 Regularization

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2020年 68卷 6319-6335页

作者： Jin, Danqi Chen, Jie Richard, Cedric Chen, Jingdong Northwestern Polytech Univ Ctr Intelligent Acoust & Immers Commun Sch Marine Sci & Technol Xian 710072 Peoples R China Minist Ind & Informat Technol Key Lab Ocean Acoust & Sensing Xian 710072 Peoples R China Univ Cote Azur CNRS F-06100 Nice France

Modeling relations between local optimum parameter vectors to estimate in multitask networks has attracted much attention over the last years. This work considers a distributed optimization problem with jointly sparse structure among nodes, that is, the local solutions have the same sparse support set. Several mixed norm have been proposed to address the jointly sparse structure in the literature. Among several candidates, the (reweighted) l(infinity,1)-norm is element-wise separable, it is more convenient to evaluate their approximate proximal operators. Thus by introducing a (reweighted) l(infinity,1)-norm penalty term at each node, and using a proximal gradient method to minimize the regularized cost, we devise a proximal multitask diffusion LMS algorithm which can promote joint-sparsity. Analyses are provided to characterize the algorithm behavior in the mean and mean-square sense. Simulation results are presented to show its effectiveness, as well as the accuracy of the theoretical findings.

关键词： Distributed optimization diffusion strategy joint sparsity proximal algorithm stochastic performance (reweighted) l(infinity,1-)norm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：