检索结果-内蒙古大学图书馆

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yuan, Ya-Xiang Zhang, Yi Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China

The proximal point algorithm plays a central role in non-smooth convex programming. The Augmented Lagrangian Method, one of the most famous optimization algorithms, has been found to be closely related to the proximal point algorithm. Due to its importance, accelerated variants of the proximal point algorithm have received considerable attention. In this paper, we first study an Ordinary Differential Equation (ODE) system, which provides valuable insights into proving the convergence rate of the desired algorithm. Using the Lyapunov function technique, we establish the convergence rate of the ODE system. Next, we apply the Symplectic Euler Method to discretize the ODE system to derive a new proximal point algorithm, called the Symplectic Proximal Point Algorithm (SPPA). By utilizing the proof techniques developed for the ODE system, we demonstrate the convergence rate of the SPPA. Additionally, it is shown that existing accelerated proximal point algorithm can be considered a special case of the SPPA in a specific manner. Furthermore, under several additional assumptions, we prove that the SPPA exhibits a finer convergence *** Codes 49J52, 65B99, 65K10, 65K15, 68Q25, 90C25, 90C26 © 2024, CC BY.

关键词： Lyapunov functions

SYMPLECTIC EXTRA-GRADIENT TYPE METHOD FOR SOLVING GENERAL NON-MONOTONE INCLUSION PROBLEM

学校读者我要写书评

暂无评论

arXiv 2024年

In recent years, accelerated extra-gradient methods have attracted much attention by researchers, for solving monotone inclusion problems. A limitation of most current accelerated extra-gradient methods lies in their direct utilization of the initial point, which can potentially decelerate numerical convergence rate. In this work, we present a new accelerated extra-gradient method, by utilizing the symplectic acceleration technique. We establish the inverse of quadratic convergence rate by employing the Lyapunov function technique. Also, we demonstrate a faster inverse of quadratic convergence rate alongside its weak convergence property under stronger assumptions. To improve practical efficiency, we introduce a line search technique for our symplectic extra-gradient method. Theoretically, we prove the convergence of the symplectic extra-gradient method with line search. Numerical tests show that this adaptation exhibits faster convergence rates in practice compared to several existing extra-gradient type *** Codes 47J20, 47H05, 65K10, 65K15, 65Y20, 90C30, 90C52 © 2024, CC BY.

关键词： Lyapunov functions

An inexact augmented lagrangian algorithm for training leaky ReLU neural network with group sparsity

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2023年第1期24卷 10097-10139页

作者： Wei Liu Xin Liu Xiaojun Chen Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing China Department of Applied Mathematics The Hong Kong Polytechnic University Hung Hom Kowloon Hong Kong

The leaky ReLU network with a group sparse regularization term has been widely used in the recent years. However, training such network yields a nonsmooth nonconvex optimization problem and there exists a lack of approaches to compute a stationary point deterministically. In this paper, we first resolve the multi-layer composite term in the original optimization problem by introducing auxiliary variables and additional constraints. We show the new model has a nonempty and bounded solution set and its feasible set satisfies the Mangasarian-Fromovitz constraint qualification. Moreover, we show the relationship between the new model and the original problem. Remarkably, we propose an inexact augmented Lagrangian algorithm for solving the new model, and show the convergence of the algorithm to a KKT point. Numerical experiments demonstrate that our algorithm is more efficient for training sparse leaky ReLU neural networks than some well-known algorithms.

关键词： sparse neural network leaky ReLU group sparsity penalty method inexact augmented Lagrangian method

A tensor factorization model of multilayer network interdependence

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2024年第1期25卷 13587-13640页

作者： Izabel Aguiar Dane Taylor Johan Ugander Institute for Computational and Mathematical Engineering Stanford University Stanford CA School of Computing Department of Mathematics and Statistics University of Wyoming Laramie WY Department of Management Science and Engineering Institute for Computational and Mathematical Engineering Stanford University Stanford CA

Multilayer networks describe the rich ways in which nodes are related by accounting for different relationships in separate layers. These multiple relationships are naturally represented by an adjacency tensor. In this work we study the use of the nonnegative Tucker decomposition (NNTuck) of such tensors under a KL loss as an expressive factor model that naturally generalizes existing stochastic block models of multilayer networks. Quantifying interdependencies between layers can identify redundancies in the structure of a network, indicate relationships between disparate layers, and potentially inform survey instruments for collecting social network data. We propose definitions of layer independence, dependence, and redundancy based on likelihood ratio tests between nested nonnegative Tucker decompositions. Using both synthetic and real-world data, we evaluate the use and interpretation of the NNTuck as a model of multilayer networks. Algorithmically, we show that using expectation maximization (EM) to maximize the log-likelihood under the NNTuck is step-by-step equivalent to tensorial multiplicative updates for the NNTuck under a KL loss, extending a previously known equivalence from nonnegative matrices to nonnegative tensors.

关键词： multilayer networks social networks stochastic blockmodels Tucker decomposition link prediction

Covariance-Based Device Activity Detection with Massive MIMO for Near-Field Correlated Channels

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Ziyue Li, Yang Liu, Ya-Feng Ma, Junjie The State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The Shenzhen Research Institute of Big Data Shenzhen518172 China The School of Science and Engineering The Chinese University of Hong Kong Shenzhen518172 China

This paper studies the device activity detection problem in a massive multiple-input multiple-output (MIMO) system for near-field communications (NFC). In this system, active devices transmit their signature sequences to the base station (BS), which detects the active devices based on the received signal. In this paper, we model the near-field channels as correlated Rician fading channels and formulate the device activity detection problem as a maximum likelihood estimation (MLE) problem. Compared to the traditional uncorrelated channel model, the correlation of channels complicates both algorithm design and theoretical analysis of the MLE problem. On the algorithmic side, we propose two computationally efficient algorithms for solving the MLE problem: an exact coordinate descent (CD) algorithm and an inexact CD algorithm. The exact CD algorithm solves the one-dimensional optimization subproblem exactly using matrix eigenvalue decomposition and polynomial root-finding. By approximating the objective function appropriately, the inexact CD algorithm solves the one-dimensional optimization subproblem inexactly with lower complexity and more robust numerical performance. Additionally, we analyze the detection performance of the MLE problem under correlated channels by comparing it with the case of uncorrelated channels. The analysis shows that when the overall number of devices N is large or the signature sequence length L is small, the detection performance of MLE under correlated channels tends to be better than that under uncorrelated channels. Conversely, when N is small or L is large, MLE performs better under uncorrelated channels than under correlated ones. Simulation results demonstrate the computational efficiency of the proposed algorithms and verify the correctness of the analysis. Copyright © 2024, The Authors. All rights reserved.

关键词： Eigenvalues and eigenfunctions

SOC-MARTNET: A MARTINGALE NEURAL NETWORK FOR THE HAMILTON-JACOBI-BELLMAN EQUATION WITHOUT EXPLICIT infU∈U H IN STOCHASTIC OPTIMAL CONTROLS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cai, Wei Fang, Shuixin Zhou, Tao Department of Mathematics Southern Methodist University DallasTX75275 United States Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China

In this paper, we propose a martingale-based neural network, SOC-MartNet, for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations where no explicit expression is needed for the infimum of the Hamiltonian, infu∈U H(t, x, u, z, p), and stochastic optimal control problems (SOCPs) with controls on both drift and volatility. We reformulate the HJB equations for the value function by training two neural networks, one for the value function and one for the optimal control with the help of two stochastic processes - a Hamiltonian process and a cost process. The control and value networks are trained such that the associated Hamiltonian process is minimized to satisfy the minimum principle of a feedback SOCP, and the cost process becomes a martingale, thus, ensuring the value function network as the solution to the corresponding HJB equation. Moreover, to enforce the martingale property for the cost process, we employ an adversarial network and construct a loss function characterizing the projection property of the conditional expectation condition of the martingale. Numerical results show that the proposed SOC-MartNet is effective and efficient for solving HJB-type equations and SOCPs with a dimension up to 10, 000 in a small number of iteration steps (less than 6000) of training. © 2024, CC BY.

关键词： Hamiltonians

A Natural Deep Ritz Method for Essential Boundary Value Problems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Haijun Zhang, Shuo LSEC Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and System Sciences Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China

Deep neural network approaches show promise in solving partial differential equations. However, unlike traditional numerical methods, they face challenges in enforcing essential boundary conditions. The widely adopted penalty-type methods, for example, offer a straightforward implementation but introduces additional complexity due to the need for hyper-parameter tuning;moreover, the use of a large penalty parameter can lead to artificial extra stiffness, complicating the optimization process. In this paper, we propose a novel, intrinsic approach to impose essential boundary conditions through a framework inspired by intrinsic structures. We demonstrate the effectiveness of this approach using the deep Ritz method applied to Poisson problems, with the potential for extension to more general equations and other deep learning techniques. Numerical results are provided to substantiate the efficiency and robustness of the proposed method. Copyright © 2024, The Authors. All rights reserved.

关键词： Numerical methods

FINITE ELEMENT SPACES BY WHITNEY k-FORMS ON CUBICAL MESHES

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Shuo LSEC Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and System Sciences Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China

Finite element spaces by Whitney k-forms on cubical meshes in Rn are presented. Based on the spaces, compatible discretizations to HΛk problems are provided, and discrete de Rham complexes and commutative diagrams ar... 详细信息

关键词： Mesh generation

Asymptotic SEP Analysis and Optimization of Linear-Quantized Precoding in Massive MIMO Systems

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Zheyu Ma, Junjie Liu, Ya-Feng Swindlehurst, A. Lee The State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The Center for Pervasive Communications and Computing University of California IrvineCA92697 United States

A promising approach to deal with the high hardware cost and energy consumption of massive MIMO transmitters is to use low-resolution digital-to-analog converters (DACs) at each antenna element. This leads to a transmission scheme where the transmitted signals are restricted to a finite set of voltage levels. This paper is concerned with the analysis and optimization of a low-cost quantized precoding strategy, referred to as linear-quantized precoding, for a downlink massive MIMO system under Rayleigh fading. In linear-quantized precoding, the signals are first processed by a linear precoding matrix and subsequently quantized component-wise by the DAC. In this paper, we analyze both the signal-to-interference-plus-noise ratio (SINR) and the symbol error probability (SEP) performances of such linear-quantized precoding schemes in an asymptotic framework where the number of transmit antennas and the number of users grow large with a fixed ratio. Our results provide a rigorous justification for the heuristic arguments based on the Bussgang decomposition that are commonly used in prior works. Based on the asymptotic analysis, we further derive the optimal precoder within a class of linear-quantized precoders that includes several popular precoders as special cases. Our numerical results demonstrate the excellent accuracy of the asymptotic analysis for finite systems and the optimality of the derived precoder. Copyright © 2023, The Authors. All rights reserved.

关键词： MIMO systems