检索结果-内蒙古大学图书馆

An inexact augmented lagrangian algorithm for training leaky ReLU neural network with group sparsity

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2023年第1期24卷 10097-10139页

作者： Wei Liu Xin Liu Xiaojun Chen Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing China Department of Applied Mathematics The Hong Kong Polytechnic University Hung Hom Kowloon Hong Kong

The leaky ReLU network with a group sparse regularization term has been widely used in the recent years. However, training such network yields a nonsmooth nonconvex optimization problem and there exists a lack of approaches to compute a stationary point deterministically. In this paper, we first resolve the multi-layer composite term in the original optimization problem by introducing auxiliary variables and additional constraints. We show the new model has a nonempty and bounded solution set and its feasible set satisfies the Mangasarian-Fromovitz constraint qualification. Moreover, we show the relationship between the new model and the original problem. Remarkably, we propose an inexact augmented Lagrangian algorithm for solving the new model, and show the convergence of the algorithm to a KKT point. Numerical experiments demonstrate that our algorithm is more efficient for training sparse leaky ReLU neural networks than some well-known algorithms.

关键词： sparse neural network leaky ReLU group sparsity penalty method inexact augmented Lagrangian method

A tensor factorization model of multilayer network interdependence

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2024年第1期25卷 13587-13640页

作者： Izabel Aguiar Dane Taylor Johan Ugander Institute for Computational and Mathematical Engineering Stanford University Stanford CA School of Computing Department of Mathematics and Statistics University of Wyoming Laramie WY Department of Management Science and Engineering Institute for Computational and Mathematical Engineering Stanford University Stanford CA

Multilayer networks describe the rich ways in which nodes are related by accounting for different relationships in separate layers. These multiple relationships are naturally represented by an adjacency tensor. In this work we study the use of the nonnegative Tucker decomposition (NNTuck) of such tensors under a KL loss as an expressive factor model that naturally generalizes existing stochastic block models of multilayer networks. Quantifying interdependencies between layers can identify redundancies in the structure of a network, indicate relationships between disparate layers, and potentially inform survey instruments for collecting social network data. We propose definitions of layer independence, dependence, and redundancy based on likelihood ratio tests between nested nonnegative Tucker decompositions. Using both synthetic and real-world data, we evaluate the use and interpretation of the NNTuck as a model of multilayer networks. Algorithmically, we show that using expectation maximization (EM) to maximize the log-likelihood under the NNTuck is step-by-step equivalent to tensorial multiplicative updates for the NNTuck under a KL loss, extending a previously known equivalence from nonnegative matrices to nonnegative tensors.

关键词： multilayer networks social networks stochastic blockmodels Tucker decomposition link prediction

Covariance-Based Device Activity Detection with Massive MIMO for Near-Field Correlated Channels

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Ziyue Li, Yang Liu, Ya-Feng Ma, Junjie The State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The Shenzhen Research Institute of Big Data Shenzhen518172 China The School of Science and Engineering The Chinese University of Hong Kong Shenzhen518172 China

This paper studies the device activity detection problem in a massive multiple-input multiple-output (MIMO) system for near-field communications (NFC). In this system, active devices transmit their signature sequences to the base station (BS), which detects the active devices based on the received signal. In this paper, we model the near-field channels as correlated Rician fading channels and formulate the device activity detection problem as a maximum likelihood estimation (MLE) problem. Compared to the traditional uncorrelated channel model, the correlation of channels complicates both algorithm design and theoretical analysis of the MLE problem. On the algorithmic side, we propose two computationally efficient algorithms for solving the MLE problem: an exact coordinate descent (CD) algorithm and an inexact CD algorithm. The exact CD algorithm solves the one-dimensional optimization subproblem exactly using matrix eigenvalue decomposition and polynomial root-finding. By approximating the objective function appropriately, the inexact CD algorithm solves the one-dimensional optimization subproblem inexactly with lower complexity and more robust numerical performance. Additionally, we analyze the detection performance of the MLE problem under correlated channels by comparing it with the case of uncorrelated channels. The analysis shows that when the overall number of devices N is large or the signature sequence length L is small, the detection performance of MLE under correlated channels tends to be better than that under uncorrelated channels. Conversely, when N is small or L is large, MLE performs better under uncorrelated channels than under correlated ones. Simulation results demonstrate the computational efficiency of the proposed algorithms and verify the correctness of the analysis. Copyright © 2024, The Authors. All rights reserved.

关键词： Eigenvalues and eigenfunctions

SOC-MARTNET: A MARTINGALE NEURAL NETWORK FOR THE HAMILTON-JACOBI-BELLMAN EQUATION WITHOUT EXPLICIT infU∈U H IN STOCHASTIC OPTIMAL CONTROLS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cai, Wei Fang, Shuixin Zhou, Tao Department of Mathematics Southern Methodist University DallasTX75275 United States Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China

In this paper, we propose a martingale-based neural network, SOC-MartNet, for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations where no explicit expression is needed for the infimum of the Hamiltonian, infu∈U H(t, x, u, z, p), and stochastic optimal control problems (SOCPs) with controls on both drift and volatility. We reformulate the HJB equations for the value function by training two neural networks, one for the value function and one for the optimal control with the help of two stochastic processes - a Hamiltonian process and a cost process. The control and value networks are trained such that the associated Hamiltonian process is minimized to satisfy the minimum principle of a feedback SOCP, and the cost process becomes a martingale, thus, ensuring the value function network as the solution to the corresponding HJB equation. Moreover, to enforce the martingale property for the cost process, we employ an adversarial network and construct a loss function characterizing the projection property of the conditional expectation condition of the martingale. Numerical results show that the proposed SOC-MartNet is effective and efficient for solving HJB-type equations and SOCPs with a dimension up to 10, 000 in a small number of iteration steps (less than 6000) of training. © 2024, CC BY.

关键词： Hamiltonians

A Natural Deep Ritz Method for Essential Boundary Value Problems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Haijun Zhang, Shuo LSEC Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and System Sciences Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China

Deep neural network approaches show promise in solving partial differential equations. However, unlike traditional numerical methods, they face challenges in enforcing essential boundary conditions. The widely adopted penalty-type methods, for example, offer a straightforward implementation but introduces additional complexity due to the need for hyper-parameter tuning;moreover, the use of a large penalty parameter can lead to artificial extra stiffness, complicating the optimization process. In this paper, we propose a novel, intrinsic approach to impose essential boundary conditions through a framework inspired by intrinsic structures. We demonstrate the effectiveness of this approach using the deep Ritz method applied to Poisson problems, with the potential for extension to more general equations and other deep learning techniques. Numerical results are provided to substantiate the efficiency and robustness of the proposed method. Copyright © 2024, The Authors. All rights reserved.

关键词： Numerical methods

FINITE ELEMENT SPACES BY WHITNEY k-FORMS ON CUBICAL MESHES

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Shuo LSEC Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and System Sciences Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China

Finite element spaces by Whitney k-forms on cubical meshes in Rn are presented. Based on the spaces, compatible discretizations to HΛk problems are provided, and discrete de Rham complexes and commutative diagrams ar... 详细信息

关键词： Mesh generation

Asymptotic SEP Analysis and Optimization of Linear-Quantized Precoding in Massive MIMO Systems

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Zheyu Ma, Junjie Liu, Ya-Feng Swindlehurst, A. Lee The State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The Center for Pervasive Communications and Computing University of California IrvineCA92697 United States

A promising approach to deal with the high hardware cost and energy consumption of massive MIMO transmitters is to use low-resolution digital-to-analog converters (DACs) at each antenna element. This leads to a transmission scheme where the transmitted signals are restricted to a finite set of voltage levels. This paper is concerned with the analysis and optimization of a low-cost quantized precoding strategy, referred to as linear-quantized precoding, for a downlink massive MIMO system under Rayleigh fading. In linear-quantized precoding, the signals are first processed by a linear precoding matrix and subsequently quantized component-wise by the DAC. In this paper, we analyze both the signal-to-interference-plus-noise ratio (SINR) and the symbol error probability (SEP) performances of such linear-quantized precoding schemes in an asymptotic framework where the number of transmit antennas and the number of users grow large with a fixed ratio. Our results provide a rigorous justification for the heuristic arguments based on the Bussgang decomposition that are commonly used in prior works. Based on the asymptotic analysis, we further derive the optimal precoder within a class of linear-quantized precoders that includes several popular precoders as special cases. Our numerical results demonstrate the excellent accuracy of the asymptotic analysis for finite systems and the optimality of the derived precoder. Copyright © 2023, The Authors. All rights reserved.

关键词： MIMO systems

QoS-Aware and Routing-Flexible Network Slicing for Service-Oriented Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Wei-Kun Liu, Ya-Feng Dai, Yu-Hong Luo, Zhi-Quan The School of Mathematics and Statistics Beijing Key Laboratory on MCAACI Beijing Institute of Technology Beijing100081 China The State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen518172 China

In this paper, we consider the network slicing (NS) problem which attempts to map multiple customized virtual network requests (also called services) to a common shared network infrastructure and manage network resources to meet diverse quality of service (QoS) requirements. We propose a mixed-integer nonlinear programming (MINLP) formulation for the considered NS problem that can flexibly route the traffic flow of the services on multiple paths and provide end-to-end delay and reliability guarantees for all services. To overcome the computational difficulty due to the intrinsic nonlinearity in the MINLP formulation, we transform the MINLP formulation into an equivalent mixed-integer linear programming (MILP) formulation and further show that their continuous relaxations are equivalent. In sharp contrast to the continuous relaxation of the MINLP formulation which is a nonconvex nonlinear programming problem, the continuous relaxation of the MILP formulation is a polynomial-time solvable linear programming problem, which significantly facilitates the algorithmic design. Based on the newly proposed MILP formulation, we develop a customized column generation (cCG) algorithm for solving the NS problem. The proposed cCG algorithm is a decomposition-based algorithm and is particularly suitable for solving large-scale NS problems. Numerical results demonstrate the efficacy of the proposed formulations and the proposed cCG algorithm. Copyright © 2024, The Authors. All rights reserved.

关键词： Mixed-integer linear programming

SCALING OPTIMIZED HERMITE APPROXIMATION METHODS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Hao Yu, Haijun School of Mathematical Sciences University of Chinese Academy of Sciences Beijing100049 China NCMIS & LSEC Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Beijing100190 China

Hermite polynomials and functions have extensive applications in scientific and engineering problems. While it is recognized that employing the scaled Hermite functions rather than the standard ones can remarkably enhance the approximation performance, the understanding of the scaling factor remains insufficient. Due to the lack of theoretical analysis, some literature still cast doubts on whether the Hermite spectral method is inferior to other methods. To dispel these doubts, we show in this article that the inefficiency of the Hermite spectral method comes from the imbalance in the decay speed of the objective function within the spatial and frequency domains. Moreover, proper scaling can render the Hermite spectral methods comparable to other methods. To illustrate this idea in more detail, we propose a novel error analysis framework for the scaled Hermite approximation. Taking the L2 projection error as an example, our framework illustrates that there are three different components of errors: the spatial truncation error, the frequency truncation error, and the Hermite spectral approximation error. Through this perspective, finding the optimal scaling factor is equivalent to balancing the spatial and frequency truncation error. As applications, we show that geometric convergence can be recovered by proper scaling for a class of functions. Furthermore, we show that proper scaling can double the convergence order for smooth functions with algebraic decay. The perplexing pre-asymptotic sub-geometric convergence when approximating algebraic decay functions can be perfectly explained by this *** Codes 65N35, 33C45, 42C05 © 2024, CC BY-SA.

关键词： Error analysis