检索结果-内蒙古大学图书馆

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xiao, Nachuan Hu, Xiaoyin Liu, Xin Toh, Kim-Chuan Institute of Operations Research and Analytics National University of Singapore 3 Research Link Singapore117602 Singapore School of Computer and Computing Science Hangzhou City University Hangzhou310015 China State Key Laboratory of Scientific and Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China Department of Mathematics Institute of Operations Research and Analytics National University of Singapore 10 Lower Kent Ridge Road Singapore119076 Singapore

In this paper, we present a comprehensive study on the convergence properties of Adam-family methods for nonsmooth optimization, especially in the training of nonsmooth neural networks. We introduce a novel two-timescale framework that adopts a two-timescale updating scheme, and prove its convergence properties under mild assumptions. Our proposed framework encompasses various popular Adam-family methods, providing convergence guarantees for these methods in training nonsmooth neural networks. Furthermore, we develop stochastic subgradient methods that incorporate gradient clipping techniques for training nonsmooth neural networks with heavy-tailed noise. Through our framework, we show that our proposed methods converge even when the evaluation noises are only assumed to be integrable. Extensive numerical experiments demonstrate the high efficiency and robustness of our proposed methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems

Tightness and equivalence of semidefinite relaxations for mimo detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jiang, Ruichen Liu, Ya-Feng Bao, Chenglong Jiang, Bo Department of Electronic Engineering Tsinghua University Beijing100084 China State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China Yau Mathematical Sciences Center Tsinghua University Beijing100084 China School of Mathematical Sciences Key Laboratory for NSLSCS of Jiangsu Province Nanjing Normal University Nanjing210023 China

The multiple-input multiple-output (MIMO) detection problem, a fundamental problem in modern digital communications, is to detect a vector of transmitted symbols from the noisy outputs of a fading MIMO channel. The maximum likelihood detector can be formulated as a complex least-squares problem with discrete variables, which is NP-hard in general. Various semidefinite relaxation (SDR) methods have been proposed in the literature to solve the problem due to their polynomial-time worst-case complexity and good detection error rate performance. In this paper, we consider two popular classes of SDR-based detectors and study the conditions under which the SDRs are tight and the relationship between different SDR models. For the enhanced complex and real SDRs proposed recently by Lu et al., we refine their analysis and derive the necessary and sufficient condition for the complex SDR to be tight, as well as a necessary condition for the real SDR to be tight. In contrast, we also show that another SDR proposed by Mobasher et al. is not tight with high probability under mild conditions. Moreover, we establish a general theorem that shows the equivalence between two subsets of positive semidefinite matrices in different dimensions by exploiting a special "separable" structure in the constraints. Our theorem recovers two existing equivalence results of SDRs defined in different settings and has the potential to find other applications due to its *** Codes 90C22, 90C20, 90C46, 90C27 Copyright © 2021, The Authors. All rights reserved.

关键词： Polynomial approximation

A Consistent and Conservative Axisymmetric Lattice Boltzmann Method for Multiphase Electro-Hydrodynamic Flows

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Liu, Xi Chai, Zhenhua Shi, Baochang Yuan, Xiaolei School of Mathematics and Statistics Huazhong University of Science and Technology Wuhan430074 China Institute of Interdisciplinary Research for Mathematics and Applied Science Huazhong University of Science and Technology Wuhan430074 China Hubei Key Laboratory of Engineering Modeling and Scientific Computing Huazhong University of Science and Technology Wuhan430074 China College of Mathematics and Information Science Hebei University Baoding071002 China

In this paper, a consistent and conservative mathematical model is first developed for multiphase electro-hydrodynamic (EHD) flows, which has some distinct features in the volume conservation, the consistency of reduction, and the consistency of mass-momentum transport. Then a two-relaxation-time axisymmetric lattice Boltzmann (LB) method is proposed for multiphase EHD flows, and the present LB method consists of three LB models, which are used to solve the axisymmetric reduction-consistent and conservative Allen-Cahn equation for the multiphase field, the axisymmetric Poisson equation for the electric potential, and the axisymmetric consistent and conservative Navier-Stokes equations for the flow field. Owing to the reduction-consistent property, the present LB method can be used to handle a set of M (1≤M≤N) EHD fluids in N-phase system. To test the capacity of the LB method, the deformations of a single two-phase droplet and a ternary-phase compound droplet under a uniform electric field are considered. It is found that different from a single droplet with two deformation modes, the compound droplet has four deformation modes, and additionally, the effects of the electric strength, the conductivity ratio and the permittivity ratio on the compound droplet are also investigated. Finally, the compound droplet in the quaternary-phase system is also explored, and there are eight deformation modes that have not been reported in the available literature. © 2023, The Authors. All rights reserved.

关键词： Navier Stokes equations

Mapping Biological Neuron Dynamics into an Interpretable Two-layer Artificial Neural Network

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ma, Jingyang Li, Songting Zhou, Douglas School of Mathematical Sciences Shanghai Jiao Tong University Shanghai200240 China Institute of Natural Sciences Shanghai Jiao Tong University Shanghai200240 China Ministry of Education Key Laboratory of Scientific and Engineering Computing Shanghai Jiao Tong University Shanghai200240 China Shanghai Frontier Science Center of Modern Analysis Shanghai Jiao Tong University Shanghai200240 China

Dendrites are crucial structures for computation of an individual neuron. It has been shown that the dynamics of a biological neuron with dendrites can be approximated by artificial neural networks (ANN) with deep structure. However, it remains unclear whether a neuron can be further captured by a simple, biologically plausible ANN. In this work, we develop a two-layer ANN, named as dendritic bilinear neural network (DBNN), to accurately predict both the sub-threshold voltage and spike time at the soma of biological neuron models with dendritic structure. Our DBNN is found to be interpretable and well captures the dendritic integration process of biological neurons including a bilinear rule revealed in previous works. In addition, we show DBNN is capable of performing diverse tasks including direction selectivity, coincidence detection, and image classification. Our work proposes a biologically interpretable ANN that characterizes the computation of biological neurons, which can be potentially implemented in the deep learning framework to improve computational ability. Copyright © 2023, The Authors. All rights reserved.

关键词： Neurons

Dendritic Integration Based Quadratic Neural Networks Outperform Traditional Aritificial Ones

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Chongming Li, Songting Zhou, Douglas School of Mathematical Sciences Shanghai Jiao Tong University Shanghai200240 China Institute of Natural Sciences Shanghai Jiao Tong University Shanghai200240 China Ministry of Education Key Laboratory of Scientific and Engineering Computing Shanghai Jiao Tong University Shanghai200240 China Shanghai Frontier Science Center of Modern Analysis Shanghai Jiao Tong University Shanghai200240 China

Incorporating biological neuronal properties into Artificial Neural Networks (ANNs) to enhance computational capabilities poses a formidable challenge in the field of machine learning. Inspired by recent findings indicating that dendrites adhere to quadratic integration rules for synaptic inputs, we propose a novel ANN model, Dendritic Integration-Based Quadratic Neural Network (DIQNN). This model shows superior performance over traditional ANNs in a variety of classification tasks. To reduce the computational cost of DIQNN, we introduce the Low-Rank DIQNN, while we find it can retain the performance of the original DIQNN. We further propose a margin to characterize the generalization error and theoretically prove this margin will increase monotonically during training. And we show the consistency between generalization and our margin using numerical experiments. Finally, by integrating this margin into the loss function, the change of test accuracy is indeed accelerated. Our work contributes a novel, brain-inspired ANN model that surpasses traditional ANNs and provides a theoretical framework to analyze the generalization error in classification tasks. Copyright © 2023, The Authors. All rights reserved.

关键词： Integration

ON THE CHARACTERIZATION, EXISTENCE AND UNIQUENESS OF STEADY SOLUTIONS TO THE HYDROSTATIC EULER EQUATIONS IN A NOZZLE

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Leung, Wang Shing Wong, Tak Kwong Xie, Chunjing Department of Mathematics Temple University 1805 N. Broad Street PhiladelphiaPA19122 United States Department of mathematics The University of Hong Kong Pokfulam Hong Kong School of mathematical Sciences Institute of Natural Sciences Ministry of Education Key Laboratory of Scientific and Engineering Computing CMA-Shanghai Shanghai Jiao Tong University 800 Dongchuan Road Shanghai China

Incompressible Euler flows in narrow domains, in which the horizontal length scale is much larger than other scales, play an important role in applications, and their leading-order behavior can be described by the hydrostatic Euler equations. In this paper, we show that steady solutions of the hydrostatic Euler equations in an infinite strip strictly away from stagnation must be shear flows. Furthermore, we prove the existence, uniqueness, and asymptotic behavior of global steady solutions to the hydrostatic Euler equations in general nozzles. In terms of stream function formulation, the hydrostatic Euler equations can be written as a degenerate elliptic equation, for which the Liouville type theorem in a strip is a consequence of the analysis for the second order ordinary differential equation (ODE). The analysis on the associated ODE also helps determine the far field behavior of solutions in general nozzles, which plays an important role in guaranteeing the equivalence of stream function formulation. One of the key ingredients for the analysis on flows in a general nozzle is a new transformation, which combines a change of variable and an Euler-Lagrange transformation. With the aid of this new transformation, the solutions in the new coordinates enjoy explicit representations so that the regularity with respect to the horizontal variable can be gained in a clear way. Copyright © 2023, The Authors. All rights reserved.

关键词： Nozzles

Unifying Farkas lemma and S-lemma: New theory and applications in nonquadratic nonconvex optimization

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yang, Meijia Xia, Yong Wang, Shu School of Mathematics and Physics University of Science and Technology Beijing Beijing100083 China State Key Laboratory of Software Development Environment LMIB of the Ministry of Education School of Mathematics and System Sciences Beihang University Beijing100191 China Institute of Computational Mathematics and Scientific/Engineering Computing State Key Laboratory of Scientific and Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China

We unify nonlinear Farkas lemma and S-lemma to a generalized alternative theorem for nonlinear nonconvex system. It provides fruitful applications in globally solving nonconvex non-quadratic optimization problems via ... 详细信息

关键词： Quadratic programming

Splitting Method for Stochastic Navier-Stokes Equations

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhu, Jie Zhu, Yujun Ming, Ju Gunzburger, Max D. School of Mathematics and Statistics Huazhong University of Science and Technology Hubei Wuhan430074 China Hubei Key Laboratory of Engineering Modeling and Scientific Computing Huazhong University of Science and Technology Hubei Wuhan430074 China Department of Scientific Computing Florida State University TallahasseeFL32304 United States

This paper investigates the two-dimensional stochastic steady-state Navier-Stokes(NS) equations with additive random noise. We introduce an innovative splitting method that decomposes the stochastic NS equations into a deterministic NS component and a stochastic equation. We rigorously analyze the proposed splitting method from the perspectives of equivalence, stability, existence and uniqueness of the solution. We also propose a modified splitting scheme, which simplified the stochastic equation by omitting its nonlinear terms. A detailed analysis of the solution properties for this modified approach is provided. Additionally, we discuss the statistical errors with both the original splitting format and the modified scheme. Our theoretical and numerical studies demonstrate that the equivalent splitting scheme exhibits significantly enhanced stability compared to the original stochastic NS equations, enabling more effective handling of nonlinear characteristics. Several numerical experiments were performed to compare the statistical errors of the splitting method and the modified splitting method. Notably, the deterministic NS equation in the splitting method does not require repeated solving, and the stochastic equation in the modified scheme is free of nonlinear terms. These features make the modified splitting method particularly advantageous for large-scale computations, as it significantly improves computational efficiency without compromising accuracy. Copyright © 2025, The Authors. All rights reserved.

关键词： Navier Stokes equations

LIOUVILLE-TYPE THEOREMS FOR STEADY NAVIER-STOKES SYSTEM UNDER HELICAL SYMMETRY OR NAVIER BOUNDARY CONDITIONS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Han, Jingwen Wang, Yun Xie, Chunjing School of Mathematical Sciences Shanghai Jiao Tong University 800 Dongchuan Road Shanghai China School of Mathematical Sciences Center for dynamical systems and differential equations Soochow University Suzhou China School of mathematical Sciences Institute of Natural Sciences Ministry of Education Key Laboratory of Scientific and Engineering Computing CMA-Shanghai Shanghai Jiao Tong University 800 Dongchuan Road Shanghai China

In this paper, the Liouville-type theorems for the steady Navier-Stokes system are investigated. First, we prove that any bounded smooth helically symmetric solution in R3 must be a constant vector. Second, for steady Navier-Stokes system in a slab supplemented with Navier boundary conditions, we prove that any bounded smooth solution must be zero if either the swirl or radial velocity is axisymmetric, or rur decays to zero as r tends to infinity. Finally, when the velocity is not big in L∞-space, the general three-dimensional steady Navier-Stokes flow in a slab with the Navier boundary conditions must be a Poiseuille type flow. The key idea of the proof is to establish Saint-Venant type estimates that characterize the growth of Dirichlet integral of nontrivial solutions. Copyright © 2023, The Authors. All rights reserved.

关键词： Boundary conditions