检索结果-内蒙古大学图书馆

Sparse neural additive model: Interpretable deep learning with feature selection via group sparsity

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Shiyun Bu, Zhiqi Chaudhari, Pratik Barnett, Ian J. Department Of Applied Mathematics And Computational Science University Of Pennsylvania United States Department Of Electrical And Systems Engineering University Of Pennsylvania United States Department Of Biostatistics Epidemiology And Informatics University Of Pennsylvania United States

Interpretable machine learning has demonstrated impressive performance while preserving explainability. In particular, neural additive models (NAM) offer the interpretability to the black-box deep learning and achieve state-of-the-art accuracy among the large family of generalized additive models. In order to empower NAM with feature selection and improve the generalization, we propose the sparse neural additive models (SNAM) that employ the group sparsity regularization (e.g. Group LASSO), where each feature is learned by a sub-network whose trainable parameters are clustered as a group. We study the theoretical properties for SNAM with novel techniques to tackle the non-parametric truth, thus extending from classical sparse linear models such as the LASSO, which only works on the parametric truth. Specifically, we show that SNAM with subgradient and proximal gradient descents provably converges to zero training loss as t → ∞, and that the estimation error of SNAM vanishes asymptotically as n → ∞. We also prove that SNAM, similar to LASSO, can have exact support recovery, i.e. perfect feature selection, with appropriate regularization. Moreover, we show that the SNAM can generalize well and preserve the 'identifiability', recovering each feature's effect. We validate our theories via extensive experiments and further testify to the good accuracy and efficiency of SNAM. Copyright © 2022, The Authors. All rights reserved.

关键词： Feature extraction

学校读者我要写书评

暂无评论

A Riemannian ADMM

arXiv 2022年

作者： Li, Jiaxiang Ma, Shiqian Srivastava, Tejes Department of Electrical and Computer Engineering University of Minnesota United States Computational Applied Mathematics and Operations Research Rice University United States Department of Computer Science University of Chicago United States

We consider a class of Riemannian optimization problems where the objective is the sum of a smooth function and a nonsmooth function, considered in the ambient space. This class of problems finds important applications in machine learning and statistics such as the sparse principal component analysis, sparse spectral clustering, and orthogonal dictionary learning. We propose a Riemannian alternating direction method of multipliers (ADMM) to solve this class of problems. Our algorithm adopts easily computable steps in each iteration. The iteration complexity of the proposed algorithm for obtaining an ϵ-stationary point is analyzed under mild assumptions. Existing ADMM for solving nonconvex problems either does not allow nonconvex constraint set, or does not allow nonsmooth objective function. Our algorithm is the first ADMM type algorithm that minimizes a nonsmooth objective over manifold – a particular nonconvex set. Numerical experiments are conducted to demonstrate the advantage of the proposed *** Codes 90C30 Copyright © 2022, The Authors. All rights reserved.

关键词： Principal component analysis

Antithetic Multilevel Methods for Elliptic and Hypo-Elliptic Diffusions with Applications

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Iguchi, Yuga Jasra, Ajay Maama, Mohamed Beskos, Alexandros Department of Statistical Science University College London LondonWC1E 6BT United Kingdom School of Data Science The Chinese University of Hong Kong Shenzhen China Applied Mathematics and Computational Science Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology Thuwal23955-6900 Saudi Arabia

In this paper we present a new antithetic multilevel Monte Carlo (MLMC) method for the estimation of expectations with respect to laws of diffusion processes that can be elliptic or hypo-elliptic. In particular, we consider the case where one has to resort to time discretization of the diffusion and numerical simulation of such schemes. Motivated by recent developments, we introduce a new MLMC estimator of expectations, which does not require simulation of intractable Lévy areas but has a weak error of order 2 and achieves the optimal computational complexity. We then show how this approach can be used in the context of the filtering problem associated to partially observed diffusions with discrete time observations. We illustrate with numerical simulations that our new approaches provide efficiency gains for several problems relative to some existing methods. © 2024, CC BY.

关键词： Diffusion

Bayesian Parameter Inference for Partially Observed SDEs driven by Fractional Brownian Motion

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Maama, Mohamed Jasra, Ajay Ombao, Hernando Applied Mathematics and Computational Science Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology Thuwal23955 Saudi Arabia Statistics Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology Thuwal23955 Saudi Arabia

In this paper we consider Bayesian parameter inference for partially observed fractional Brownian motion (fBM) models. The approach we follow is to time-discretize the hidden process and then to design Markov chain Monte Carlo (MCMC) algorithms to sample from the posterior density on the parameters given data. We rely on a novel representation of the time discretization, which seeks to sample from an approximation of the posterior and then corrects via importance sampling;the approximation reduces the time (in terms of total observation time T) by O(T). This method is extended by using a multilevel MCMC method which can reduce the computational cost to achieve a given mean square error (MSE) versus using a single time discretization. Our methods are illustrated on simulated and real data. Copyright © 2022, The Authors. All rights reserved.

关键词： Mean square error

Exploring 3D Reconstruction with Drone Images: Advances and Challenges in Urban Environments

学校读者我要写书评

暂无评论

Exploring 3D Reconstruction with Drone Images: Advances and ...

Annual Conference of Industrial Electronics Society

作者： Thiago João Miranda Baldivieso Taise Grazielle Da Silva Batista Fabio Suim Chagas Luiz Carlos Pacheco Rodrigues Velho Paulo Fernando Ferreira Rosa Defense Engineering Graduate Program Military Institute of Engineering Rio de Janeiro Brazil State Institute of the Environment Inea/RJ Rio de Janeiro Brazil Department of Science and Industry Systems University of South-Eastern Norway Kongsberg Norway Department of Computer Graphics Institute for Pure and Applied Mathematics Rio de Janeiro Brazil Electrical Engineering Graduate Program Military Institute of Engineering Rio de Janeiro Brazil

ISBN: (数字)9781665464543

ISBN: (纸本)9781665464550

This working-in-progress paper aims to present a three-dimensional reconstruction using aerial images in different environments. The experiments were conducted with aircraft in both external and internal settings, starting with image acquisition, followed by the application of specific photogrammetry software—both commercial and open-source—and concluding with a qualitative evaluation of the results.

关键词： Industrial electronics Three-dimensional displays Urban areas Aircraft Image reconstruction Photogrammetry Drones

PiRL: Participant-Invariant Representation Learning for Healthcare

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cao, Zhaoyang Yu, Han Yang, Huiyuan Sano, Akane Department of Computational Applied Mathematics and Operations Research Rice University HoustonTX77030 United States Department of Electrical and Computer Engineering Rice University HoustonTX77030 United States

Due to individual heterogeneity, performance gaps are observed between generic (one-size-fits-all) models and person-specific models in data-driven health applications. However, in real-world applications, generic models are usually more favorable due to new-user-adaptation issues and system complexities, etc. To improve the performance of the generic model, we propose a representation learning framework that learns participant-invariant representations, named PiRL. The proposed framework utilizes maximum mean discrepancy (MMD) loss and domain-adversarial training to encourage the model to learn participant-invariant representations. Further, a triplet loss, which constrains the model for inter-class alignment of the representations, is utilized to optimize the learned representations for downstream health applications. We evaluated our frameworks on two public datasets related to physical and mental health, for detecting sleep apnea and stress, respectively. As preliminary results, we found the proposed approach shows around a 5% increase in accuracy compared to the baseline. Copyright © 2022, The Authors. All rights reserved.

关键词： Machine learning

Efficiently Escaping Saddle Points in Bilevel Optimization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Huang, Minhui Chen, Xuxing Ji, Kaiyi Ma, Shiqian Lai, Lifeng Department of Electrical and Computer Engineering University of California Davis United States Department of Mathematics University of California Davis United States Department of Computer Science and Engineering University at Buffalo United States Department of Computational Applied Mathematics and Operations Research Rice University United States

Bilevel optimization is one of the fundamental problems in machine learning and optimization. Recent theoretical developments in bilevel optimization focus on finding the first-order stationary points for nonconvex-strongly-convex cases. In this paper, we analyze algorithms that can escape saddle points in nonconvex-strongly-convex bilevel optimization. Specifically, we show that the perturbed approximate implicit differentiation (AID) with a warm start strategy finds ∊-approximate local minimum of bilevel optimization in O͂(∊−2) iterations with high probability. Moreover, we propose an inexact NEgative-curvature-Originated-from-Noise Algorithm (iNEON), a pure first-order algorithm that can escape saddle point and find local minimum of stochastic bilevel optimization. As a by-product, we provide the first nonasymptotic analysis of perturbed multi-step gradient descent ascent (GDmax) algorithm that converges to local minimax point for minimax problems. Copyright © 2022, The Authors. All rights reserved.

关键词： Optimization

Deep Potentials for Materials Science

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wen, Tongqi Zhang, Linfeng Wang, Han Weinan, E. Srolovitz, David J. Department of Mechanical Engineering The University of Hong Kong Hong Kong DP Technology Beijing China AI for Science Institute Beijing China Laboratory of Computational Physics Institute of Applied Physics and Computational Mathematics Beijing China HEDPS CAPT College of Engineering Peking University Beijing China School of Mathematical Sciences Peking University Beijing China Department of Mathematics and Program in Applied and Computational Mathematics Princeton University Princeton United States Shenzhen China

To fill the gap between accurate (and expensive) ab initio calculations and efficient atomistic simulations based on empirical interatomic potentials, a new class of descriptions of atomic interactions has emerged and been widely applied;i.e., machine learning potentials (MLPs). One recently developed type of MLP is the Deep Potential (DP) method. In this review, we provide an introduction to DP methods in computational materials science. The theory underlying the DP method is presented along with a step-by-step introduction to their development and use. We also review materials applications of DPs in a wide range of materials systems. The DP Library provides a platform for the development of DPs and a database of extant DPs. We discuss the accuracy and efficiency of DPs compared with ab initio methods and empirical potentials. Copyright © 2022, The Authors. All rights reserved.

关键词： Calculations

Two-dimensional Closed-Form Analytical Model of Laterally-Excited Film Bulk Acoustic Wave Resonator Multiferroic Antennas

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ippet-Letembet, Louis-Charles Xu, Rui-Fu Jeanty, Robin Yao, Zhi Candler, Robert N. Chen, Shih-Yuan Graduate Institute of Communication Engineering National Taiwan University Taipei10617 Taiwan Applied Mathematics and Computational Research Division Lawrence Berkeley National Laboratory BerkeleyCA94720 United States California NanoSystems Institute Los AngelesCA90095 United States Department of Electrical and Computer Engineering University of California Los AngelesCA90095 United States Department of Electrical Engineering National Taiwan University Taipei10617 Taiwan

To overcome the physical limitations of electrically small antennas, strain-mediated magnetoelectric antennas have been studied experimentally and theoretically. However, current closed-form analytical models include solely one-dimensional approaches. This paper proposes a two-dimensional closed-form analytical model of a laterally-excited multiferroic antenna. To model thickness-shear vibrations, we propose an extended Mason model to two dimensions. This theory solves unidirectionally-coupled elastodynamics-magnetization dynamics-electrodynamics. Without using the infinitesimal dipole approximation, this model provides a closed-form solution of the radiated electromagnetic field with an explicit dependency of the antenna response on the physical parameters, enabling their optimization. Furthermore, we compare the efficiency-bandwidth product of different multiferroic antennas with Chu’s limit in the Akhiezer regime, which delimits the highest achievable mechanical quality factor for acoustic resonators at GHz frequencies. We show that phonon-phonon coupling of Akhiezer damping is most likely to limit multiferroic antenna radiation, not Chu’s limit. Moreover, judicious choice of materials can enhance the maximum efficiency by more than an order of magnitude. Copyright © 2024, The Authors. All rights reserved.

关键词： Analytical models