检索结果-内蒙古大学图书馆

A Priori Estimates For Two-layer Neural Networks

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Weinan, E. Ma, Chao Wu, Lei Department of Mathematics Princeton University United States Program in Applied and Computational Mathematics Princeton University United States Beijing Institute of Big Data Research China School of Mathematical Sciences Peking University China

New estimates for the population risk are established for two-layer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the over-parametrized regime when the network size is much larger than the size of the dataset. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model, in contrast with most existing results which are a posteriori in nature. Using these a priori estimates, we provide a perspective for understanding why two-layer neural networks perform better than kernel *** Codes 41A46, 41A63, 62J02, 65D05 Copyright © 2018, The Authors. All rights reserved.

关键词： Network layers

Geometry based data generation 18

学校读者我要写书评

暂无评论

Geometry based data generation

Proceedings of the 32nd International Conference on Neural Information Processing Systems

作者： Ofir Lindenbaum Jay S. Stanley, III Guy Wolf Smita Krishnaswamy Applied Mathematics Program Yale University New Haven CT Computational Biology & Bioinformatics Program Yale University New Haven CT Departments of Genetics & Computer Science Yale University New Haven CT

We propose a new type of generative model for high-dimensional data that learns a manifold geometry of the data, rather than density, and can generate points evenly along this manifold. This is in contrast to existing generative models that represent data density, and are strongly affected by noise and other artifacts of data collection. We demonstrate how this approach corrects sampling biases and artifacts, thus improves several downstream data analysis tasks, such as clustering and classification. Finally, we demonstrate that this approach is especially useful in biology where, despite the advent of single-cell technologies, rare subpopulations and gene-interaction relationships are affected by biased sampling. We show that SUGAR can generate hypothetical populations, and it is able to reveal intrinsic patterns and mutual-information relationships between genes on a single-cell RNA sequencing dataset of hematopoiesis.

关键词：

Multigoal-oriented optimal control problems with nonlinear PDE constraints

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Endtmayer, B. Langer, U. Neitzel, I. Wick, T. Wollner, W. Doctoral Program on Computational Mathematics Johannes Kepler University Altenbergerstr. 69 LinzA-4040 Austria Johann Radon Institute for Computational and Applied Mathematics Austrian Academy of Sciences Altenbergerstr. 69 LinzA-4040 Austria Institut für Numerische Simulation Endenicher Allee 19b Bonn53115 Germany Institut für Angewandte Mathematik Leibniz Universität Hannover Welfengarten 1 Hannover30167 Germany Leibniz Universität Hannover Germany Technische Universität Darmstadt Fachbereich Mathematik Dolivostr. 15 Darmstadt64293 Germany

In this work, we consider an optimal control problem subject to a nonlinear PDE constraint and apply it to the regularized p-Laplace equation. To this end, a reduced unconstrained optimization problem in terms of the control variable is formulated. Based on the reduced approach, we then derive an a posteriori error representation and mesh adaptivity for multiple quantities of interest. All quantities are combined to one, and then the dual-weighted residual (DWR) method is applied to this combined functional. Furthermore, the estimator allows for balancing the discretization error and the nonlinear iteration error. These developments allow us to formulate an adaptive solution strategy, which is finally substantiated via several numerical examples. Copyright © 2019, The Authors. All rights reserved.

关键词： Optimal control systems

Ten quick tips for deep learning in biology

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Lee, Benjamin D. Gitter, Anthony Greene, Casey S. Raschka, Sebastian Maguire, Finlay Titus, Alexander J. Kessler, Michael D. Lee, Alexandra J. Chevrette, Marc G. Stewart, Paul Allen Britto-Borges, Thiago Cofer, Evan M. Yu, Kun-Hsing Carmona, Juan Jose Fertig, Elana J. Kalinin, Alexandr A. Signal, Beth Lengerich, Benjamin J. Triche, Timothy J. Boca, Simina M. In-Q-Tel Labs School of Engineering and Applied Sciences Harvard University Department of Genetics Harvard Medical School United States Department of Biostatistics and Medical Informatics University of Wisconsin-Madison MadisonWI United States Morgridge Institute for Research MadisonWI United States Department of Systems Pharmacology and Translational Therapeutics Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Department of Biochemistry and Molecular Genetics University of Colorado School of Medicine AuroraCO United States Center for Health AI University of Colorado School of Medicine AuroraCO United States Department of Statistics University of Wisconsin Madison United States Faculty of Computer Science Dalhousie University Canada University of New Hampshire Bioeconomy.XYZ United States Department of Oncology Johns Hopkins University United States Institute for Genome Sciences University of Maryland School of Medicine United States Genomics and Computational Biology Graduate Program University of Pennsylvania United States Department of Systems Pharmacology and Translational Therapeutics University of Pennsylvania United States Wisconsin Institute for Discovery Department of Plant Pathology University of Wisconsin-Madison United States Department of Biostatistics and Bioinformatics Moffitt Cancer Center TampaFL United States Section of Bioinformatics and Systems Cardiology Klaus Tschira Institute for Integrative Computational Cardiology University Hospital Heidelberg Germany University Hospital Heidelberg Germany Lewis-Sigler Institute for Integrative Genomics Princeton University PrincetonNJ United States Graduate Program in Quantitative and Computational Biology Princeton University PrincetonNJ United States Department of Biomedical Informatics Harvard Medical School United States Department of Pathology Brigham and Women's Hospital United States Philips Healthcare CambridgeMA United States Philips Research

Machine learning is a modern approach to problem-solving and task automation. In particular, machine learning is concerned with the development and applications of algorithms that can recognize patterns in data and use them for predictive modeling. Artificial neural networks are a particular class of machine learning algorithms and models that evolved into what is now described as deep learning. Given the computational advances made in the last decade, deep learning can now be applied to massive data sets and in innumerable contexts. Therefore, deep learning has become its own subfield of machine learning. In the context of biological research, it has been increasingly used to derive novel insights from high-dimensional biological data. To make the biological applications of deep learning more accessible to scientists who have some experience with machine learning, we solicited input from a community of researchers with varied biological and deep learning interests. These individuals collaboratively contributed to this manuscript's writing using the GitHub version control platform and the Manubot manuscript generation toolset. The goal was to articulate a practical, accessible, and concise set of guidelines and suggestions to follow when using deep learning. In the course of our discussions, several themes became clear: the importance of understanding and applying machine learning fundamentals as a baseline for utilizing deep learning, the necessity for extensive model comparisons with careful evaluation, and the need for critical thought in interpreting results generated by deep learning, among others. © 2021, CC BY.

关键词： Learning algorithms

Stable super-resolution of images: A theoretical study

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Eftekhari, Armin Bendory, Tamir Tang, Gongguo Institute of Electrical Engineering École Polytechnique Fédérale de Lausanne Program in Applied and Computational Mathematics Princeton University Department of Electrical Engineering Colorado School of Mines

We study the ubiquitous super-resolution problem, in which one aims at localizing positive point sources in an image, blurred by the point spread function of the imaging device. To recover the point sources, we propose to solve a convex feasibility program, which simply finds a nonnegative Borel measure that agrees with the observations collected by the imaging device. In the absence of imaging noise, we show that solving this convex program uniquely retrieves the point sources, provided that the imaging device collects enough observations. This result holds true if the point spread function of the imaging device can be decomposed into horizontal and vertical components, and if the translations of these components form a Chebyshev system, namely, a system of continuous functions that loosely behave like algebraic polynomials. Building upon recent results for one-dimensional signals [1], we prove that this super-resolution algorithm is stable (in the generalized Wasserstein metric) to model mismatch (namely, when the image is not sparse) and to additive imaging noise. In particular, the recovery error depends on the noise level and how well the image can be approximated with well-separated point sources. As an example, we verify these claims for the important case of a Gaussian point spread function. The proofs rely on the construction of novel interpolating polynomials, which are the main technical contribution of this paper, and partially resolve the question raised in [2] about the extension of the standard machinery to higher dimensions. Copyright © 2018, The Authors. All rights reserved.

关键词： Optical transfer function

Mixed mode oscillations and phase locking in coupled FitzHugh-Nagumo model neurons

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Davison, Elizabeth N. Aminzare, Zahra Leonard, Naomi Ehrich Dey, Biswadip Department of Mechanical and Aerospace Engineering Princeton University PrincetonNJ08540 United States Program in Applied and Computational Mathematics Princeton University PrincetonNJ08544 United States

We study the dynamics of a low-dimensional system of coupled model neurons as a step towards understanding the vastly complex network of neurons in the brain. We analyze the bifurcation structure of a system of two model neurons with unidirectional coupling as a function of two physiologically relevant parameters: the external current input only to the first neuron and the strength of the coupling from the first to the second neuron. Leveraging a timescale separation, we prove necessary conditions for multiple timescale phenomena observed in the coupled system, including canard solutions and mixed mode oscillations. For a larger network of model neurons, we present a sufficient condition for phase locking when external inputs are heterogeneous. Finally, we generalize our results to directed trees of model neurons with heterogeneous inputs.34A26, 34C15, 34C60, 34D15, 34E17, 37G05, 37G10, 92B20, 92C20 Copyright © 2018, The Authors. All rights reserved.

关键词： Bifurcation (mathematics)

Estimation in the group action channel

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Abbe, Emmanuel Pereira, João M. Singer, Amit Program in Applied and Computational Mathematics Princeton University PrincetonNJ United States Electrical Engineering Department Princeton University PrincetonNJ United States Department of Mathematics Princeton University PrincetonNJ United States

MSC Codes 94A15, 62B10We analyze the problem of estimating a signal from multiple measurements on a group action channel that linearly transforms a signal by a random group action followed by a fixed projection and additive Gaussian noise. This channel is motivated by applications such as multi-reference alignment and cryoelectron microscopy. We focus on the large noise regime prevalent in these applications. We give a lower bound on the mean square error (MSE) of any asymptotically unbiased estimator of the signal's orbit in terms of the signal's moment tensors, which implies that the MSE is bounded away from 0 when N=σ2dis bounded from above, where N is the number of observations, σ is the noise standard deviation, and d is the so-called moment order cutoff. In contrast, the maximum likelihood estimator is shown to be consistent if N=σ2ddiverges. Copyright © 2018, The Authors. All rights reserved.

关键词： Mean square error

The Athena++ Adaptive Mesh Refinement Framework: Design and Magnetohydrodynamic Solvers

学校读者我要写书评

暂无评论

Astrophysical Journal, Supplement Series 2020年第1期249卷

作者： Stone, James M. Tomida, Kengo White, Christopher J. Felker, Kyle G. Department of Astrophysical Sciences Princeton University Princeton 08544 NJ United States Program in Applied and Computational Mathematics Princeton University Princeton 08544 NJ United States Department of Earth and Space Science Osaka University Toyonaka Osaka 560-0043 Japan Kavli Institute for Theoretical Physics University of California Santa Barbara Santa Barbara 93107 CA United States School of Natural Sciences Institute for Advanced Study Princeton 08544 NJ United States Astronomical Institute Tohoku University Sendai Miyagi 980-8578 Japan Argonne National Laboratory Lemont 60439 IL United States

The design and implementation of a new framework for adaptive mesh refinement calculations are described. It is intended primarily for applications in astrophysical fluid dynamics, but its flexible and modular design enables its use for a wide variety of physics. The framework works with both uniform and nonuniform grids in Cartesian and curvilinear coordinate systems. It adopts a dynamic execution model based on a simple design called a "task list"that improves parallel performance by overlapping communication and computation, simplifies the inclusion of a diverse range of physics, and even enables multiphysics models involving different physics in different regions of the calculation. We describe physics modules implemented in this framework for both nonrelativistic and relativistic magnetohydrodynamics (MHD). These modules adopt mature and robust algorithms originally developed for the Athena MHD code and incorporate new extensions: support for curvilinear coordinates, higher-order time integrators, more realistic physics such as a general equation of state, and diffusion terms that can be integrated with super-time-stepping algorithms. The modules show excellent performance and scaling, with well over 80% parallel efficiency on over half a million threads. The source code has been made publicly available. © 2020. The American Astronomical Society. All rights reserved..

关键词： Astronomy software Magnetohydrodynamics

Communication-computation efficient gradient coding

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Ye, Min Abbe, Emmanuel Department of Electrical Engineering Princeton University PrincetonNJ United States Program in Applied and Computational Mathematics Department of Electrical Engineering Princeton University School of Mathematics Institute for Advanced Study PrincetonNJ08544 United States

This paper develops coding techniques to reduce the running time of distributed learning tasks. It characterizes the fundamental tradeoff to compute gradients (and more generally vector summations) in terms of three parameters: computation load, straggler tolerance and communication cost. It further gives an explicit coding scheme that achieves the optimal tradeoff based on recursive polynomial constructions, coding both across data subsets and vector components. As a result, the proposed scheme allows to minimize the running time for gradient computations. Implementations are made on Amazon EC2 clusters using Python with mpi4py package. Results show that the proposed scheme maintains the same generalization error while reducing the running time by 32% compared to uncoded schemes and 23% compared to prior coded schemes focusing only on stragglers (Tandon et al., ICML 2017). Copyright © 2018, The Authors. All rights reserved.

关键词： Python