检索结果-内蒙古大学图书馆

Nullspacenet: Nullspace convoluional neural network with differentiable loss function

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Abdelpakey, Mohamed H. Shehata, Mohamed S. University of British Columbia Department of Computer Science Math Physics and Statistics KelownaBC Canada Memorial University of Newfoundland Computer Engineering and Applied Science St. John’sNL Canada

We propose NullSpaceNet, a novel network that maps from the pixel level input to a joint-nullspace (as opposed to the traditional feature space), where the newly learned joint-nullspace features have clearer interpretation and are more separable. NullSpaceNet ensures that all inputs from the same class are collapsed into one point in this new joint-nullspace, and the different classes are collapsed into different points with high separation margins. Moreover, a novel differentiable loss function is proposed that has a closed-form solution with no free-parameters. NullSpaceNet exhibits superior performance when tested against VGG16 with fully-connected layer over 4 different datasets, with accuracy gain of up to 4.55%, a reduction in learnable parameters from ∼ 135Mto ∼ 19M, and reduction in inference time of ∼ 99% in favor of NullSpaceNet. This means that NullSpaceNet needs less than 1% of the time it takes a traditional CNN to classify a batch of images with better accuracy. Copyright © 2020, The Authors. All rights reserved.

关键词： Machine learning

On Biased Compression for Distributed Learning

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Beznosikov, Aleksandr Horváth, Samuel Richtárik, Peter Safaryan, Mher Computer Electrical and Math. Sciences and Engineering Division King Abdullah University of Science and Technology Thuwal23955 Saudi Arabia Skolkovo Institute of Science and Technology Moscow121205 Russia School of Applied Mathematics and Informatics Moscow Institute of Physics and Technology Moscow141701 Russia

In the last few years, various communication compression techniques have emerged as an indispensable tool helping to alleviate the communication bottleneck in distributed learning. However, despite the fact biased compressors often show superior performance in practice when compared to the much more studied and understood unbiased compressors, very little is known about them. In this work we study three classes of biased compression operators, two of which are new, and their performance when applied to (stochastic) gradient descent and distributed (stochastic) gradient descent. We show for the first time that biased compressors can lead to linear convergence rates both in the single node and distributed settings. We prove that distributed compressed SGD method, employed with error feedback mechanism, enjoys the ergodic rate O ( δLexp[−µKδL ] + (CKµ+δD) ) , where δ ≥ 1 is a compression parameter which grows when more compression is applied, L and µ are the smoothness and strong convexity constants, C captures stochastic gradient noise (C = 0 if full gradients are computed on each node) and D captures the variance of the gradients at the optimum (D = 0 for over-parameterized models). Further, via a theoretical study of several synthetic and empirical distributions of communicated gradients, we shed light on why and by how much biased compressors outperform their unbiased variants. Finally, we propose several new biased compressors with promising theoretical guarantees and practical performance. Copyright © 2020, The Authors. All rights reserved.

关键词： Stochastic systems

Noisy circumnutations facilitate self-organized shade avoidance in sunflowers

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Nguyen, Chantal Dromi, Imri Kempinski, Aharon Gall, Gabriella E.C. Peleg, Orit Meroz, Yasmine BioFrontiers Institute University of Colorado BoulderCO United States School of Plant Science and Food Security Tel Aviv University Tel Aviv Israel Zukunftskolleg Department of Biology University of Konstanz Konstanz Germany Centre for the Advanced Study of Collective Behaviour Konstanz Germany Department of Computer Science University of Colorado BoulderCO United States Department of Physics University of Colorado BoulderCO United States Department of Applied Math University of Colorado BoulderCO United States Department of Ecology and Evolutionary Biology University of Colorado BoulderCO United States Santa Fe Institute Santa FeNM United States

Circumnutations are widespread in plants and typically associated with exploratory movements, however a quantitative understanding of their role remains elusive. In this study we report, for the first time, the role of noisy circumnutations in facilitating an optimal growth pattern within a crowded group of mutually shading plants. We revisit the problem of self-organization observed for sunflowers, mediated by shade response interactions. Our analysis reveals that circumnutation movements conform to a bounded random walk characterized by a remarkably broad distribution of velocities, covering three orders of magnitude. In motile animal systems such wide distributions of movement velocities are frequently identified with enhancement of behavioral processes, suggesting that circumnutations may serve as a source of functional noise. To test our hypothesis, we developed a parsimonious model of interacting growing disks, informed by experiments, successfully capturing the characteristic dynamics of individual and multiple interacting plants. Employing our simulation framework we examine the role of circumnutations in the system, and find that the observed breadth of the velocity distribution confers advantageous effects by facilitating exploration of potential configurations, leading to an optimized arrangement with minimal shading. These findings represent the first report of functional noise in plant movements, and establishes a theoretical foundation for investigating how plants navigate their environment by employing computational processes such as task-oriented processes, optimization, and active sensing. (218 words) Copyright © 2022, The Authors. All rights reserved.

关键词： Velocity distribution

Continuous and Discrete Data Assimilation with Noisy Observations for the Rayleigh-Benard Convection: A Computational Study

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hammoud, Mohamad Abed El Rahman Le Maître, Olivier Titi, Edriss S. Hoteit, Ibrahim Knio, Omar Division of Physical Sciences and Engineering King Abdullah University of Science and Technology Thuwal23955 Saudi Arabia Centre de Mathématiques Appliquées CNRS and Inria Ecole Polytechnique Palaiseau91120 France Department of Applied Mathematics and Theoretical Physics University of Cambridge CambridgeCB3 0WA United Kingdom Department of Mathematics Texas A & M University College StationTX77843 United States Division of Computer Electrical and Mathematical Sciences and Engineering King Abdullah University of Science and Technology Thuwal23955 Saudi Arabia

Obtaining accurate high-resolution representations of model outputs is essential to describe the system dynamics. In general, however, only spatially- and temporally-coarse observations of the system states are available. These observations can also be corrupted by noise. Downscaling is a process/scheme in which one uses coarse scale observations to reconstruct the high-resolution solution of the system states. Continuous Data Assimilation (CDA) is a recently introduced downscaling algorithm that constructs an increasingly accurate representation of the system states by continuously nudging the large scales using the coarse observations. We introduce a Discrete Data Assimilation (DDA) algorithm as a downscaling algorithm based on CDA with discrete-in-time nudging. We then investigate the performance of the CDA and DDA algorithms for downscaling noisy observations of the Rayleigh-Bénard convection system in the chaotic regime. In this computational study, a set of noisy observations was generated by perturbing a reference solution with Gaussian noise before downscaling them. The downscaled fields are then assessed using various error- and ensemble-based skill scores. The CDA solution was shown to converge towards the reference solution faster than that of DDA but at the cost of a higher asymptotic error. The numerical results also suggest a quadratic relationship between the 2 error and the noise level for both CDA and DDA. Cubic and quadratic dependences of the DDA and CDA expected errors on the spatial resolution of the observations were obtained, respectively. © 2022, CC BY.

关键词： Errors

Fusion hierarchies, T -systems and Y -systems for the dilute A(2)2 loop models on a strip

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Boileau, Florence Morin-Duchesne, Alexi Saint-Aubin, Yvan Département de physique Université de Montréal Montréal QCH3C 3J7 Canada Department of Applied Mathematics Computer Science and Statistics Ghent University Ghent9000 Belgium Département de mathématiques et de statistique Université de Montréal MontréalQCH3C 3J7 Canada

We study the dilute A(2)2 loop models on the geometry of a strip of width N . Two families of boundary conditions are known to satisfy the boundary Yang-Baxter equation. Fixing the boundary condition on the two ends of the strip leads to four models. We construct the fusion hierarchy of commuting transfer matrices for the model as well as its T - and Y -systems, for these four boundary conditions and with a generic crossing parameter λ. For λ/π rational and thus q = −e4iλ a root of unity, we prove a linear relation satisfied by the fused transfer matrices that closes the fusion hierarchy into a finite system. The fusion relations allow us to compute the two leading terms in the large-N expansion of the free energy, namely the bulk and boundary free energies. These are found to be in agreement with numerical data obtained for small N . The present work complements a previous study (A. Morin-Duchesne, P.A. Pearce, J. Stat. Mech. (2019)) that investigated the dilute A(2)2 loop models with periodic boundary conditions. Copyright © 2022, The Authors. All rights reserved.

关键词： Boundary conditions

Deep compositional spatial models

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zammit-Mangion, Andrew Ng, Tin Lok James Vu, Quan Filippone, Maurizio School of Math. and Applied Stat. University of Wollongong Australia Department of Data Science EURECOM France

Nonstationary, anisotropic spatial processes are often used when modelling, analysing and predicting complex environmental phenomena. One such class of processes considers a stationary, isotropic process on a warped spatial domain. The warping function is generally difficult to fit and not constrained to be bijective, often resulting in ‘space-folding.’ Here, we propose modelling a bijective warping function through a composition of multiple elemental bijective functions in a deep-learning framework. We consider two cases;first, when these functions are known up to some weights that need to be estimated, and, second, when the weights in each layer are random. Inspired by recent methodological and technological advances in deep learning and deep Gaussian processes, we employ approximate Bayesian methods to make inference with these models using graphical processing units. Through simulation studies in one and two dimensions we show that the deep compositional spatial models are quick to fit, and are able to provide better predictions and uncertainty quantification than other deep stochastic models of similar complexity. We also show their remarkable capacity to model highly nonstationary, anisotropic spatial data using radiances from the MODIS instrument aboard the Aqua satellite. Copyright © 2019, The Authors. All rights reserved.

关键词： Stochastic systems

A deterministic Susceptible–Infected–Recovered model for studying the impact of media on epidemic dynamics

学校读者我要写书评

暂无评论

Healthcare Analytics 2023年 3卷 100189-100189页

作者： Fome, Anna D. Rwezaura, Herieth Diagne, Mamadou L. Collinson, Shannon Tchuenche, Jean M. Department of Economics Mathematics and Statistics Jordan University College Morogoro Tanzania Mathematics Department University of Dar es Salaam P.O. Box 35062 Dar es Salaam Tanzania Département de Mathématiques UFR des Sciences et Technologies Université de Thiès BP 967 Thiès Senegal Centre for Disease Modelling York University Canada School of Computational and Communication Sciences and Engineering Nelson Mandela African Institution of Science and Technology P.O. Box 447 Arusha Tanzania School of Computer Science and Applied Mathematics University of the Witwatersrand Private Bag 3 Wits 2050 Johannesburg South Africa

It is important to understand how the provision of information influences individuals’ behavior and its impact on disease transmission dynamics. We formulate a simple deterministic Susceptible–Infected–Recovered (SIR) model with vital dynamics and incorporate, one at a time, four media functions used in previous studies to mimic media reporting of a disease outbreak to investigate their impact on the dynamics of an epidemic. The most sensitive model parameter for the epidemic outcome at initial disease transmission and disease prevalence are obtained through normalized forward-sensitivity analyses. Media reporting does not impact initial disease transmission but affects the effective reproduction number as the media coverage begins. Numerical results show that the epidemic will affect a large proportion of the population with no media functions. Still, the epidemic time line may be short. In contrast, the key epidemiological quantities (such as age at infection, peak magnitude, peak time, end of the epidemic, and final epidemic size) vary depending on the media function used. The choice of the media function in a pandemic is critical. The media reporting on the spread of disease during pandemics should be done with care since it could counter-intuitively lead to under or overestimating the disease outbreak. © 2023 The Author(s)

关键词： Deterministic model Disease outbreak Epidemic dynamics Media coverage Sensitivity analysis Susceptible–infected–recovered

A comparison of canny edge detection implementations with hadoop

学校读者我要写书评

暂无评论

A comparison of canny edge detection implementations with ha...

Future of Information and Communication Conference, FICC 2018

作者： Smalley, Josiah Oliveira, Suely Applied Math and Computational Sciences University of Iowa Iowa CityIA52242 United States Department of Computer Science University of Iowa Iowa CityIA52242 United States

ISBN: (纸本)9783030034047

Edge detection plays a large role in digital image processing;it allows for easier identification of objects within the image from a human perspective and also opens the door for automated object detection via machine learning. One of the common edge detection algorithms, developed by John F. Canny in 1986, is a multi-stage algorithm making use of gradients within the image to calculate potential edges. Hadoop is a Java library built on the MapReduce framework and was designed to be used for distributed processing of big data projects. We, instead, choose to adapt the Canny edge detection algorithm to run via Hadoop using two methods: a streaming Python implementation and a Java implementation to compare their run times and to determine whether or not using Hadoop for such a problem is desirable over the classic sequential implementation. In Sect. 1, we introduce edge detection, explaining what it is and why it is important. In Sect. 2, we explore Canny’s algorithm. In Sect. 3, we then explain our methodology of parallelizing the sequential code and the issues which arise during the process. In Sect. 4, we display the results of our implementations. In Sect. 5 we conclude with a possible explanation of why our code does not perform as well as we had anticipated;due to memory limitations of our specific Hadoop cluster, the implementations created do not perform well for processing typical images, but do allow for the processing of very large images. We also explore possibilities for future work. © 2019, Springer Nature Switzerland AG.

关键词： Python

Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules 22

学校读者我要写书评

暂无评论

Beyond accuracy: generalization properties of bio-plausible ...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Yuhan Helena Liu Arna Ghosh Blake A. Richards Eric Shea-Brown Guillaume Lajoie Department of Applied Mathematics University of Washington Seattle WA and Allen Institute for Brain Science Seattle WA and Computational Neuroscience Center University of Washington Seattle WA School of Computer Science McGill University Montreal QC Canada and Mila - Quebec AI Institute Montreal QC Canada School of Computer Science McGill University Montreal QC Canada and Mila - Quebec AI Institute Montreal QC Canada and Department of Neurology and Neurosurgery Montreal Neurological Institute McGill University Montreal QC Canada and Canada CIFAR AI Chair CIFAR Toronto ON Canada Mila - Quebec AI Institute Montreal QC Canada and Canada CIFAR AI Chair CIFAR Toronto ON Canada and Dept. de Mathématiques et Statistiques Université de Montréal Montreal QC Canada

ISBN: (纸本)9781713871088

To unveil how the brain learns, ongoing work seeks biologically-plausible approximations of gradient descent algorithms for training recurrent neural networks (RNNs). Yet, beyond task accuracy, it is unclear if such learning rules converge to solutions that exhibit different levels of generalization than their non-biologically-plausible counterparts. Leveraging results from deep learning theory based on loss landscape curvature, we ask: how do biologically-plausible gradient approximations affect generalization? We first demonstrate that state-of-the-art biologically-plausible learning rules for training RNNs exhibit worse and more variable generalization performance compared to their machine learning counterparts that follow the true gradient more closely. Next, we verify that such generalization performance is correlated significantly with loss landscape curvature, and we show that biologically-plausible learning rules tend to approach high-curvature regions in synaptic weight space. Using tools from dynamical systems, we derive theoretical arguments and present a theorem explaining this phenomenon. This predicts our numerical results, and explains why biologically-plausible rules lead to worse and more variable generalization properties. Finally, we suggest potential remedies that could be used by the brain to mitigate this effect. To our knowledge, our analysis is the first to identify the reason for this generalization gap between artificial and biologically-plausible learning rules, which can help guide future investigations into how the brain learns solutions that generalize.

关键词：