检索结果-内蒙古大学图书馆

Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hutzenthaler, Martin Jentzen, Arnulf Pohl, Katharina Riekert, Adrian Scarpa, Luca Faculty of Mathematics University of Duisburg-Essen Essen Germany School of Data Science Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China Applied Mathematics: Institute for Analysis and Numerics Faculty of Mathematics and Computer Science University of Münster Münster Germany Department of Mathematics Politecnico di Milano Milano Italy

In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day it remains an open problem of research to provide a mathematical convergence analysis which rigorously explains the success of SGD type optimization methods in the training of DNNs. In this work we study SGD type optimization methods in the training of fully-connected feedforward DNNs with rectified linear unit (ReLU) activation. We first establish general regularity properties for the risk functions and their generalized gradient functions appearing in the training of such DNNs and, thereafter, we investigate the plain vanilla SGD optimization method in the training of such DNNs under the assumption that the target function under consideration is a constant function. Specifically, we prove under the assumption that the learning rates (the step sizes of the SGD optimization method) are sufficiently small but not L1-summable and under the assumption that the target function is a constant function that the expectation of the risk of the considered SGD process converges in the training of such DNNs to zero as the number of SGD steps increases to infinity. Copyright © 2021, The Authors. All rights reserved.

关键词： Deep neural networks

Local Fourier analysis of a space-time multigrid method for DG-SEM for the linear advection equation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Versbach, Lea M. Birken, Philipp Linders, Viktor Gassner, Gregor Centre for Mathematical Sciences Numerical Analysis Lund University Lund Sweden Center for Data and Simulation Sciences Department of Mathematics and Computer Science Weyertal 86-90 Köln50931 Germany

In this paper we present a Local Fourier analysis of a space-time multigrid solver for a hyperbolic test problem. The space-time discretization is based on arbitrarily high order discontinuous Galerkin spectral element methods in time and a first order finite volume method in space. We apply a block Jacobi smoother and consider coarsening in space-time, as well as temporal coarsening only. Asymptotic convergence factors for the smoother and the two-grid method for both coarsening strategies are presented. For high CFL numbers, the convergence factors for both strategies are 0.5 for first order, and 0.375 for second order accurate temporal approximations. Numerical experiments in one and two spatial dimensions for space-time DG-SEM discretizations of varying order gives even better convergence rates of around 0.3 and 0.25 for sufficiently high CFL *** Codes 65M55, 65M22, 65M60, 65T99 © 2021, CC BY-SA.

关键词： Fourier analysis

Sensitivity analysis for unmeasured confounding via effect extrapolation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Loh, Wen Wei Vansteelandt, Stijn Department of Data Analysis Ghent University Gent Belgium Department of Applied Mathematics Computer Science and Statistics Ghent University Ghent Belgium Department of Medical Statistics London School of Hygiene and Tropical Medicine London United Kingdom

Inferring the causal effect of a non-randomly assigned exposure on an outcome requires adjusting for common causes of the exposure and outcome to avoid biased conclusions. Notwithstanding the efforts investigators routinely make to measure and adjust for such common causes (or confounders), some confounders typically remain unmeasured, raising the prospect of biased inference in observational studies. Therefore, it is crucial that investigators can practically assess their substantive conclusions' relative (in)sensitivity to potential unmeasured confounding. In this article, we propose a sensitivity analysis strategy that is informed by the stability of the exposure effect over different, well-chosen subsets of the measured confounders. The proposal entails first approximating the process for recording confounders to learn about how the effect is potentially affected by varying amounts of unmeasured confounding, then extrapolating to the effect had hypothetical unmeasured confounders been additionally adjusted for. A large set of measured confounders can thus be exploited to provide insight into the likely presence of unmeasured confounding bias, albeit under an assumption about how data on the confounders are recorded. The proposal's ability to reveal the true effect and ensure valid inference after extrapolation is empirically compared with existing methods using simulation studies. We demonstrate the procedure using two different publicly available datasets commonly used for causal inference. © 2021, CC BY.

关键词： Extrapolation

Interval Evaluation of Stationary State Probabilities for Markov Set-Chain Models

学校读者我要写书评

暂无评论

Interval Evaluation of Stationary State Probabilities for Ma...

International Conference on Advanced computer Information Technologies (ACIT)

作者： Leonid Lyubchyk Galyna Grinberg Maria Lubchick Alexey Galuza Olena Akhiiezer Department of Computer Mathematics and Data Analysis National Technical University “Kharkiv Polytechnic Institute” Kharkiv Ukraine

ISBN: (数字)9781728167602

ISBN: (纸本)9781728167619

Interval estimation problem for stationary state probability distribution of set-chain Markov model uncertainties of transition matrix parameters is considered. To obtain final probability vector interval estimates, an optimization approach is proposed using regularized Lagrange function. To solve the obtained regularized bilinear programming problem, computational gradient algorithms are used with ensures the stability of resulting estimates. An example of Markov model of a Bonus-Malus system with interval uncertainties of claim flow intensity is presented.

关键词： Markov processes Uncertainty Stationary state Mathematical model Probability distribution Biological system modeling

Identifying 3D genome organization in diploid organisms via euclidean distance geometry

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Belyaeva, Anastasiya Kubjas, Kaie Sun, Lawrence J. Uhler, Caroline Laboratory for Information and Decision Systems Department of Electrical Engineering and Computer Science Institute for Data Systems and Society Massachusetts Institute of Technology CambridgeMA United States Department of Mathematics and Systems Analysis Aalto University Espoo Finland

The spatial organization of the DNA in the cell nucleus plays an important role for gene regulation, DNA replication, and genomic integrity. Through the development of chromosome conformation capture experiments (such as 3C, 4C, Hi-C) it is now possible to obtain the contact frequencies of the DNA at the whole-genome level. In this paper, we study the problem of reconstructing the 3D organization of the genome from such whole-genome contact frequencies. A standard approach is to transform the contact frequencies into noisy distance measurements and then apply semidefinite programming (SDP) formulations to obtain the 3D configuration. However, neglected in such reconstructions is the fact that most eukaryotes including humans are diploid and therefore contain two copies of each genomic locus. We prove that the 3D organization of the DNA is not identifiable from distance measurements derived from contact frequencies in diploid organisms. In fact, there are infinitely many solutions even in the noise-free setting. We then discuss various additional biologically relevant and experimentally measurable constraints (including distances between neighboring genomic loci and higher-order interactions) and prove identifiability under these conditions. Furthermore, we provide SDP formulations for computing the 3D embedding of the DNA with these additional constraints and show that we can recover the true 3D embedding with high accuracy from both noiseless and noisy measurements. Finally, we apply our algorithm to real pairwise and higher-order contact frequency data and show that we can recover known genome organization patterns. © 2021, CC BY.

关键词： DNA

LDoS attack detection method based on traffic time-frequency characteristics

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Yu Duan, Xueyuan Wang, Kun Li, Bin Department of Information Security Naval University of Engineering Wuhan430033 China College of Computer and Information Technology Xinyang Normal University Xinyang464000 China Henan Key Laboratory of Analysis and Applications of Education Big Data Xinyang464000 China School of Mathematics and Information Engineering Xinyang Vocational and Technical College Xinyang464000 China

For the traditional denial-of-service attack detection methods have complex algorithms and high computational overhead, which are difficult to meet the demand of online detection;and the experimental environment is mostly a simulation platform, which is difficult to deploy in real network environment, we propose a real network environment-oriented LDoS attack detection method based on the time-frequency characteristics of traffic data. All the traffic data flowing through the Web server is obtained through the acquisition storage system, and the detection data set is constructed using pre-processing;the simple features of the flow fragments are used as input, and the deep neural network is used to learn the time-frequency domain features of normal traffic features and generate reconstructed sequences, and the LDoS attack is discriminated based on the differences between the reconstructed sequences and the input data in the time-frequency domain. The experimental results show that the proposed method can accurately detect the attack features in the flow fragments in a very short time and achieve high detection accuracy for complex and diverse LDoS attacks;since only the statistical features of the packets are used, there is no need to parse the packet data, which can be adapted to different network environments. Copyright © 2022, The Authors. All rights reserved.

关键词： Denial-of-service attack

A COMPARATIVE SURVEY OF VISION TRANSFORMERS FOR FEATURE EXTRACTION IN TEXTURE analysis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Scabini, Leonardo Sacilotti, Andre Zielinski, Kallil M. Ribas, Lucas C. De Baets, Bernard Bruno, Odemir M. São Carlos Institute of Physics University of São Paulo SP São Carlos13560-970 Brazil Institute of Mathematics and Computer Sciences University of São Paulo SP São Carlos135666-590 Brazil Institute of Biosciences Humanities and Exact Sciences São Paulo State University São José do Rio Preto15054-000 Brazil KERMIT Department of Data Analysis and Mathematical Modelling Ghent University Coupure links 653 Ghent9000 Belgium

Texture, a significant visual attribute in images, has been extensively investigated across various image recognition applications. Convolutional Neural Networks (CNNs), which have been successful in many computer vision tasks, are currently among the best texture analysis approaches. On the other hand, Vision Transformers (ViTs) have been surpassing the performance of CNNs on tasks such as object recognition, causing a paradigm shift in the field. However, ViTs have so far not been scrutinized for texture recognition, hindering a proper appreciation of their potential in this specific setting. For this reason, this work explores various pre-trained ViT architectures when transferred to tasks that rely on textures. We review 21 different ViT variants and perform an extensive evaluation and comparison with CNNs and hand-engineered models on several tasks, such as assessing robustness to changes in texture rotation, scale, and illumination, and distinguishing color textures, material textures, and texture attributes. The goal is to understand the potential and differences among these models when directly applied to texture recognition, using pre-trained ViTs primarily for feature extraction and employing linear classifiers for evaluation. We also evaluate their efficiency, which is one of the main drawbacks in contrast to other methods. Our results show that ViTs generally outperform both CNNs and hand-engineered models, especially when using stronger pre-training and tasks involving in-the-wild textures (images from the internet). We highlight the following promising models: ViT-B with DINO pre-training, BeiTv2, and the Swin architecture, as well as the EfficientFormer as a low-cost alternative. In terms of efficiency, although having a higher number of GFLOPs and parameters, ViT-B and BeiT(v2) can achieve a lower feature extraction time on GPUs compared to ResNet50. © 2024, CC BY-NC-SA.

关键词： Textures

Risk Management in Stochastic Problems of Stock Investment 13

学校读者我要写书评

暂无评论

Risk Management in Stochastic Problems of Stock Investment

13th International Conference Management of Large-Scale System Development, MLSD 2020

作者： Gorelik, Victor Zolotova, Tatiana Moscow State Pedagogical University Department of Simulation Systems and Operations Research Institute of Mathematics and Computer Science Dorodnicyn Computing Centre Frc Csc Ras Moscow Russia Financial University under the Government of the Russian Federation Department of Data Analysis Decision Making and Financial Technologies Moscow Russia

ISBN: (数字)9781728110943

ISBN: (纸本)9781728110943

We consider a game with nature as a decision-making model in stochastic problems of stock investment. The mathematical expectation of the investor's gain is taken as an assessment of efficiency. The standard deviation is used as an assessment of risk. The emerging two-criterion problem is formalized using the method of translating one criterion into a constraint. The case of pure strategies (the choice of one investment option) and the case of mixed strategies (diversification of investment) are considered. For the case of mixed strategies, either the problem of maximizing a linear function with one linear and one quadratic constraint or the problem of minimizing a quadratic function with two linear constraints arises. Analytical solution methods based on the Karush-Kuhn-Tucker optimality conditions are obtained for both problems. Practical examples of the application of these methods using real statistical data are given. © 2020 IEEE.

关键词： Investments

Artificial intelligence for modelling infectious disease epidemics

学校读者我要写书评

暂无评论

Nature 2025年第8051期638卷 623-635页

作者： Kraemer, Moritz U. G. Tsui, Joseph L.-H. Chang, Serina Y. Lytras, Spyros Khurana, Mark P. Vanderslott, Samantha Bajaj, Sumali Scheidwasser, Neil Curran-Sebastian, Jacob Liam Semenova, Elizaveta Zhang, Mengyan Unwin, H. Juliette T. Watson, Oliver J. Mills, Cathal Dasgupta, Abhishek Ferretti, Luca Scarpino, Samuel V. Koua, Etien Morgan, Oliver Tegally, Houriiyah Paquet, Ulrich Moutsianas, Loukas Fraser, Christophe Ferguson, Neil M. Topol, Eric J. Duchêne, David A. Stadler, Tanja Kingori, Patricia Parker, Michael J. Dominici, Francesca Shadbolt, Nigel Suchard, Marc A. Ratmann, Oliver Flaxman, Seth Holmes, Edward C. Gomez-Rodriguez, Manuel Schölkopf, Bernhard Donnelly, Christl A. Pybus, Oliver G. Cauchemez, Simon Bhatt, Samir Pandemic Sciences Institute University of Oxford Oxford United Kingdom Department of Biology University of Oxford Oxford United Kingdom Department of Electrical Engineering and Computer Science University of California Berkeley Berkeley CA United States UCSF UC Berkeley Joint Program in Computational Precision Health Berkeley CA United States Division of Systems Virology Department of Microbiology and Immunology The Institute of Medical Science The University of Tokyo Tokyo Japan Section of Epidemiology Department of Public Health University of Copenhagen Copenhagen Denmark Oxford Vaccine Group University of Oxford and NIHR Oxford Biomedical Research Centre Oxford United Kingdom Department of Epidemiology and Biostatistics Imperial College London London United Kingdom Department of Computer Science University of Oxford Oxford United Kingdom School of Mathematics University of Bristol Bristol United Kingdom MRC Centre for Global Infectious Disease Analysis School of Public Health Imperial College London London United Kingdom Department of Statistics University of Oxford Oxford United Kingdom Doctoral Training Centre University of Oxford Oxford United Kingdom Institute for Experiential AI Northeastern University MA Boston Thailand Santa Fe Institute Santa Fe NM United States World Health Organization Regional Office for Africa Brazzaville Congo WHO Hub for Pandemic and Epidemic Intelligence Health Emergencies Programme World Health Organization Berlin Germany Centre for Epidemic Response and Innovation (CERI) School for Data Science and Computational Thinking Stellenbosch University Stellenbosch South Africa African Institute for Mathematical Sciences (AIMS) South Africa Muizenberg Cape Town South Africa Genomics England London United Kingdom Scripps Research La Jolla CA United States Department of Biosystems Science and Engineering ETH Zürich Basel Switzerland Swiss Institute of Bioinformatics Lausanne Switzerland The Ethox Centre Nuffield

Infectious disease threats to individual and public health are numerous, varied and frequently unexpected. Artificial intelligence (AI) and related technologies, which are already supporting human decision making in economics, medicine and social science, have the potential to transform the scope and power of infectious disease epidemiology. Here we consider the application to infectious disease modelling of AI systems that combine machine learning, computational statistics, information retrieval and data science. We first outline how recent advances in AI can accelerate breakthroughs in answering key epidemiological questions and we discuss specific AI methods that can be applied to routinely collected infectious disease surveillance data. Second, we elaborate on the social context of AI for infectious disease epidemiology, including issues such as explainability, safety, accountability and ethics. Finally, we summarize some limitations of AI applications in this field and provide recommendations for how infectious disease epidemiology can harness most effectively current and future developments in AI. © Springer Nature Limited 2025.

关键词：