检索结果-内蒙古大学图书馆

CONSTRAINED CONSENSUS-BASED OPTIMIZATION AND NUMERICAL HEURISTICS FOR THE FEW PARTICLE REGIME

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Beddrich, Jonas Chenchene, Enis Fornasier, Massimo Huang, Hui Wohlmuth, Barbara Faculty of Mathematics University of Vienna Austria Department of Mathematics Munich Data Science Institute Technical University of Munich Garching by Munich & Munich Center for Machine Learning Munich Germany Department of Mathematics and Scientific Computing University of Graz Austria Department of Mathematics Technical University of Munich Garching by Munich Germany

Consensus-based optimization (CBO) is a versatile multi-particle optimization method for performing nonconvex and nonsmooth global optimizations in high dimensions. Proofs of global convergence in probability have been achieved for a broad class of objective functions in unconstrained optimizations. In this work we adapt the algorithm for solving constrained optimizations on compact and unbounded domains with boundary by leveraging emerging reflective boundary conditions. In particular, we close a relevant gap in the literature by providing a global convergence proof for the many-particle regime comprehensive of convergence rates. On the one hand, for the sake of minimizing running cost, it is desirable to keep the number of particles small. On the other hand, reducing the number of particles implies a diminished capability of exploration of the algorithm. Hence numerical heuristics are needed to ensure convergence of CBO in the few-particle regime. In this work, we also significantly improve the convergence and complexity of CBO by utilizing an adaptive region control mechanism and by choosing geometry-specific random noise. In particular, by combining a hierarchical noise structure with a multigrid finite element method, we are able to compute global minimizers for a constrained p-Allen–Cahn problem with obstacles, a very challenging variational *** Codes 90C26, 90C56, 90C59, 65K10, 35Q90, 60H10 © 2024, CC BY.

关键词： Constrained optimization

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bulin, Johannes Hamaekers, Jan Department of Virtual Material Design Fraunhofer Institute for Algorithms and Scientific Computing Schloss Birlinghoven Sankt Augustin53757 Germany Fraunhofer Center for Machine Learning Schloss Birlinghoven Sankt Augustin53757 Germany

Determining whether two particle systems are similar is a common problem in particle simulations. When the comparison should be invariant under permutations, orthogonal transformations, and translations of the systems, special techniques are needed. We present an algorithm that can test particle systems of finite size for similarity and, if they are similar, can find the optimal alignment between them. Our approach is based on an invariant version of the root mean square deviation (RMSD) measure and is capable of finding the globally optimal solution in O(n3) operations where n is the number of three-dimensional particles. Copyright © 2021, The Authors. All rights reserved.

关键词： machine learning

FrequencyLowCut Pooling - Plug & Play against Catastrophic Overfitting

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Grabinski, Julia Jung, Steffen Keuper, Janis Keuper, Margret Visual Computing Siegen University Germany Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Institute for Machine Learning and Analytics Offenburg University Germany Max Planck Institute for Informatics Saarland Informatics Campus Germany

Over the last years, Convolutional Neural Networks (CNNs) have been the dominating neural architecture in a wide range of computer vision tasks. From an image and signal processing point of view, this success might be a bit surprising as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. Sampling Theorem in their down-sampling operations. However, since poor sampling appeared not to affect model accuracy, this issue has been broadly neglected until model robustness started to receive more attention. Recent work [18] in the context of adversarial attacks and distribution shifts, showed after all, that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by poor downsampling operations. This paper builds on these findings and introduces an aliasing free down-sampling operation which can easily be plugged into any CNN architecture: FrequencyLowCut pooling. Our experiments show, that in combination with simple and Fast Gradient Sign Method (FGSM) adversarial training, our hyper-parameter free operator substantially improves model robustness and avoids catastrophic overfitting. Our code is available at https://***/GeJulia/flc_pooling Copyright © 2022, The Authors. All rights reserved.

关键词： Signal sampling

Improving generative model-based unfolding with Schrödinger bridges

学校读者我要写书评

暂无评论

Physical Review D 2024年第7期109卷 076011-076011页

作者： Sascha Diefenbacher Guan-Horng Liu Vinicius Mikuni Benjamin Nachman Weili Nie Physics Division Lawrence Berkeley National Laboratory Berkeley California 94720 USA Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology Atlanta Georgia 30332 USA National Energy Research Scientific Computing Center Berkeley Lab Berkeley California 94720 USA Berkeley Institute for Data Science University of California Berkeley California 94720 USA Machine Learning Research Group NVIDIA Research

machine learning-based unfolding has enabled unbinned and high-dimensional differential cross section measurements. Two main approaches have emerged in this research area; one based on discriminative models and one based on generative models. The main advantage of discriminative models is that they learn a small correction to a starting simulation while generative models scale better to regions of phase space with little data. We propose to use Schrödinger bridges and diffusion models to create sbunfold, an unfolding approach that combines the strengths of both discriminative and generative models. The key feature of sbunfold is that its generative model maps one set of events into another without having to go through a known probability density as is the case for normalizing flows and standard diffusion models. We show that sbunfold achieves excellent performance compared to state of the art methods on a synthetic Z+jets dataset.

关键词： machine learning Particle detectors Probability theory

Scalable hyperparameter optimization with lazy Gaussian processes

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Ram, Raju Müller, Sabine Pfreundt, Franz-Josef Gauger, Nicolas R. Keuper, Janis Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Fraunhofer Center Machine Learning Germany Scientific Computing Group TU Kaiserslautern Germany Institute for Machine Learning and Analytics Offenburg University Germany

Most machine learning methods require careful selection of hyper-parameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced to overcome tedious manual (try and error) tuning of these parameters. Due to its very high sample efficiency, Bayesian Optimization over a Gaussian Processes modeling of the parameter space has become the method of choice. Unfortunately, this approach suffers from a cubic compute complexity due to underlying Cholesky factorization, which makes it very hard to be scaled beyond a small number of sampling steps. In this paper, we present a novel, highly accurate approximation of the underlying Gaussian Process. Reducing its computational complexity from cubic to quadratic allows an efficient strong scaling of Bayesian Optimization while outperforming the previous approach regarding optimization accuracy. The first experiments show speedups of a factor of 162 in single node and further speed up by a factor of 5 in a parallel environment. Copyright © 2020, The Authors. All rights reserved.

关键词： Gaussian distribution

SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Harder, Paula Pfreundt, Franz-Josef Keuper, Margret Keuper, Janis Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Scientic Computing University of Kaiserslautern Kaiserlautern Germany Fraunhofer Center Machine Learning Germany Data and Web Science Group University of Mannheim Germany Offenburg University Germany

—Despite the success of convolutional neural networks (CNNs) in many computer vision and image analysis tasks, they remain vulnerable against so-called adversarial attacks: Small, crafted perturbations in the input images can lead to false predictions. A possible defense is to detect adversarial examples. In this work, we show how analysis in the Fourier domain of input images and feature maps can be used to distinguish benign test samples from adversarial images. We propose two novel detection methods: Our first method employs the magnitude spectrum of the input images to detect an adversarial attack. This simple and robust classifier can successfully detect adversarial perturbations of three commonly used attack methods. The second method builds upon the first and additionally extracts the phase of Fourier coefficients of feature-maps at different layers of the network. With this extension, we are able to improve adversarial detection rates compared to state-of-the-art detectors on five different attack methods. © 2021, CC BY.

关键词： Image classification

A scalable generative model for dynamical system reconstruction from neuroimaging data

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Volkmann, Eric Brändle, Alena Durstewitz, Daniel Koppe, Georgia Medical Faculty Mannheim Heidelberg University Mannheim Germany Institute for Machine Learning Johannes Kepler University Linz Austria Interdisciplinary Center for Scientific Computing Heidelberg University Heidelberg Germany Faculty of Physics and Astronomy Heidelberg University Heidelberg Germany Hector Institute for AI in Psychiatry Dept. for Psychiatry and Psychotherapy CIMH Germany Faculty of Mathematics and Computer Science Heidelberg University Heidelberg Germany

Data-driven inference of the generative dynamics underlying a set of observed time series is of growing interest in machine learning and the natural sciences. In neuroscience, such methods promise to alleviate the need to handcraft models based on biophysical principles and allow to automatize the inference of inter-individual differences in brain dynamics. Recent breakthroughs in training techniques for state space models (SSMs) specifically geared toward dynamical systems (DS) reconstruction (DSR) enable to recover the underlying system including its geometrical (attractor) and long-term statistical invariants from even short time series. These techniques are based on control-theoretic ideas, like modern variants of teacher forcing (TF), to ensure stable loss gradient propagation while training. However, as it currently stands, these techniques are not directly applicable to data modalities where current observations depend on an entire history of previous states due to a signal's filtering properties, as common in neuroscience (and physiology more generally). Prominent examples are the blood oxygenation level dependent (BOLD) signal in functional magnetic resonance imaging (fMRI) or Ca2+ imaging data. Such types of signals render the SSM's decoder model non-invertible, a requirement for previous TF-based methods. Here, exploiting the recent success of control techniques for training SSMs, we propose a novel algorithm that solves this problem and scales exceptionally well with model dimensionality and filter length. We demonstrate its efficiency in reconstructing dynamical systems, including their state space geometry and long-term temporal properties, from just short BOLD time series. © 2024, CC BY-NC-SA.

关键词： Magnetic resonance imaging

Deep neural networks and PIDE discretizations

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bohn, Bastian Griebel, Michael Kannan, Dinesh Fraunhofer Center for Machine Learning Schloss Birlinghoven Sankt Augustin53754 Germany Fraunhofer Institute for Algorithms and Scientific Computing SCAI Schloss Birlinghoven Sankt Augustin53754 Germany Institute for Numerical Simulation University of Bonn Friedrich-Hirzebruch-Allee 7 Bonn53115 Germany

In this paper, we propose neural networks that tackle the problems of stability and field-of-view of a Convolutional Neural Network (CNN). As an alternative to increasing the network's depth or width to improve performance, we propose integral-based spatially nonlocal operators which are related to global weighted Laplacian, fractional Laplacian and inverse fractional Laplacian operators that arise in several problems in the physical sciences. The forward propagation of such networks is inspired by partial integro-differential equations (PIDEs). We test the effectiveness of the proposed neural architectures on benchmark image classification datasets and semantic segmentation tasks in autonomous driving. Moreover, we investigate the extra computational costs of these dense operators and the stability of forward propagation of the proposed neural *** Codes 65D15, 65L07, 68T05, 68W25, 47G20, 47G30 © 2021, CC BY.

关键词： Deep neural networks

Combating mode collapse in GAN training: An empirical analysis using hessian eigenvalues

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Lopez, Ricard Durall Chatzimichailidis, Avraam Labus, Peter Keuper, Janis Fraunhofer ITWM Germany IWR University of Heidelberg Germany Chair for Scientific Computing TU Kaiserslautern Germany Fraunhofer Center Machine Learning Germany Institute for Machine Learning and Analytics Offenburg University Germany

Generative adversarial networks (GANs) provide state-of-the-art results in image generation. However, despite being so powerful, they still remain very challenging to train. This is in particular caused by their highly non-convex optimization space leading to a number of instabilities. Among them, mode collapse stands out as one of the most daunting ones. This undesirable event occurs when the model can only fit a few modes of the data distribution, while ignoring the majority of them. In this work, we combat mode collapse using second-order gradient information. To do so, we analyse the loss surface through its Hessian eigenvalues, and show that mode collapse is related to the convergence towards sharp minima. In particular, we observe how the eigenvalues of the G are directly correlated with the occurrence of mode collapse. Finally, motivated by these findings, we design a new optimization algorithm called nudged-Adam (NuGAN) that uses spectral information to overcome mode collapse, leading to empirically more stable convergence properties. © 2020, CC BY.

关键词： Eigenvalues and eigenfunctions