检索结果-内蒙古大学图书馆

Making method of moments great again? - How can GANs learn distributions

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Yuanzhi Dou, Zehao Machine Learning Department Carnegie Mellon University United States Department of Statistics and Data Science Yale University United States

Generative Adversarial Networks (GANs) are widely used models to learn complex real-world distributions. In GANs, the training of the generator usually stops when the discriminator can no longer distinguish the generator's output from the set of training examples. A central question of GANs is that when the training stops, whether the generated distribution is actually close to the target distribution, and how the training process reaches to such configurations efficiently? In this paper, we established a theoretical results towards understanding this generator-discriminator training process. We empirically observe that during the earlier stage of the GANs training, the discriminator is trying to force the generator to match the low degree moments between the generator's output and the target distribution. Moreover, only by matching these empirical moments over polynomially many training examples, we prove that the generator can already learn notable class of distributions, including those that can be generated by two-layer neural networks. Copyright © 2020, The Authors. All rights reserved.

关键词： Generative adversarial networks

Dimension-agnostic inference using cross U-statistics

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Kim, Ilmun Ramdas, Aaditya Departments of Statistics & Data Science Applied Statistics Yonsei University Seoul Korea Republic of Departments of Statistics & Data Science Machine Learning Carnegie Mellon University Pittsburgh United States

Classical asymptotic theory for statistical inference usually involves calibrating a statistic by fixing the dimension d while letting the sample size n increase to infinity. Recently, much effort has been dedicated towards understanding how these methods behave in high-dimensional settings, where d and n both increase to infinity together. This often leads to different inference procedures, depending on the assumptions about the dimensionality, leaving the practitioner in a bind: given a dataset with 100 samples in 20 dimensions, should they calibrate by assuming n d, or d/n ≈ 0.2? This paper considers the goal of dimension-agnostic inference-developing methods whose validity does not depend on any assumption on d versus n. We introduce an approach that uses variational representations of existing test statistics along with sample splitting and self-normalization to produce a refined test statistic with a Gaussian limiting distribution, regardless of how d scales with n. The resulting statistic can be viewed as a careful modification of degenerate U-statistics, dropping diagonal blocks and retaining off-diagonal blocks. We exemplify our technique for some classical problems including one-sample mean and covariance testing, and show that our tests have minimax rate-optimal power against appropriate local alternatives. In most settings, our cross U-statistic matches the high-dimensional power of the corresponding (degenerate) U-statistic up to a √2 factor. © 2020, CC BY.

关键词： Statistics

HOW DOES THE BRAIN COMPUTE WITH PROBABILITIES?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Haefner, Ralf M. Beck, Jeff Savin, Cristina Salmasi, Mehrdad Pitkow, Xaq Department of Brain and Cognitive Sciences University of Rochester RochesterNY United States Department of Neurobiology Duke University DurhamNC United States Departments of Neural Science and Data Science New York University New YorkNY United States Gatsby Computational Neuroscience Unit Max Planck UCL Centre for Computational Psychiatry and Ageing Research University College London United Kingdom Neuroscience Institute Department of Machine Learning Carnegie Mellon University PittsburghPA United States Department of Neuroscience Center for Neuroscience and Artificial Intelligence Baylor College of Medicine HoustonTX United States Department of Electrical and Computer Engineering Department of Computer Science Rice University HoustonTX United States

This perspective piece is the result of a Generative Adversarial Collaboration (GAC) tackling the question 'How does neural activity represent probability distributions?'. We have addressed three major obstacles to progress on answering this question: first, we provide a unified language for defining competing hypotheses. Second, we explain the fundamentals of three prominent proposals for probabilistic computations - Probabilistic Population Codes (PPCs), Distributed Distributional Codes (DDCs), and Neural Sampling Codes (NSCs) - and describe similarities and differences in that common language. Third, we review key empirical data previously taken as evidence for at least one of these proposal, and describe how it may or may not be explainable by alternative proposals. Finally, we describe some key challenges in resolving the debate, and propose potential directions to address them through a combination of theory and *** Codes 92 Copyright © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Harder, Paula Pfreundt, Franz-Josef Keuper, Margret Keuper, Janis Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Scientic Computing University of Kaiserslautern Kaiserlautern Germany Fraunhofer Center Machine Learning Germany Data and Web Science Group University of Mannheim Germany Offenburg University Germany

—Despite the success of convolutional neural networks (CNNs) in many computer vision and image analysis tasks, they remain vulnerable against so-called adversarial attacks: Small, crafted perturbations in the input images can lead to false predictions. A possible defense is to detect adversarial examples. In this work, we show how analysis in the Fourier domain of input images and feature maps can be used to distinguish benign test samples from adversarial images. We propose two novel detection methods: Our first method employs the magnitude spectrum of the input images to detect an adversarial attack. This simple and robust classifier can successfully detect adversarial perturbations of three commonly used attack methods. The second method builds upon the first and additionally extracts the phase of Fourier coefficients of feature-maps at different layers of the network. With this extension, we are able to improve adversarial detection rates compared to state-of-the-art detectors on five different attack methods. © 2021, CC BY.

关键词： Image classification

Species-specific responses of canopy greenness to the extreme droughts of 2018 and 2022 for four abundant tree species in Germany

学校读者我要写书评

暂无评论

science of the Total Environment 2025年 958卷 177938页

作者： Wang, Yixuan Rammig, Anja Blickensdörfer, Lukas Wang, Yuanyuan Zhu, Xiao Xiang Buras, Allan Professorship for Land Surface-Atmosphere Interactions Technical University of Munich Hans-Carl-v.-Carlowitz-Platz 2 Freising85354 Germany Thünen Institute of Farm Economics Bundesallee 63 Braunschweig38116 Germany Thünen Institute of Forest Ecosystems Alfred-Moeller-Straße 1 Eberswalde16225 Germany Earth Observation Lab Geography Department Humboldt University of Berlin Unter den Linden 6 Berlin10099 Germany Chair of Data Science in Earth Observation Technical University of Munich Arcisstraße 21 Munich80333 Germany Munich Center for Machine Learning Arcisstraße 21 Munich80333 Germany

Germany experienced extreme drought periods in 2018 and 2022, which significantly affected forests. These drought periods were natural experiments, providing valuable insights into how different tree species respond to drought. The quantification of species-specific drought responses may help to identify the most climate-change-resilient tree species, thereby informing effective forest regeneration strategies. In this study, we used remotely sensed peak-season canopy greenness as a proxy for tree vitality to estimate the drought response of four widely abundant tree species in Germany (oak, beech, spruce, and pine). We focused on two questions: (1) How were the four tree species affected by these two droughts? (2) Which environmental parameters primarily determined canopy greenness? To address these questions, we combined a recently published tree species classification map with remotely sensed canopy greenness and environmental variables related to plant available water capacity (PAWC) and atmospheric vapor pressure deficit (VPD). Our results indicate that the more isohydric species featured a greater decline in canopy greenness under these droughts compared to the more anisohydric species despite similar soil moisture conditions. Based on spatial lag models, we found that the influence of PAWC on canopy greenness increases with increasing isohydricity while the influence of VPD decreases. Our statistical analysis indicates that oak was the only species with significantly higher canopy greenness in 2022 compared to 2018. Yet, all species are likely to be susceptible to accumulated drought effects, such as insufficient recovery time and increased vulnerability to biotic pathogens, in the coming years. Our study provides critical insights into the diverse responses of different tree species to changing environments over a large environmental gradient in Central Europe and sheds light on the complex interactions between soil moisture, climate variables, and canopy gre

关键词： Forest ecology

Exploring Self-Attention for Crop-type Classification Explainability

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Obadic, Ivica Roscher, Ribana Oliveira, Dario Augusto Borges Zhu, Xiao Xiang Data Science in Earth Observation Technical University of Munich Munich Center for Machine Learning [MCML Arcisstraße 21 Munich80333 Germany Research Center Jülich Institute of Bio- and Geosciences Plant Sciences Wilhelm-Johnen-Straße Jülich52428 Germany International AI Future Lab: Artificial Intelligence for Earth Observation TUM and with School of Applied Mathematics Getulio Vargas Foundation Rio de Janeiro Brazil Data Science in Earth Observation Technical University of Munich Arcisstraße 21 Munich80333 Germany

Automated crop-type classification using Sentinel-2 satellite time series is essential to support agriculture monitoring. Recently, deep learning models based on transformer encoders became a promising approach for crop-type classification. Using explainable machine learning to reveal the inner workings of these models is an important step towards improving stakeholders’ trust and efficient agriculture monitoring. In this paper, we introduce a novel explainability framework that aims to shed a light on the essential crop disambiguation patterns learned by a state-of-the-art transformer encoder model. More specifically, we process the attention weights of a trained transformer encoder to reveal the critical dates for crop disambiguation and use domain knowledge to uncover the phenological events that support the model performance. We also present a sensitivity analysis approach to understand better the attention capability for revealing crop-specific phenological events. We report compelling results showing that attention patterns strongly relate to key dates, and consequently, to the critical phenological events for crop-type classification. These findings might be relevant for improving stakeholder trust and optimizing agriculture monitoring processes. Additionally, our sensitivity analysis demonstrates the limitation of attention weights for identifying the important events in the crop phenology as we empirically show that the unveiled phenological events depend on the other crops in the data considered during training. © 2022, CC BY.

关键词： Time series

Few-shot Class-incremental learning for 3D Point Cloud Objects

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chowdhury, Townim Cheraghian, Ali Ramasinghe, Sameera Ahmadi, Sahar Saberi, Morteza Rahman, Shafin Dept. of Electrical and Computer Engineering North South University Bangladesh School of Engineering Australian National University Australia Data61 Commonwealth Scientific and Industrial Research Organisation Australia Australian Institute for Machine Learning University of Adelaide Australia Business School The University of New South Wales Australia School of Computer Science and DSI University of Technology Sydney Australia

Few-shot class-incremental learning (FSCIL) aims to incrementally fine-tune a model (trained on base classes) for a novel set of classes using a few examples without forgetting the previous training. Recent efforts address this problem primarily on 2D images. However, due to the advancement of camera technology, 3D point cloud data has become more available than ever, which warrants considering FSCIL on 3D data. This paper addresses FSCIL in the 3D domain. In addition to well-known issues of catastrophic forgetting of past knowledge and overfitting of few-shot data, 3D FSCIL can bring newer challenges. For example, base classes may contain many synthetic instances in a realistic scenario. In contrast, only a few real-scanned samples (from RGBD sensors) of novel classes are available in incremental steps. Due to the data variation from synthetic to real, FSCIL endures additional challenges, degrading performance in later incremental steps. We attempt to solve this problem using Microshapes (orthogonal basis vectors) by describing any 3D objects using a pre-defined set of rules. It supports incremental training with few-shot examples minimizing synthetic to real data variation. We propose new test protocols for 3D FSCIL using popular synthetic datasets (ModelNet and ShapeNet) and 3D real-scanned datasets (ScanObjectNN and CO3D). By comparing state-of-the-art methods, we establish the effectiveness of our approach in the 3D domain. Code is available at: https://***/townim-faisal/FSCIL-3D. © 2022, CC BY.

关键词：

Autonomous data extraction from peer reviewed literature for training machine learning models of oxidation potentials

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lee, Siwoo Heinen, Stefan Khan, Danish Von Lilienfeld, O. Anatole Department of Chemistry University of Toronto St. George campus TorontoON Canada Vector Institute for Artificial Intelligence TorontoONM5S 1M1 Canada Acceleration Consortium University of Toronto 80 St George St TorontoONM5S 3H6 Canada Department of Materials Science and Engineering University of Toronto St. George campus TorontoON Canada Department of Physics University of Toronto St. George campus TorontoON Canada Machine Learning Group Technische Universität Berlin Berlin Institute for the Foundations of Learning and Data Berlin Germany

We present an automated data-collection pipeline involving a convolutional neural network and a large language model to extract user-specified tabular data from peer-reviewed literature. The pipeline is applied to 74 reports published between 1957 and 2014 with experimentally-measured oxidation potentials for 592 organic molecules (-0.75-3.58 V). After data curation (solvents, reference electrodes, and missed data points), we trained multiple supervised machine learning models reaching prediction errors similar to experimental uncertainty (∼0.2 V). For experimental measurements of identical molecules reported in multiple studies, we identified the most likely value based on out-of-sample machine learning predictions. Using the trained machine learning models, we then estimated oxidation potentials of ∼132k small organic molecules from the QM9 data set, with predicted values spanning 0.21-3.46 V. Analysis of the QM9 predictions in terms of plausible descriptor-property trends suggests that aliphaticity increases the oxidation potential of an organic molecule on average from ∼1.5 V to ∼2 V, while an increase in number of heavy atoms lowers it systematically. The pipeline introduced offers significant reductions in human labor otherwise required for conventional manual data collection of experimental results, and exemplifies how to accelerate scientific research through automation. © 2023, CC0.

关键词： Pipelines

Emulation of cosmological mass maps with conditional generative adversarial networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Perraudin, Nathanaël Marcon, Sandro Lucchi, Aurelien Kacprzak, Tomasz Swiss Data Science Center ETH Zurich Institute for Machine Learning ETH Zurich Institute for Particle Physics and Astrophysics ETH Zurich

Weak gravitational lensing mass maps play a crucial role in understanding the evolution of structures in the universe and our ability to constrain cosmological models. The prediction of these mass maps is based on expensive N-body simulations, which can create a computational bottleneck for cosmological analyses. Simulation-based emulators of map summary statistics, such as the matter power spectrum and its covariance, are starting to play increasingly important role, as the analytical predictions are expected to reach their precision limits for upcoming experiments. Creating an emulator of the cosmological mass maps themselves, rather than their summary statistics, is a more challenging task. Modern deep generative models, such as Generative Adversarial Networks (GAN), have demonstrated their potential to achieve this goal. Most existing GAN approaches produce simulations for a fixed value of the cosmological parameters, which limits their practical applicability. We propose a novel conditional GAN model that is able to generate mass maps for any pair of matter density Ωm and matter clustering strength σ8, parameters which have the largest impact on the evolution of structures in the universe, for a given source galaxy redshift distribution n(z). Our results show that our conditional GAN can interpolate efficiently within the space of simulated cosmologies, and generate maps anywhere inside this space with good visual quality high statistical accuracy. We perform an extensive quantitative comparison of the N-body and GAN -generated maps using a range of metrics: the pixel histograms, peak counts, power spectra, bispectra, Minkowski functionals, correlation matrices of the power spectra, the Multi-Scale Structural Similarity Index (MS-SSIM) and our equivalent of the Fréchet Inception Distance (FID). We find a very good agreement on these metrics, with typical differences are 1 and the data2 publicly available. Copyright © 2020, The Authors. All rights reserved.

关键词： Generative adversarial networks