检索结果-内蒙古大学图书馆

Optimal neural summarization for full-field weak lensing cosmological implicit inference

Astronomy and Astrophysics 2025年 697卷

作者： Lanzieri, Denise Zeghal, Justine Lucas Makinen, T. Boucaud, Alexandre Starck, Jean-Luc Lanusse, François Université Paris Cité Université Paris-Saclay CEA CNRS AIM Gif-sur-Yvette F-91191 France Université Paris Cité CNRS Paris F-75013 France Imperial Centre for Inference and Cosmology (ICIC) & Astrophysics Group Imperial College London Blackett Laboratory Prince Consort Road London SW7 2AZ United Kingdom Université Paris-Saclay Université Paris Cité CEA CNRS AIM Gif-sur-Yvette 91191 France Sony Computer Science Laboratories - Rome Joint Initiative CREF-SONY Centro Ricerche Enrico Fermi Via Panisperna 89/A Rome 00184 Italy Institutes of Computer Science and Astrophysics Foundation for Research and Technology Hellas (FORTH) Heraklion 70013 Greece Center for Computational Astrophysics Flatiron Institute 162 5th Ave New York 10010 NY United States Department of Physics Université de Montréal Montréal H2V 0B3 Canada Mila - Quebec Artificial Intelligence Institute Montréal H2S 3H1 Canada Ciela - Montreal Institute for Astrophysical Data Analysis and Machine Learning Montréal H2V 0B3 Canada

Context. Traditionally, weak lensing cosmological surveys have been analyzed using summary statistics that were either motivated by their analytically tractable likelihoods (e.g., power spectrum) or by their ability to access some higher-order information (e.g., peak counts), but at the cost of requiring a simulation-based inference approach. In both cases, even if the statistics can be very informative, they are not designed nor guaranteed to be statistically sufficient (i.e., to capture all the cosmological information content of the data). With the rise of deep learning, however, it has becomes possible to create summary statistics that are specifically optimized to extract the full cosmological information content of the data. Yet, a fairly wide range of loss functions have been used in practice in the weak lensing literature to train such neural networks, leading to the natural question of whether a given loss should be preferred and whether sufficient statistics can be achieved in theory and in practice under these different choices. Aims. We compare different neural summarization strategies that have been proposed in the literature to identify the loss function that leads to theoretically optimal summary statistics for performing full-field cosmological inference. In doing so, we aim to provide guidelines and insights to the community to help guide future neural network-based cosmological inference analyses. Methods. We designed an experimental setup that allows us to isolate the specific impact of the loss function used to train neural summary statistics on weak lensing data at fixed neural architecture and simulation-based inference pipeline. To achieve this, we developed the sbi_lens JAX package, which implements an automatically differentiable lognormal weak lensing simulator and the tools needed to perform explicit full-field inference with a Hamiltonian Monte Carlo (HMC) sampler over this model. Using sbi_lens, we simulated a wCDM LSST Year 10 weak lens

关键词： Gravitational lensing: weak Large-scale structure of Universe Methods: statistical

来源：评论

学校读者我要写书评

暂无评论

Towards robust explanations for deep neural networks

arXiv

引用

arXiv 2020年

作者： Dombrowski, Ann-Kathrin Anders, Christopher J. Müller, Klaus-Robert Kessel, Pan Machine Learning Group Technische Universität Berlin Germany Department of Artificial Intelligence Korea University Seoul Korea Republic of Max Planck Institute for Informatics Saarbrücken Germany BIFOLD - Berlin Institute for the Foundations of Learning and Data Berlin Germany Google Research Brain team Berlin Germany

Explanation methods shed light on the decision process of black-box classifiers such as deep neural networks. But their usefulness can be compromised because they are susceptible to manipulations. With this work, we aim to enhance the resilience of explanations. We develop a unified theoretical framework for deriving bounds on the maximal manipulability of a model. Based on these theoretical insights, we present three different techniques to boost robustness against manipulation: training with weight decay, smoothing activation functions, and minimizing the Hessian of the network. Our experimental results confirm the effectiveness of these approaches. © 2020, CC BY-SA.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Transferring Traffic Predictions to Urban Regions Without Target data

Transferring Traffic Predictions to Urban Regions Without Ta...

引用

International Conference on Intelligent Transportation

作者： Stefan Schestakov Simon Gottschalk Nicolas Tempelmeier Thorben Funke Elena Demidova L3S Research Center Leibniz University Hannover Hannover Germany Volkswagen AG Commercial Vehicles Hannover Germany Data Science & Intelligent Systems (DSIS) Research Group University of Bonn and Lamarr Institute for Machine Learning and Artificial Intelligence Bonn Germany

ISBN: (数字)9798331505929

ISBN: (纸本)9798331505936

The scarcity of spatiotemporal traffic data for many urban regions significantly limits the availability of location-specific predictive models for traffic management, mobility services, and road safety. For example, whereas some cities release taxi data for several years, for many other cities traffic data is not available. Although existing transfer learning approaches aim to alleviate this problem, they require at least some data to be available in the target urban region. Traffic prediction in the regions without data remains a critical and unsolved task. To this extent, we propose Zero-ST, a novel transfer learning approach that leverages inductive road features and road topology to learn latent road representations encoding traffic pattern similarity. Our experimental evaluation on real-world data demonstrates the effectiveness of Zero-ST for the traffic speed prediction task in several urban regions. On this task, Zero-ST achieves an error reduction of at least 4.77% concerning MAE compared to the baselines.

关键词： Roads Transfer learning Urban areas Traffic control Predictive models Road safety Spatiotemporal phenomena Topology Public transportation Knowledge transfer

来源：评论

学校读者我要写书评

暂无评论

So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems

arXiv

引用

arXiv 2022年

作者： Frank, J. Thorben Unke, Oliver T. Müller, Klaus-Robert Machine Learning Group TU Berlin Berlin10587 Germany BIFOLD Berlin Institute for the Foundations of Learning and Data Germany Google Research Brain team Berlin Germany Department of Artificial Intelligence Korea University Seoul136-713 Korea Republic of Max Planck Institut für Informatik Saarbrücken66123 Germany

The application of machine learning methods in quantum chemistry has enabled the study of numerous chemical phenomena, which are computationally intractable with traditional ab-initio methods. However, some quantum mechanical properties of molecules and materials depend on non-local electronic effects, which are often neglected due to the difficulty of modeling them efficiently. This work proposes a modified attention mechanism adapted to the underlying physics, which allows to recover the relevant non-local effects. Namely, we introduce spherical harmonic coordinates (SPHCs) to reflect higher-order geometric information for each atom in a molecule, enabling a non-local formulation of attention in the SPHC space. Our proposed model SO3KRATES3 – a self-attention based message passing neural network – uncouples geometric information from atomic features, making them independently amenable to attention mechanisms. Thereby we construct spherical filters, which extend the concept of continuous filters in Euclidean space to SPHC space and serve as foundation for a spherical self-attention mechanism. We show that in contrast to other published methods, SO3KRATES is able to describe nonlocal quantum mechanical effects over arbitrary length scales. Further, we find evidence that the inclusion of higher-order geometric correlations increases data efficiency and improves generalization. SO3KRATES matches or exceeds state-of-the-art performance on popular benchmarks, notably, requiring a significantly lower number of parameters (0.25–0.4x) while at the same time giving a substantial speedup (6–14x for training and 2–11x for inference) compared to other models. © 2022, CC BY.

关键词： Molecules

来源：评论

学校读者我要写书评

暂无评论

learning Trivializing Gradient Flows for Lattice Gauge Theories

arXiv

引用

arXiv 2022年

作者： Bacchio, Simone Kessel, Pan Schaefer, Stefan Vaitl, Lorenz Computation-based Science and Technology Research Center The Cyprus Institute Nicosia Cyprus Machine Learning Group Technische Universität Berlin Berlin Germany BIFOLD—Berlin Institute for the Foundations of Learning and Data Berlin Germany John Von Neumann-Institut für Computing NIC Deutsches Elektronen-Synchrotron DESY Germany

We propose a unifying approach that starts from the perturbative construction of trivializing maps by Lüscher and then improves on it by learning. The resulting continuous normalizing flow model can be implemented using common tools of lattice field theory and requires several orders of magnitude fewer parameters than any existing machine learning approach. Specifically, our model can achieve competitive performance with as few as 14 parameters while existing deep-learning models have around 1 million parameters for SU(3) Yang–Mills theory on a 162 lattice. This has obvious consequences for training speed and interpretability. It also provides a plausible path for scaling machine-learning approaches toward realistic theories. © 2022, CC BY.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement

RecycleNet: Latent Feature Recycling Leads to Iterative Deci...

引用

IEEE Workshop on Applications of Computer Vision (WACV)

作者： Gregor Koehler Tassilo Wald Constantin Ulrich David Zimmerer Paul F. Jaeger Jörg K. H. Franke Simon Kohl Fabian Isensee Klaus H. Maier-Hein Division of Medical Image Computing German Cancer Research Center (DKFZ) Heidelberg Germany Helmholtz Information and Data Science School for Health Karlsruhe/Heidelberg Germany Helmholtz Imaging DKFZ National Center for Tumor Diseases (NCT) NCT Heidelberg a Partnership Between DKFZ University Medical Center Heidelberg Interactive Machine Learning Group DKFZ Machine Learning Lab University of Freiburg Freiburg Germany Latent Labs (***) London UK Applied Computer Vision Lab DKFZ Pattern Analysis and Learning Group Heidelberg University Hospital Heidelberg Germany

Despite the remarkable success of deep learning systems over the last decade, a key difference still remains between neural network and human decision-making: As humans, we can not only form a decision on the spot, but also ponder, revisiting an initial guess from different angles, distilling relevant information, arriving at a better decision. Here, we propose RecycleNet, a latent feature recycling method, instilling the pondering capability for neural networks to refine initial decisions over a number of recycling steps, where outputs are fed back into earlier network layers in an iterative fashion. This approach makes minimal assumptions about the neural network architecture and thus can be implemented in a wide variety of contexts. Using medical image segmentation as the evaluation environment, we show that latent feature recycling enables the network to iteratively refine initial predictions even beyond the iterations seen during training, converging towards an improved decision. We evaluate this across a variety of segmentation benchmarks and show consistent improvements even compared with top-performing segmentation methods. This allows trading increased computation time for improved performance, which can be beneficial, especially for safety-critical applications.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Reproducibility and Geometric Intrinsic Dimensionality: An Investigation on Graph Neural Network research

arXiv

引用

arXiv 2024年

作者： Hille, Tobias Stubbemann, Maximilian Hanika, Tom Knowledge & Data Engineering Group University of Kassel Kassel Germany Interdisciplinary Research Center for Information System Design University of Kassel Kassel Germany Information Systems and Machine Learning Lab University of Hildesheim Hildesheim Germany Institute of Computer Science University of Hildesheim Hildesheim Germany

Difficulties in replication and reproducibility of empirical evidences in machine learning research have become a prominent topic in recent years. Ensuring that machine learning research results are sound and reliable requires reproducibility, which verifies the reliability of research findings using the same code and data. This promotes open and accessible research, robust experimental workflows, and the rapid integration of new findings. Evaluating the degree to which research publications support these different aspects of reproducibility is one goal of the present work. For this we introduce an ontology of reproducibility in machine learning and apply it to methods for graph neural networks. Building on these efforts we turn towards another critical challenge in machine learning, namely the curse of dimensionality, which poses challenges in data collection, representation, and analysis, making it harder to find representative data and impeding the training and inference processes. Using the closely linked concept of geometric intrinsic dimension we investigate to which extend the used machine learning models are influenced by the intrinsic dimension of the data sets they are trained *** Codes 68T01 68T07 68T09 51F99 Copyright © 2024, The Authors. All rights reserved.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Reconstructing Kernel-based machine learning Force Fields with Super-linear Convergence

arXiv

引用

arXiv 2022年

作者： Blücher, Stefan Müller, Klaus-Robert Chmiela, Stefan BIFOLD Berlin Institue for the Foundations of Learning and Data Berlin10587 Germany Technische Universität Berlin Machine Learning Group Berlin10587 Germany Korea University Department of Artificial Intelligence Seoul136-713 Korea Republic of Max Planck Institute for Informatics Saarbrücken66123 Germany Google Research Brain team Berlin Germany

Kernel machines have sustained continuous progress in the field of quantum chemistry. In particular, they have proven to be successful in the low-data regime of force field reconstruction. This is because many equivariances and invariances due to physical symmetries can be incorporated into the kernel function to compensate for much larger datasets. So far, the scalability of kernel machines has however been hindered by its quadratic memory and cubical runtime complexity in the number of training points. While it is known, that iterative Krylov subspace solvers can overcome these burdens, their convergence crucially relies on effective preconditioners, which are elusive in practice. Effective preconditioners need to partially pre-solve the learning problem in a computationally cheap and numerically robust manner. Here, we consider the broad class of Nyström-type methods to construct preconditioners based on successively more sophisticated low-rank approximations of the original kernel matrix, each of which provides a different set of computational trade-offs. All considered methods aim to identify a representative subset of inducing (kernel) columns to approximate the dominant kernel spectrum. © 2022, CC BY-NC-ND.

关键词： Quantum chemistry

来源：评论

学校读者我要写书评

暂无评论

RBGNet: Ray-based grouping for 3D Object Detection

arXiv

引用

arXiv 2022年

作者： Wang, Haiyang Shi, Shaoshuai Yang, Ze Fang, Rongyao Qian, Qi Li, Hongsheng Schiele, Bernt Wang, Liwei Center for Data Science Peking University China Max Planck Institute for Informatics Germany University of Toronto Canada The Chinese University of Hong Kong Hong Kong Alibaba Group China Key Laboratory of Machine Perception MOE School of Artificial Intelligence Peking University China International Center for Machine Learning Research Peking University China

As a fundamental problem in computer vision, 3D object detection is experiencing rapid growth. To extract the point-wise features from the irregularly and sparsely distributed points, previous methods usually take a feature grouping module to aggregate the point features to an object candidate. However, these methods have not yet leveraged the surface geometry of foreground objects to enhance grouping and 3D box generation. In this paper, we propose the RBGNet framework, a voting-based 3D detector for accurate 3D object detection from point clouds. In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers. Considering the fact that foreground points are more meaningful for box estimation, we design a novel foreground biased sampling strategy in downsample process to sample more points on object surfaces and further boost the detection performance. Our model achieves state-of-the-art 3D detection performance on ScanNet V2 and SUN RGB-D with remarkable performance gains. Code will be available at https://***/Haiyang-W/RBGNet. Copyright © 2022, The Authors. All rights reserved.

关键词： Object recognition

来源：评论

学校读者我要写书评

暂无评论

Analyzing the Structure of Attention in a Transformer Language Model

arXiv

引用

arXiv 2019年

作者： Vig, Jesse Belinkov, Yonatan Palo Alto Research Center Machine Learning and Data Science Group Interaction and Analytics Lab Palo AltoCA United States Harvard John A. Paulson School of Engineering and Applied Sciences MIT Computer Science and Artificial Intelligence Laboratory CambridgeMA United States

The Transformer is a fully attention-based alternative to recurrent networks that has achieved state-of-the-art results across a range of NLP tasks. In this paper, we analyze the structure of attention in a Transformer language model, the GPT-2 small pretrained model. We visualize attention for individual instances and analyze the interaction between attention and syntax over a large corpus. We find that attention targets different parts of speech at different layer depths within the model, and that attention aligns with dependency relations most strongly in the middle layers. We also find that the deepest layers of the model capture the most distant relationships. Finally, we extract exemplar sentences that reveal highly specific patterns targeted by particular attention heads. Copyright © 2019, The Authors. All rights reserved.

关键词： Recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：