检索结果-内蒙古大学图书馆

Uncertainty Quantification in machine learning for engineering Design and Health Prognostics: A Tutorial

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Nemani, Venkat Biggio, Luca Huan, Xun Hu, Zhen Fink, Olga Tran, Anh Wang, Yan Zhang, Xiaoge Hu, Chao Department of Mechanical Engineering Iowa State University AmesIA50011 United States Data Analytics Lab ETH Zürich Switzerland Department of Mechanical Engineering University of Michigan Ann ArborMI48109 United States Department of Industrial and Manufacturing Systems Engineering University of Michigan-Dearborn DearbornMI48128 United States Intelligent Maintenance and Operations Systems EPFL Lausanne12309 Switzerland Scientific Machine Learning Sandia National Laboratories AlbuquerqueNM87123 United States George W. Woodruff School of Mechanical Engineering Georgia Institute of Technology AtlantaGA30332 United States Department of Industrial and Systems Engineering The Hong Kong Polytechnic University Kowloon Hong Kong New Territories Hong Kong Department of Mechanical Engineering University of Connecticut StorrsCT06269 United States

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2). © 2023, CC BY.

关键词： machine learning

Weisfeiler and Leman go machine learning: the story so far

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of machine learning Research 2023年第1期24卷 15865-15923页

作者： Christopher Morris Yaron Lipman Haggai Maron Bastian Rieck Nils M. Kriege Martin Grohe Matthias Fey Karsten Borgwardt Department of Computer Science RWTH Aachen University Aachen Germany Meta AI Research Department of Computer Science and Applied Mathematics Weizmann Institute of Science Rehovot Israel NVIDIA Research Tel Aviv Israel AIDOS Lab Institute of AI for Health Helmholtz Zentrum München and Technical University of Munich Munich Germany Faculty of Computer Science and Research Network Data Science University of Vienna Vienna Austria Kumo.AI Mountain View CA Machine Learning & Computational Biology Lab Department of Biosystems Science and Engineering ETH Zürich Basel Switzerland and Swiss Institute of Bioinformatics Lausanne Switzerland

In recent years, algorithms and neural architectures based on the Weisfeiler-Leman algorithm, a well-known heuristic for the graph isomorphism problem, have emerged as a powerful tool for machine learning with graphs and relational data. Here, we give a comprehensive overview of the algorithm's use in a machine-learning setting, focusing on the supervised regime. We discuss the theoretical background, show how to use it for supervised graph and node representation learning, discuss recent extensions, and outline the algorithm's connection to (permutation-)equivariant neural architectures. Moreover, we give an overview of current applications and future directions to stimulate further research.

关键词： machine learning for graphs graph neural networks Weisfeiler-Leman algorithm expressivity equivariance

Relative energies without electronic perturbations via Alchemical Integral Transform

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Krug, Simon León von Rudorff, Guido Falk von Lilienfeld, O. Anatole University of Vienna Computational Materials Physics Kolingasse 14-16 Vienna1090 Austria Machine Learning Group Technische Universität Berlin Institute for the Foundations of Learning and Data Berlin10587 Germany University of California Los Angeles 460 Portola Plaza Los AngelesCA90095 United States Vector Institute for Artificial Intelligence TorontoONM5S 1M1 Canada Departments of Chemistry Materials Science and Engineering and Physics University of Toronto St. George Campus TorontoON Canada

We show that the energy of a perturbed system can be fully recovered from the unperturbed system's electron density. We derive an alchemical integral transform by parametrizing space in terms of transmutations, the chain rule and integration by parts. Within the radius of convergence, the zeroth order yields the energy expansion at all orders, restricting the textbook statement by Wigner that the p-th order wave function derivative is necessary to describe the (2p + 1)-th energy derivative. Without the need for derivatives of the electron density, this allows to cover entire chemical neighborhoods from just one quantum calculation instead of single systems one-by-one. Numerical evidence presented indicates that predictive accuracy is achieved in the range of mHa for the harmonic oscillator or the Morse potential, and in the range of machine accuracy for hydrogen-like atoms. Considering iso-electronic nuclear charge variations by one proton in all multi-electron atoms from He to Ne, alchemical integral transform based estimates of the relative energy deviate by only few mHa from corresponding Hartree-Fock reference numbers. © 2022, CC BY.

关键词： Carrier concentration

Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhu, Tianyu Hiller, Markus Ehsanpour, Mahsa Ma, Rongkai Drummond, Tom Reid, Ian Rezatofighi, Hamid The Department of Electrical and Computer Systems Engineering Monash University Australia The School of Computing and Information Systems The University of Melbourne Australia The Australian Institute for Machine Learning The University of Adelaide Australia The Department of Data Science and AI Monash University Australia The Australian Centre for Robotic Vision Australia

Tracking a time-varying indefinite number of objects in a video sequence over time remains a challenge despite recent advances in the field. Most existing approaches are not able to properly handle multi-object tracking challenges such as occlusion, in part because they ignore long-term temporal information. To address these shortcomings, we present MO3TR: a truly end-to-end Transformer-based online multi-object tracking (MOT) framework that learns to handle occlusions, track initiation and termination without the need for an explicit data association module or any heuristics. MO3TR encodes object interactions into long-term temporal embeddings using a combination of spatial and temporal Transformers, and recursively uses the information jointly with the input data to estimate the states of all tracked objects over time. The spatial attention mechanism enables our framework to learn implicit representations between all the objects and the objects to the measurements, while the temporal attention mechanism focuses on specific parts of past information, allowing our approach to resolve occlusions over multiple frames. Our experiments demonstrate the potential of this new approach, achieving results on par with or better than the current state-of-the-art on multiple MOT metrics for several popular multi-object tracking benchmarks. © 2021, CC BY.

关键词： Tracking (position)

Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bassi, Pedro R.A.S. Li, Wenxuan Tang, Yucheng Isensee, Fabian Wang, Zifu Chen, Jieneng Chou, Yu-Cheng Roy, Saikat Kirchhoff, Yannick Rokuss, Maximilian Huang, Ziyan Ye, Jin He, Junjun Wald, Tassilo Ulrich, Constantin Baumgartner, Michael Maier-Hein, Klaus H. Jaeger, Paul Ye, Yiwen Xie, Yutong Zhang, Jianpeng Chen, Ziyang Xia, Yong Xing, Zhaohu Zhu, Lei Sadegheih, Yousef Bozorgpour, Afshin Kumari, Pratibha Azad, Reza Merhof, Dorit Shi, Pengcheng Ma, Ting Du, Yuxin Bai, Fan Huang, Tiejun Zhao, Bo Wang, Haonan Li, Xiaomeng Gu, Hanxue Dong, Haoyu Yang, Jichen Mazurowski, Maciej A. Gupta, Saumya Wu, Linshan Zhuang, Jiaxin Chen, Hao Roth, Holger Xu, Daguang Blaschko, Matthew B. Decherchi, Sergio Cavalli, Andrea Yuille, Alan L. Zhou, Zongwei Department of Computer Science Johns Hopkins University United States Department of Pharmacy and Biotechnology University of Bologna Italy Center for Biomolecular Nanotechnologies Istituto Italiano di Tecnologia Italy NVIDIA United States Germany Germany ESAT-PSI KU Leuven Belgium Faculty of Mathematics and Computer Science Heidelberg University Germany HIDSS4Health - Helmholtz Information and Data Science School for Health Germany Shanghai Jiao Tong University China Shanghai Artificial Intelligence Laboratory China Pattern Analysis and Learning Group Department of Radiation Oncology Heidelberg University Hospital Germany DKFZ Germany School of Computer Science and Engineering Northwestern Polytechnical University China Australian Institute for Machine Learning The University of Adelaide Australia College of Computer Science and Technology Zhejiang University China Hong Kong University of Science and Technology Guangzhou China Hong Kong University of Science and Technology Hong Kong Faculty of Informatics and Data Science University of Regensburg Germany Faculty of Electrical Engineering and Information Technology RWTH Aachen University Germany Fraunhofer Institute for Digital Medicine MEVIS Germany Electronic & Information Engineering School Harbin Institute of Technology Shenzhen China China The Chinese University of Hong Kong Hong Kong Peking University China Department of Electrical and Computer Engineering Duke University United States Stony Brook University United States Department of Computer Science and Engineering Department of Chemical and Biological Engineering Division of Life Science Hong Kong University of Science and Technology Hong Kong Data Science and Computation Facility Fondazione Istituto Italiano di Tecnologia Italy Ecole Polytechnique Fédérale de Lausanne Switzerland

How can we test AI performance? This question seems trivial, but it isn’t. Standard benchmarks often have problems such as in-distribution and small-size test sets, oversimplified metrics, unfair comparisons, and short-term outcome pressure. As a consequence, good performance on standard benchmarks does not guarantee success in real-world scenarios. To address these problems, we present Touchstone, a large-scale collaborative segmentation benchmark of 9 types of abdominal organs. This benchmark is based on 5,195 training CT scans from 76 hospitals around the world and 5,903 testing CT scans from 11 additional hospitals. This diverse test set enhances the statistical significance of benchmark results and rigorously evaluates AI algorithms across out-of-distribution scenarios. We invited 14 inventors of 19 AI algorithms to train their algorithms, while our team, as a third party, independently evaluated these algorithms. In addition, we also evaluated pre-existing AI frameworks—which, differing from algorithms, are more flexible and can support different algorithms—including MONAI from NVIDIA, nnU-Net from DKFZ, and numerous other open-source frameworks. We are committed to expanding this benchmark to encourage more innovation of AI algorithms for the medical domain. Copyright © 2024, The Authors. All rights reserved.

关键词： Open systems

Evolutionary Monte Carlo of QM properties in chemical space: Electrolyte design

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Karandashev, Konstantin Weinreich, Jan Heinen, Stefan Arrieta, Daniel Jose Arismendi von Rudorff, Guido Falk Hermansson, Kersti von Lilienfeld, O. Anatole University of Vienna Faculty of Physics Kolingasse 14-16 WienAT-1090 Austria Vector Institute for Artificial Intelligence TorontoONM5S 1M1 Canada Department of Chemistry-Angström Laboratory Uppsala University Box 538 UppsalaSE-75121 Sweden University Kassel Department of Chemistry Heinrich-Plett-Str.40 Kassel34132 Germany Heinrich-Plett-Strase 40 Kassel34132 Germany Departments of Chemistry Materials Science and Engineering and Physics University of Toronto St. George Campus TorontoON Canada Machine Learning Group Technische Universität Berlin Institute for the Foundations of Learning and Data Berlin10587 Germany

Optimizing a target function over the space of organic molecules is an important problem appearing in many fields of applied science, but also a very difficult one due to the vast number of possible molecular systems. We propose an Evolutionary Monte Carlo algorithm for solving such problems which is capable of straightforwardly tuning both exploration and exploitation characteristics of an optimization procedure while retaining favourable properties of genetic algorithms. The method, dubbed MOSAiCS (Metropolis Optimization by Sampling Adaptively in Chemical Space), is tested on problems related to optimizing components of battery electrolytes, namely minimizing solvation energy in water or maximizing dipole moment while enforcing a lower bound on the HOMO-LUMO gap;optimization was done over sets of molecular graphs inspired by QM9 and Electrolyte Genome Project (EGP) datasets. MOSAiCS reliably generated molecular candidates with good target quantity values, which were in most cases better than the ones found in QM9 or EGP. While the optimization results presented in this work sometimes required up to 106 QM calculations and were thus only feasible thanks to computationally efficient ab initio approximations of properties of interest, we discuss possible strategies for accelerating MOSAiCS using machine learning approaches. Copyright © 2023, The Authors. All rights reserved.

关键词： Electrolytes

VAE-GAN based zero-shot outlier detection 4

学校读者我要写书评

暂无评论

VAE-GAN based zero-shot outlier detection

4th International Symposium on Computer Science and Intelligent Control, ISCSIC 2020

作者： Ibrahim, Bekkouch Imad Nicolae, Dragos Constantin Khan, Adil Ali, Syed Imran Khattak, Asad Machine Learning and Knowledge Representation Lab Institute of Data Science and AI Innopolis Tatarstan Russia Institutul de Cercetari Pentru Inteligenta Artificiala Mihai Draganescu Romania Department of Computer Engineering Kyung Hee University Yongin-si Korea Republic of College of Technological Innovations Zayed University Abu Dhabi United Arab Emirates

ISBN: (纸本)9781450388894

Outlier detection is one of the main fields in machine learning and it has been growing rapidly due to its wide range of applications. In the last few years, deep learning-based methods have outperformed machine learning and handcrafted outlier detection techniques, and our method is no different. We present a new twist to generative models which leverages variational autoencoders as a source for uniform distributions which can be used to separate the inliers from the outliers. Both the generative and adversarial parts of the model are used to obtain three main losses (Reconstruction loss, KL-divergence, Discriminative loss) which in return are wrapped with a one-class SVM which is used to make the predictions. We evaluated our method against several datasets both for images and tabular data and it has shown great results for the zero-shot outlier detection problem and was able to easily generalize it for supervised outlier detection tasks on which the performance has increased. For comparison, we evaluated our method against several of the common outlier detection techniques such as DBSCAN-based outlier detection, GMM, K-means and one class SVM directly, and we have outperformed all of them on all datasets. © 2020 ACM.

关键词： Anomaly detection

Towards Ground Truth Explainability on Tabular data

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Barr, Brian Xu, Ke Silva, Claudio Bertini, Enrico Reilly, Robert Bayan Bruss, C. Wittenbach, Jason D. Center for Machine Learning Capital One New YorkNY United States Tandon School of Engineering New York University NY United States Center for Data Science New York University NY United States Card Machine Learning and Technology Capital One PlanoTX United States Center for Machine Learning Capital One Maclean VA United States Center for Machine Learning Capital One CambridgeMA United States

In data science, there is a long history of using synthetic data for method development, feature selection and feature engineering. Our current interest in synthetic data comes from recent work in explainability. Today’s datasets are typically larger and more complex - requiring less interpretable models. In the setting of post hoc explainability, there is no ground truth for explanations. Inspired by recent work in explaining image classifiers that does provide ground truth, we propose a similar solution for tabular data. Using copulas, a concise specification of the desired statistical properties of a dataset, users can build intuition around explainability using controlled data sets and experimentation. The current capabilities are demonstrated on three use cases: one dimensional logistic regression, impact of correlation from informative features, impact of correlation from redundant variables. Copyright © 2020, The Authors. All rights reserved.

关键词： machine learning

Efficient hierarchical Bayesian inference for spatio-temporal regression models in neuroimaging

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hashemi, Ali Gao, Yijing Cai, Chang Ghosh, Sanjay Müller, Klaus-Robert Nagarajan, Srikantan S. Haufe, Stefan Uncertainty Inverse Modeling and Machine Learning Group Technische Universität Berlin Germany Machine Learning Group Technische Universität Berlin Germany Department of Radiology and Biomedical Imaging University of California San Francisco United States National Engineering Research Center for E-Learning Central China Normal University China BIFOLD – Berlin Institute for the Foundations of Learning and Data Berlin Germany Department of Artificial Intelligence Korea University Korea Republic of Max Planck Institute for Informatics Saarbrücken Germany Physikalisch-Technische Bundesanstalt Berlin Germany Charité – Universitätsmedizin Berlin Germany Bernstein Center for Computational Neuroscience Berlin Germany

Several problems in neuroimaging and beyond require inference on the parameters of multi-task sparse hierarchical regression models. Examples include M/EEG inverse problems, neural encoding models for task-based fMRI analyses, and temperature monitoring of climate or CPU and GPU. In these domains, both the model parameters to be inferred and the measurement noise may exhibit a complex spatio-temporal structure. Existing work either neglects the temporal structure or leads to computationally demanding inference schemes. Overcoming these limitations, we devise a novel flexible hierarchical Bayesian framework within which the spatio-temporal dynamics of model parameters and noise are modeled to have Kronecker product covariance structure. Inference in our framework is based on majorization-minimization optimization and has guaranteed convergence properties. Our highly efficient algorithms exploit the intrinsic Riemannian geometry of temporal autocovariance matrices. For stationary dynamics described by Toeplitz matrices, the theory of circulant embeddings is employed. We prove convex bounding properties and derive update rules of the resulting algorithms. On both synthetic and real neural data from M/EEG, we demonstrate that our methods lead to improved performance. © 2021, CC BY-NC-ND.

关键词： Inverse problems