检索结果-内蒙古大学图书馆

Computationally Efficient Approximations for Matrix-based Rényi's Entropy

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Gong, Tieliang Dong, Yuxin Yu, Shujian Dong, Bo The School of Computer Science and Technology Xi'an Jiaotong University Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Xi’an710049 China The Machine Learning Group UiT - The Arctic University of Norway Department of Computer Science Vrije University Amsterdam Amsterdam Netherlands

The recently developed matrix-based Rényi's αorder entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi-definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the underlying data distribution. This intriguing property makes this new information measurement widely adopted in multiple statistical inference and learning tasks. However, the computation of such quantity involves the trace operator on a PSD matrix G to power α (i.e., tr(Gα)), with a normal complexity of nearly O(n3), which severely hampers its practical usage when the number of samples (i.e., n) is large. In this work, we present computationally efficient approximations to this new entropy functional that can reduce its complexity to even significantly less than O(n2). To this end, we leverage the recent progress on Randomized Numerical Linear Algebra, developing Taylor, Chebyshev and Lanczos approximations to tr(Gα) for arbitrary values of α by converting it into a matrix-vector multiplication problem. We also establish the connection between the matrix-based Rényi's entropy and PSD matrix approximation, which enables exploiting both clustering and block low-rank structure of G to further reduce the computational cost. We theoretically provide approximation accuracy guarantees and illustrate the properties for different approximations. Large-scale experimental evaluations on both synthetic and real-world data corroborate our theoretical findings, showing promising speedup with negligible loss in accuracy. Copyright © 2021, The Authors. All rights reserved.

关键词： Matrix algebra

Hybridizing Target- and SHAP-encoded Features for Algorithm Selection in Mixed-variable Black-box Optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Dietrich, Konstantin Prager, Raphael Patrick Doerr, Carola Trautmann, Heike Big Data Analytics in Transportation TU Dresden Germany ScaDS.AI Dresden Germany Data Science: Statistics and Optimization University of Münster Germany Sorbonne Université CNRS LIP6 Paris France Machine Learning and Optimisation Paderborn University Germany Data Management and Biometrics Group University of Twente Netherlands

Exploratory landscape analysis (ELA) is a well-established tool to characterize optimization problems via numerical features. ELA is used for problem comprehension, algorithm design, and applications such as automated algorithm selection and configuration. Until recently, however, ELA was limited to search spaces with either continuous or discrete variables, neglecting problems with mixed variable types. This gap was addressed in a recent study that uses an approach based on target-encoding to compute exploratory landscape features for mixed-variable problems. In this work, we investigate an alternative encoding scheme based on SHAP values. While these features do not lead to better results in the algorithm selection setting considered in previous work, the two different encoding mechanisms exhibit complementary performance. Combining both feature sets into a hybrid approach outperforms each encoding mechanism individually. Finally, we experiment with two different ways of meta-selecting between the two feature sets. Both approaches are capable of taking advantage of the performance complementarity of the models trained on target-encoded and SHAP-encoded feature sets, respectively. Copyright © 2024, The Authors. All rights reserved.

关键词： Optimization

EXPRESSIVITY AND SPEECH SYNTHESIS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Triantafyllopoulos, Andreas Schuller, Björn W. Technical University of Munich MRI Munich Germany GLAM – Group on Language Audio & Music Imperial College London United Kingdom MCML – Munich Center for Machine Learning Munich Germany MDSI – Munich Data Science Institute Munich Germany

Imbuing machines with the ability to talk has been a longtime pursuit of artificial intelligence (AI) research. From the very beginning, the community has not only aimed to synthesise high-fidelity speech that accurately conveys the semantic meaning of an utterance, but also to colour it with inflections that cover the same range of affective expressions that humans are capable of. After many years of research, it appears that we are on the cusp of achieving this when it comes to single, isolated utterances. This unveils an abundance of potential avenues to explore when it comes to combining these single utterances with the aim of synthesising more complex, longer-term behaviours. In the present chapter, we outline the methodological advances that brought us so far and sketch out the ongoing efforts to reach that coveted next level of artificial expressivity. We also discuss the societal implications coupled with rapidly advancing expressive speech synthesis (ESS) technology and highlight ways to mitigate those risks and ensure the alignment of ESS capabilities with ethical norms. Copyright © 2024, The Authors. All rights reserved.

关键词： Speech synthesis

Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Bender, Sidney Anders, Christopher J. Chormai, Pattarawat Marxfeld, Heike Herrmann, Jan Montavon, Grégoire Machine Learning Group Technische Universität Berlin Germany BIFOLD – Berlin Institute for the Foundations of Learning and Data Berlin Germany Max Planck School of Cognition Leipzig Germany BASF SE Ludwigshafen am Rhein Germany Department of Mathematics and Computer Science Freie Universität Berlin Germany

This paper introduces a novel technique called counterfactual knowledge distillation (CFKD) to detect and remove reliance on confounders in deep learning models with the help of human expert feedback. Confounders are spurious features that models tend to rely on, which can result in unexpected errors in regulated or safety-critical domains. The paper highlights the benefit of CFKD in such domains and shows some advantages of counterfactual explanations over other types of explanations. We propose an experiment scheme to quantitatively evaluate the success of CFKD and different teachers that can give feedback to the model. We also introduce a new metric that is better correlated with true test performance than validation accuracy. The paper demonstrates the effectiveness of CFKD on synthetically augmented datasets and on real-world histopathological datasets. © 2023, CC0.

关键词： Distillation

Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation

学校读者我要写书评

暂无评论

Towards Fixing Clever-Hans Predictors with Counterfactual Kn...

International Conference on Computer Vision Workshops (ICCV Workshops)

作者： Sidney Bender Christopher J. Anders Pattarawat Chormai Heike Marxfeld Jan Herrmann Grégoire Montavon Machine Learning Group Technische Universität Berlin Germany BIFOLD – Berlin Institute for the Foundations of Learning and Data Berlin Germany Max Planck School of Cognition Leipzig Germany BASF SE Ludwigshafen am Rhein Germany Department of Mathematics and Computer Science Freie Universität Berlin Germany

关键词：

SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Harder, Paula Pfreundt, Franz-Josef Keuper, Margret Keuper, Janis Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Scientic Computing University of Kaiserslautern Kaiserlautern Germany Fraunhofer Center Machine Learning Germany Data and Web Science Group University of Mannheim Germany Offenburg University Germany

—Despite the success of convolutional neural networks (CNNs) in many computer vision and image analysis tasks, they remain vulnerable against so-called adversarial attacks: Small, crafted perturbations in the input images can lead to false predictions. A possible defense is to detect adversarial examples. In this work, we show how analysis in the Fourier domain of input images and feature maps can be used to distinguish benign test samples from adversarial images. We propose two novel detection methods: Our first method employs the magnitude spectrum of the input images to detect an adversarial attack. This simple and robust classifier can successfully detect adversarial perturbations of three commonly used attack methods. The second method builds upon the first and additionally extracts the phase of Fourier coefficients of feature-maps at different layers of the network. With this extension, we are able to improve adversarial detection rates compared to state-of-the-art detectors on five different attack methods. © 2021, CC BY.

关键词： Image classification

Analyzing the Structure of Attention in a Transformer Language Model

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Vig, Jesse Belinkov, Yonatan Palo Alto Research Center Machine Learning and Data Science Group Interaction and Analytics Lab Palo AltoCA United States Harvard John A. Paulson School of Engineering and Applied Sciences MIT Computer Science and Artificial Intelligence Laboratory CambridgeMA United States

The Transformer is a fully attention-based alternative to recurrent networks that has achieved state-of-the-art results across a range of NLP tasks. In this paper, we analyze the structure of attention in a Transformer language model, the GPT-2 small pretrained model. We visualize attention for individual instances and analyze the interaction between attention and syntax over a large corpus. We find that attention targets different parts of speech at different layer depths within the model, and that attention aligns with dependency relations most strongly in the middle layers. We also find that the deepest layers of the model capture the most distant relationships. Finally, we extract exemplar sentences that reveal highly specific patterns targeted by particular attention heads. Copyright © 2019, The Authors. All rights reserved.

关键词： Recurrent neural networks

Myocarditis Diagnosis: A Method using Mutual learning-Based ABC and Reinforcement learning

学校读者我要写书评

暂无评论

Myocarditis Diagnosis: A Method using Mutual Learning-Based ...

International Symposium on Computational Intelligence and Informatics

作者： Saba Danaei Arsam Bostani Seyed Vahid Moravvej Fardin Mohammadi Roohallah Alizadehsani Afshin Shoeibi Hamid Alinejad-Rokny Saeid Nahavandi Adiban Institute of Higher Education Semnan Iran Department of mechanical engineering of biosystems Urmia university Department of exercise physiology & health science University of tehran Internship in UNSW BioMedical Machine Learning Lab Sydney NSW Australia Institute for Intelligent Systems Research and Innovation (IISRI) Deakin University Waurn Ponds Victoria Australia UNSW Data Science Hub The University of New South Wales (UNSW Sydney) Sydney New South Wales Australia BioMedical Machine Learning Lab The Graduate School of Biomedical Engineering UNSW Sydney Sydney NSW Australia

ISBN: (纸本)9798350398830

Myocarditis occurs when the heart muscle becomes inflamed and inflammation occurs when your body’s immune system responds to infections. It can be diagnosed using cardiac magnetic resonance image (MRI), a non-invasive imaging technique with the possibility of operator bias. This paper proposes a hybrid method of deep reinforcement learning-based algorithms and meta-heuristics algorithms. A mutual learning-based artificial bee colony (ML-ABC) is employed for initial weight, which adjusts the candidate food source generated with the higher fitness between two individuals determined by a mutual learning factor. Moreover, a sequential decision-making process investigates the imbalanced classification issue, in which a convolutional neural network (CNN) is used as the foundation for policy architecture. At first, initial weights are produced using the ML-ABC algorithm. After that, the agent receives a sample at each phase and classifies it, obtaining environmental rewards. The minority class receives more rewards than the majority class. Eventually, the agent discovers an ideal strategy with the aid of a specific reward function and a beneficial learning environment. We evaluate our proposed approach on the Z-Alizadeh Sani myocarditis dataset based on standard criteria and demonstrate that the proposed method gives superior myocarditis diagnosis performance.

关键词： Mechatronics Computational modeling Metaheuristics Reinforcement learning Myocardium Markov processes Classification algorithms

XAI for Transformers: Better Explanations through Conservative Propagation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ali, Ameen Ali Schnake, Thomas Eberle, Oliver Montavon, Grégoire Müller, Klaus-Robert Wolf, Lior The School of Computer Science Tel-Aviv University Israel Machine Learning Group Technische Universität Berlin Berlin Germany BIFOLD Berlin Institute for the Foundations of Learning and Data Berlin Germany Department of Artificial Intelligence Korea University Seoul Korea Republic of Max Planck Institute for Informatics Saarbrücken Germany

Transformers have become an important workhorse of machine learning, with numerous applications. This necessitates the development of reliable methods for increasing their transparency. Multiple interpretability methods, often based on gradient information, have been proposed. We show that the gradient in a Transformer reflects the function only locally, and thus fails to reliably identify the contribution of input features to the prediction. We identify Attention Heads and LayerNorm as the main reasons for such unreliable explanations and propose a more stable way for propagation through these layers. Our proposal, which can be seen as a proper extension of the well-established LRP method to Transformers, is shown both theoretically and empirically to overcome the deficiency of a simple gradient-based approach, and achieves state-of-the-art explanation performance on a broad range of Transformer models and datasets. Copyright © 2022, The Authors. All rights reserved.

关键词： machine learning