检索结果-内蒙古大学图书馆

Audio Enhancement for Computer Audition—An Iterative Training Paradigm Using Sample Importance

学校读者我要写书评

暂无评论

Journal of Computer science & Technology 2024年第4期39卷 895-911页

作者： Manuel Milling Shuo Liu Andreas Triantafyllopoulos Ilhan Aslan Björn W.Schuller Chair of Embedded Intelligence for Health Care and Wellbeing University of AugsburgAugsburg 86159Germany Chair of Health Informatics München rechts der IsarTechnical University of MunichMunich 81675Germany Munich Center for Machine Learning Munich 80333Germany Huawei Technologies MunichMunich 80992Germany Munich Data Science Institute Garching 85748Germany Group on Language Audio and MusicImperial College LondonLondon SW72AZU.K.

Neural network models for audio tasks,such as automatic speech recognition(ASR)and acoustic scene classification(ASC),are susceptible to noise contamination for real-life *** improve audio quality,an enhancement module,which can be developed independently,is explicitly used at the front-end of the target audio *** this paper,we present an end-to-end learning solution to jointly optimise the models for audio enhancement(AE)and the subsequent *** guide the optimisation of the AE module towards a target application,and especially to overcome difficult samples,we make use of the sample-wise performance measure as an indication of sample *** experiments,we consider four representative applications to evaluate our training paradigm,i.e.,ASR,speech command recognition(SCR),speech emotion recognition(SER),and *** applications are associated with speech and nonspeech tasks concerning semantic and non-semantic features,transient and global information,and the experimental results indicate that our proposed approach can considerably boost the noise robustness of the models,especially at low signal-to-noise ratios,for a wide range of computer audition tasks in everyday-life noisy environments.

关键词： audio enhancement computer audition joint optimisation multi-task learning voice suppression

SHADE: Deep Density-based Clustering

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Beer, Anna Weber, Pascal Miklautz, Lukas Leiber, Collin Durani, Walid Böhm, Christian Plant, Claudia Data Mining and Machine Learning University of Vienna Vienna Austria UniVie Doctoral School Computer Science Vienna Austria Database Systems and Data Mining LMU Munich Munich Germany Munich Center for Machine Learning Munich Germany UniVie Vienna Austria

Detecting arbitrarily shaped clusters in high-dimensional noisy data is challenging for current clustering methods. We introduce SHADE (Structure-preserving High-dimensional Analysis with Density-based Exploration), the first deep clustering algorithm that incorporates density-connectivity into its loss function. Similar to existing deep clustering algorithms, SHADE supports high-dimensional and large data sets with the expressive power of a deep autoencoder. In contrast to most existing deep clustering methods that rely on a centroid-based clustering objective, SHADE incorporates a novel loss function that captures density-connectivity. SHADE thereby learns a representation that enhances the separation of density-connected clusters. SHADE detects a stable clustering and noise points fully automatically without any user input. It outperforms existing methods in clustering quality, especially on data that contain non-Gaussian clusters, such as video data. Moreover, the embedded space of SHADE is suitable for visualization and interpretation of the clustering results as the individual shapes of the clusters are preserved. © 2024, CC BY.

关键词： Clustering algorithms

Quantization of Bandlimited Graph Signals

学校读者我要写书评

暂无评论

Quantization of Bandlimited Graph Signals

2023 International Conference on Sampling Theory and Applications, SampTA 2023

作者： Krahmer, Felix Lyu, He Saab, Rayan Veselovska, Anna Wang, Rongrong Technical University of Munich and Munich Center for Machine Learning Department of Mathematics & Munich Data Science Institute Garching/Munich Germany University of California San Diego Department of Mathematics & Halicioglu Data Science Institute San Diego United States Michigan State University Department of Computational Mathematics Science and Engineering Department of Mathematics East Lansing United States

ISBN: (纸本)9798350328851

Graph models and graph-based signals are becoming increasingly important in machine learning, natural sciences, and modern signal processing. In this paper, we address the problem of quantizing bandlimited graph signals. We introduce two classes of noise-shaping algorithms for graph signals that differ in their sampling methodologies. We demonstrate that these algorithms can be efficiently used to construct quantized representatives of bandlimited graph-based signals with bounded amplitude. Moreover, for one of the algorithms, we provide theoretical guarantees on the relative error between the quantized representative and the true signal. © 2023 IEEE.

关键词： Quantization (signal)

Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient 41

学校读者我要写书评

暂无评论

Double Variance Reduction: A Smoothing Trick for Composite O...

41st International Conference on machine learning, ICML 2024

作者： Di, Hao Ye, Haishan Zhang, Yueling Chang, Xiangyu Dai, Guang Tsang, Ivor W. Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University China SGIT AI Lab State Grid Corporation of China China International Business School Beijing Foreign Studies University Beijing China Singapore College of Computing and Data Science NTU Singapore

Variance reduction techniques are designed to decrease the sampling variance, thereby accelerating convergence rates of first-order (FO) and zeroth-order (ZO) optimization methods. However, in composite optimization problems, ZO methods encounter an additional variance called the coordinate-wise variance, which stems from the random gradient estimation. To reduce this variance, prior works require estimating all partial derivatives, essentially approximating FO information. This approach demands O(d) function evaluations (d is the dimension size), which incurs substantial computational costs and is prohibitive in high-dimensional scenarios. This paper proposes the Zeroth-order Proximal Double Variance Reduction (ZPDVR) method, which utilizes the averaging trick to reduce both sampling and coordinate-wise variances. Compared to prior methods, ZPDVR relies solely on random gradient estimates, calls the stochastic zeroth-order oracle (SZO) in expectation O(1) times per iteration, and achieves the optimal O(d(n+κ) log(1/ϵ)) SZO query complexity in the strongly convex and smooth setting, where κ represents the condition number and ϵ is the desired accuracy. Empirical results validate ZPDVR's linear convergence and demonstrate its superior performance over other related methods. Copyright 2024 by the author(s)

关键词： Stochastic systems

Thai Conversational Chatbot Classification Using BiLSTM and data Augmentation 1st

学校读者我要写书评

暂无评论

Thai Conversational Chatbot Classification Using BiLSTM and ...

1st International Conference on data science and Artificial Intelligence, DSAI 2023

作者： Lhasiw, Nunthawat Tanantong, Tanatorn Sanglerdsinlapachai, Nuttapong Thammasat Research Unit in Data Innovation and Artificial Intelligence Department of Computer Science Faculty of Science and Technology Thammasat University Pathum Thani Thailand Strategic Analytics Networks with Machine Learning and AI Research Team National Electronics and Computer Technology Center Pathum Thani Thailand

ISBN: (纸本)9789819979684

Chatbot platforms, e.g., Facebook and Line, have revolutionized human interaction in the digital age. In order to develop an automatic chatbot classification, there are several challenges especially for Thai chat messages. Conversational messages are usually short and ambiguous. Therefore, it is difficult to find a dataset for constructing an effective classification model. To address the limited size of the dataset, data augmentation techniques can be possibly applied. data augmentation involves generating synthetic messages by applying various transformations to existing data samples while preserving their original meaning. In this study, the size and diversity of the dataset is increased by two methods, i.e., text augmentation using word2vec from Thai2Fit and English-Thai machine translation models proposed by VISTEC. Based on the augmented messages, a Deep learning technique, BiLSTM, is used to construct a chatbot classification model. The experimental obtained results demonstrate that data augmentation can help to increase the classification performance. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd 2023.

关键词： Classification (of information)

EXACT CERTIFICATION OF (GRAPH) NEURAL NETWORKS AGAINST LABEL POISONING

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sabanayagam, Mahalakshmi Gosch, Lukas Günnemann, Stephan Ghoshdastidar, Debarghya School of Computation Information and Technology Germany Munich Data Science Institute Technical University of Munich Germany Munich Center for Machine Learning Germany

machine learning models are highly vulnerable to label flipping, i.e., the adversarial modification (poisoning) of training labels to compromise performance. Thus, deriving robustness certificates is important to guarantee that test predictions remain unaffected and to understand worst-case robustness behavior. However, for Graph Neural Networks (GNNs), the problem of certifying label flipping has so far been unsolved. We change this by introducing an exact certification method, deriving both sample-wise and collective certificates. Our method leverages the Neural Tangent Kernel (NTK) to capture the training dynamics of wide networks enabling us to reformulate the bilevel optimization problem representing label flipping into a Mixed-Integer Linear Program (MILP). We apply our method to certify a broad range of GNN architectures in node classification tasks. Thereby, concerning the worst-case robustness to label flipping: (i) we establish hierarchies of GNNs on different benchmark graphs;(ii) quantify the effect of architectural choices such as activations, depth and skip-connections;and surprisingly, (iii) uncover a novel phenomenon of the robustness plateauing for intermediate perturbation budgets across all investigated datasets and architectures. While we focus on GNNs, our certificates are applicable to sufficiently wide NNs in general through their NTK. Thus, our work presents the first exact certificate to a poisoning attack ever derived for neural networks, which could be of independent interest. The code is available at https://***/saper0/qpcert. © 2024, CC BY.

关键词： Budget control

Uncertainty Quantification For Learned ISTA 33

学校读者我要写书评

暂无评论

Uncertainty Quantification For Learned ISTA

33rd IEEE International Workshop on machine learning for Signal Processing, MLSP 2023

作者： Hoppe, Frederik Verdun, Claudio Mayrink Laus, Hannah Krahmer, Felix Rauhut, Holger Rwth Aachen University Chair of Mathematics of Information Processing Aachen Germany Technical University of Munich Department of Mathematics Munich Germany Munich Center for Machine Learning Munich Germany Munich Data Science Institute Technical University of Munich Munich Germany

ISBN: (纸本)9798350324112

Model-based deep learning solutions to inverse problems have attracted increasing attention in recent years as they bridge state-of-the-art numerical performance with interpretability. In addition, the incorporated prior domain knowledge can make the training more efficient as the smaller number of parameters allows the training step to be executed with smaller datasets. Algorithm unrolling schemes stand out among these model-based learning techniques. Despite their rapid advancement and their close connection to traditional high-dimensional statistical methods, they lack certainty estimates and a theory for uncertainty quantification is still elusive. This work provides a step towards closing this gap proposing a rigorous way to obtain confidence intervals for the LISTA estimator. © 2023 IEEE.

关键词： Deep learning

A MEMORY EFFICIENT RANDOMIZED SUBSPACE OPTIMIZATION METHOD FOR TRAINING LARGE LANGUAGE MODELS

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Yiming Zhang, Yuan Liu, Yin Yuan, Kun Wen, Zaiwen Beijing International Center for Mathematical Research Peking University Beijing China Center for Data Science Peking University Beijing China Center for Machine Learning Research Peking University Beijing China

The memory challenges associated with training Large Language Models (LLMs) have become a critical concern, particularly when using the Adam optimizer. To address this issue, numerous memory-efficient techniques have been proposed, with GaLore standing out as a notable example designed to reduce the memory footprint of optimizer states. However, these approaches do not alleviate the memory burden imposed by activations, rendering them unsuitable for scenarios involving long context sequences or large mini-batches. Moreover, their convergence properties are still not well-understood in the literature. In this work, we introduce a Randomized Subspace Optimization framework for pre-training and fine-tuning LLMs. Our approach decomposes the high-dimensional training problem into a series of lower-dimensional subproblems. At each iteration, a random subspace is selected, and the parameters within that subspace are optimized. This structured reduction in dimensionality allows our method to simultaneously reduce memory usage for both activations and optimizer states. We establish comprehensive convergence guarantees and derive rates for various scenarios, accommodating different optimization strategies to solve the subproblems. Extensive experiments validate the superior memory and communication efficiency of our method, achieving performance comparable to GaLore and Adam. Copyright © 2025, The Authors. All rights reserved.

关键词：

On the Effectiveness of Heterogeneous Ensemble Methods for Re-identification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Klüttermann, Simon Rutinowski, Jérôme Nguyen, Anh Grimme, Britta Roidl, Moritz Müller, Emmanuel TU Dortmund University Dortmund Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany Research Center Trustworthy Data Science and Security Germany

In this contribution, we introduce a novel ensemble method for the re-identification of industrial entities, using images of chipwood pallets and galvanized metal plates as dataset examples. Our algorithms replace commonly used, complex siamese neural networks with an ensemble of simplified, rudimentary models, providing wider applicability, especially in hardware-restricted scenarios. Each ensemble sub-model uses different types of extracted features of the given data as its input, allowing for the creation of effective ensembles in a fraction of the training duration needed for more complex state-of-the-art models. We reach state-of-the-art performance at our task, with a Rank-1 accuracy of over 77% and a Rank-10 accuracy of over 99%, and introduce five distinct feature extraction approaches, and study their combination using different ensemble methods. © 2024, CC BY.

关键词： Complex networks