检索结果-内蒙古大学图书馆

CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2022年

作者： Abdulatif, Sherif Cao, Ruizhe Yang, Bin The Institute of Signal Processing and System Theory University of Stuttgart Germany

Convolution-augmented transformers (Conformers) are recently proposed in various speech-domain applications, such as automatic speech recognition (ASR) and speech separation, as they can capture both local and global dependencies. In this paper, we propose a conformer-based metric generative adversarial network (CMGAN) for speech enhancement (SE) in the time-frequency (TF) domain. The generator encodes the magnitude and complex spectrogram information using two-stage conformer blocks to model both time and frequency dependencies. The decoder then decouples the estimation into a magnitude mask decoder branch to filter out unwanted distortions and a complex refinement branch to further improve the magnitude estimation and implicitly enhance the phase information. Additionally, we include a metric discriminator to alleviate metric mismatch by optimizing the generator with respect to a corresponding evaluation score. Objective and subjective evaluations illustrate that CMGAN is able to show superior performance compared to state-of-the-art methods in three speech enhancement tasks (denoising, dereverberation and super-resolution). For instance, quantitative denoising analysis on Voice Bank+DEMAND dataset indicates that CMGAN outperforms various previous models with a margin, i.e., PESQ of 3.41 and SSNR of 11.10 dB. © 2022, CC BY.

关键词： Generative adversarial networks

EEG motor imagery decoding: A framework for comparative analysis with channel attention mechanisms

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wimpff, Martin Gizzi, Leonardo Zerfowski, Jan Yang, Bin Institute of Signal Processing and System Theory University of Stuttgart Germany Fraunhofer Institute for Manufacturing Engineering and Automation IPA Germany Germany

Objective The objective of this study is to investigate the application of various channel attention mechanisms within the domain of brain-computer interface (BCI) for motor imagery decoding. Channel attention mechanisms can be seen as a powerful evolution of spatial filters traditionally used for motor imagery decoding. This study systematically compares such mechanisms by integrating them into a lightweight architecture framework to evaluate their impact. Approach We carefully construct a straightforward and lightweight baseline architecture designed to seamlessly integrate different channel attention mechanisms. This approach is contrary to previous works which only investigate one attention mechanism and usually build a very complex, sometimes nested architecture. Our framework allows us to evaluate and compare the impact of different attention mechanisms under the same circumstances. The easy integration of different channel attention mechanisms as well as the low computational complexity enables us to conduct a wide range of experiments on four datasets to thoroughly assess the effectiveness of the baseline model and the attention mechanisms. Results Our experiments demonstrate the strength and generalizability of our architecture framework as well as how channel attention mechanisms can improve the performance while maintaining the small memory footprint and low computational complexity of our baseline architecture. Significance Our architecture emphasizes simplicity, offering easy integration of channel attention mechanisms, while maintaining a high degree of generalizability across datasets, making it a versatile and efficient solution for EEG motor imagery decoding within brain-computer interfaces. © 2023, CC BY.

关键词： Decoding

TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Bartler, Alexander Bender, Florian Wiewel, Felix Yang, Bin Institute of Signal Processing and System Theory University of Stuttgart Stuttgart Germany

Nowadays, deep neural networks outperform humans in many tasks. However, if the input distribution drifts away from the one used in training, their performance drops significantly. Recently published research has shown that adapting the model parameters to the test sample can mitigate this performance degradation. In this paper, we therefore propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples. Using the provided prototypes of SwAV and our derived test-time loss, we align the representation of unseen test samples with the self-supervised learned prototypes. We show the success of our method on the common benchmark dataset CIFAR10-C. © 2022, CC BY.

关键词： Deep neural networks

Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Marsden, Robert A. F., Wiewel M., Döbler Y., Yang B., Yang Institute of Signal Processing and System Theory University of Stuttgart Stuttgart Germany

In recent years, there has been tremendous progress in the field of semantic image segmentation. However, one remaining challenging problem is that segmentation models do not generalize to unseen domains. To overcome this problem, one either has to label lots of data covering the whole variety of possible domains, which is often infeasible in practice, or apply unsupervised domain adaptation (UDA), only requiring labeled source data. In this work, we focus on UDA and additionally address the case of adapting not only to a single domain, but to a sequence of target domains. This requires mechanisms preventing the model from forgetting its previously learned knowledge. To adapt a segmentation model to a target domain, we follow the idea of utilizing light-weight style transfer to convert the style of labeled source images into the style of the target domain, while retaining the source content. To mitigate the distributional shift between the source and the target domain, the model is fine-tuned on the transferred source images in a second step. Existing lightweight style transfer approaches relying on adaptive instance normalization (AdaIN) or Fourier transformation (FDA) still lack performance and do not substantially improve upon common data augmentation, such as color jittering. The reason for this is that these methods do not focus on region- or class-specific differences, but mainly capture the most salient style. Therefore, we propose a simple and light-weight framework that incorporates two class-conditional AdaIN layers. To extract the class-specific target moments needed for the transfer layers, we use unfiltered pseudo-labels, which we show to be an effective approximation compared to real labels. We extensively validate our approach (CACE) on a synthetic sequence and further propose a challenging sequence consisting of real domains. CACE outperforms existing methods visually and quantitatively. © 2022, CC BY.

关键词： Semantic Segmentation

CMGAN: Conformer-based Metric GAN for Speech Enhancement

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cao, Ruizhe Abdulatif, Sherif Yang, Bin University of Stuttgart Institute of Signal Processing and System Theory Stuttgart Germany

Recently, convolution-augmented transformer (Conformer) has achieved promising performance in automatic speech recognition (ASR) and time-domain speech enhancement (SE), as it can capture both local and global dependencies in the speech signal. In this paper, we propose a conformer-based metric generative adversarial network (CMGAN) for SE in the time-frequency (TF) domain. In the generator, we utilize two-stage conformer blocks to aggregate all magnitude and complex spectrogram information by modeling both time and frequency dependencies. The estimation of magnitude and complex spectrogram is decoupled in the decoder stage and then jointly incorporated to reconstruct the enhanced speech. In addition, a metric discriminator is employed to further improve the quality of the enhanced estimated speech by optimizing the generator with respect to a corresponding evaluation score. Quantitative analysis on Voice Bank+DEMAND dataset indicates the capability of CMGAN in outperforming various previous models with a margin, i.e., PESQ of 3.41 and SSNR of 11.10 dB. Copyright © 2022, The Authors. All rights reserved.

关键词： Generative adversarial networks

Usis: Unsupervised Semantic Image Synthesis

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Eskandar, George Abdelsamad, Mohamed Armanious, Karim Yang, Bin Institute of Signal Processing and System Theory University of Stuttgart Stuttgart Germany

Semantic Image Synthesis (SIS) is a subclass of I2I (I2) translation where a photorealistic image is synthesized from a segmentation mask. SIS has mainly been addressed as a supervised problem. However, state-of-the-art methods depend on a massive amount of labeled data and cannot be applied in an unpaired setting. On the other hand, generic unpaired I2I frameworks underperform in comparison. In this work, we propose a new framework, Unsupervised Semantic Image Synthesis USIS), as a first step toward closing the performance gap between paired and unpaired settings. We design a simple and effective learning scheme that combines the fragmented benefits of cycle losses and relationship preservation constraints. Then, we make the discovery that, contrary to I2I translation, discriminator design is crucial for label-to-image translation. To this end, we design a new discriminator with a wavelet-based encoder and a decoder to reconstruct the real images. The self-supervised reconstruction loss in the decoder prevents the encoder from overfitting on a few wavelet coefficients. We test our methodology on 3 challenging datasets and set a new standard for unpaired SIS. The generated images demonstrate significantly better diversity, quality and multimodality. © 2022, The Authors. All rights reserved.

关键词： Generative adversarial networks

Towards Pragmatic Semantic Image Synthesis for Urban Scenes

学校读者我要写书评

暂无评论

Towards Pragmatic Semantic Image Synthesis for Urban Scenes

IEEE Symposium on Intelligent Vehicle

作者： George Eskandar Diandian Guo Karim Guirguis Bin Yang Institute of Signal Processing and System Theory University of Stuttgart Stuttgart Germany Bosch Center for Artificial Intelligence Renningen Germany

The need for large amounts of training and validation data is a huge concern in scaling AI algorithms for autonomous driving. Semantic Image Synthesis (SIS), or label-to-image translation, promises to address this issue by translating semantic layouts to images, providing a controllable generation of photorealistic data. However, they require a large amount of paired data, incurring extra costs. In this work, we present a new task: given a dataset with synthetic images and labels and a dataset with unlabeled real images, our goal is to learn a model that can generate images with the content of the input mask and the appearance of real images. This new task reframes the well-known unsupervised SIS task in a more practical setting, where we leverage cheaply available synthetic data from a driving simulator to learn how to generate photorealistic images of urban scenes. This stands in contrast to previous works, which assume that labels and images come from the same domain but are unpaired during training. We find that previous unsupervised works underperform on this task, as they do not handle distribution shifts between two different domains. To bypass these problems, we propose a novel framework with two main contributions. First, we leverage the synthetic image as a guide to the content of the generated image by penalizing the difference between their high-level features on a patch level. Second, in contrast to previous works which employ one discriminator that overfits the target domain semantic distribution, we employ a discriminator for the whole image and multiscale discriminators on the image patches. Extensive comparisons on the benchmarks benchmarks GTA-V → Cityscapes and GTA-V → Mapillary show the superior performance of the proposed model against state-of-the-art on this task.

关键词：

A Comparative Study on the Potential of Unsupervised Deep Learning-based Feature Selection in Radiomics 44

学校读者我要写书评

暂无评论

A Comparative Study on the Potential of Unsupervised Deep Le...

44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022

作者： Haueise, Tobias Liebgott, Annika Yang, Bin University Hospital Tuebingen Section on Experimental Radiology Tuebingen Germany Institute of Diabetes Research and Metabolic Diseases at Helmholtz Munich Tuebingen Germany Institute of Signal Processing and System Theory University of Stuttgart Stuttgart Germany

ISBN: (数字)9781728127828

ISBN: (纸本)9781728127828

In Radiomics, deep learning-based systems for medical image analysis play an increasing role. However, due to the better explainability, feature-based systems are still preferred, especially by physicians. Often, high-dimensional data and low sample size pose different challenges (e.g. increased risk of overfitting) to machine learning systems. By removing irrelevant and redundant features from the data, feature selection is an effective way of pre-processing. The research in this study is focused on unsupervised deep learning-based methods for feature selection. Five recently proposed algorithms are compared regarding their applicability and efficiency on seven data sets in three different sample applications. It was found that deep learning-based feature selection leads to improved classification results compared to conventional methods, especially for small feature subsets. Clinical Relevance - The exploration of distinctive features and the ability to rank their importance without the need for outcome information is a potential field of application for unsupervised feature selection methods. Especially in multiparametric radiology, the number of features is increasing. The identification of new potential biomarkers is important both for treatment and prevention. © 2022 IEEE.

关键词： Feature Selection

Towards Pragmatic Semantic Image Synthesis for Urban Scenes

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Eskandar, George Guo, Diandian Guirguis, Karim Yang, Bin Institute of Signal Processing and System Theory University of Stuttgart Stuttgart Germany Bosch Center for Artificial Intelligence Renningen Germany

The need for large amounts of training and validation data is a huge concern in scaling AI algorithms for autonomous driving. Semantic Image Synthesis (SIS), or label-to-image translation, promises to address this issue by translating semantic layouts to images, providing a controllable generation of photorealistic data. However, they require a large amount of paired data, incurring extra costs. In this work, we present a new task: given a dataset with synthetic images and labels and a dataset with unlabeled real images, our goal is to learn a model that can generate images with the content of the input mask and the appearance of real images. This new task reframes the well-known unsupervised SIS task in a more practical setting, where we leverage cheaply available synthetic data from a driving simulator to learn how to generate photorealistic images of urban scenes. This stands in contrast to previous works, which assume that labels and images come from the same domain but are unpaired during training. We find that previous unsupervised works underperform on this task, as they do not handle distribution shifts between two different domains. To bypass these problems, we propose a novel framework with two main contributions. First, we leverage the synthetic image as a guide to the content of the generated image by penalizing the difference between their high-level features on a patch level. Second, in contrast to previous works which employ one discriminator that overfits the target domain semantic distribution, we employ a discriminator for the whole image and multiscale discriminators on the image patches. Extensive comparisons on the benchmarks GTA-V → Cityscapes and GTA-V → Mapillary show the superior performance of the proposed model against state-of-the-art on this task. Copyright © 2023, The Authors. All rights reserved.

关键词： Benchmarking