检索结果-内蒙古大学图书馆

CA-SER: Cross-Attention Feature Fusion for Speech Emotion Recognition 27

学校读者我要写书评

暂无评论

CA-SER: Cross-Attention Feature Fusion for Speech Emotion Re...

27th European Conference on Artificial Intelligence, ECAI 2024

作者： Deeb, Bashar M. Savchenko, Andrey Makarov, Ilya MIPT Moscow Russia Laboratory of Algorithms and Technologies for Network Analysis HSE University Nizhny Novgorod Russia Sber AI Lab Moscow Russia Moscow Russia ISP RAS Research Center for Trusted Artificial Intelligence Moscow Russia

ISBN: (纸本)9781643685489

In this paper, we introduce a novel tool for speech emotion recognition, CA-SER, that borrows self-supervised learning to extract semantic speech representations from a pre-trained wav2vec 2.0 model and combine them with spectral audio features to improve speech emotion recognition. Our approach involves a self-attention encoder on MFCC features to capture meaningful patterns in audio sequences. These MFCC features are combined with high-level representations using a multi-head cross-attention mechanism. Evaluation of speech emotion recognition on the IEMOCAP dataset shows that our system achieves a weighted accuracy of 74.6%, outperforming most existing techniques. © 2024 The Authors.

关键词： Self-supervised learning

Dota Underlords Game Is NP-Complete

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Ponomarenko, Alexander A. Sirotkin, Dmitry V. Laboratory of Algorithms and Technologies for Network Analysis National Research University Higher School of Economics Nizhny Novgorod Russia

In this paper, we demonstrate how the problem of the optimal team choice in the popular computer game Dota Underlords can be reduced to the problem of linear integer programming. We propose a model and solve it for the real data. We also prove that this problem belongs to the NP-complete class and show that it reduces to the maximum edge weighted clique problem. Copyright © 2020, The Authors. All rights reserved.

关键词： Computer games

Optimization of Gain in Symmetrized Itakura-Saito Discrimination for Pronunciation Learning 19th

学校读者我要写书评

暂无评论

Optimization of Gain in Symmetrized Itakura-Saito Discrimina...

19th International Conference on Mathematical Optimization Theory and Operations Research, MOTOR 2020

作者： Savchenko, Andrey V. Savchenko, Vladimir V. Savchenko, Lyudmila V. Laboratory of Algorithms and Technologies for Network Analysis National Research University Higher School of Economics Nizhny Novgorod Russia Nizhny Novgorod State Linguistic University Nizhny Novgorod Russia Department of Information Systems and Technologies National Research University Higher School of Economics Nizhny Novgorod Russia

ISBN: (纸本)9783030499877

This paper considers an assessment and evaluation of the pronunciation quality in computer-aided language learning systems. We propose the novel distortion measure for speech processing by using the gain optimization of the symmetrized Itakura-Saito divergence. This dissimilarity is implemented in a complete algorithm for pronunciation learning and improvement. At its first stage, a user has to achieve a stable pronunciation of all sounds by matching them with sounds of an ideal speaker. At the second stage, the recognition of sounds and their short sequences is carried out to guarantee the distinguishability of learned sounds. The training set may contain not only ideal sounds but the best utterances of a user obtained at the previous step. Finally, the word recognition accuracy is estimated by using deep neural networks fine-tuned on the best words from a user. Experimental study shows that the proposed procedure makes it possible to achieve high efficiency for learning of sounds and their sequences even in the presence of noise in an observed utterance. © 2020, Springer Nature Switzerland AG.

关键词： Convolutional neural networks

Faster Exploration of Some Temporal Graphs 1

学校读者我要写书评

暂无评论

Faster Exploration of Some Temporal Graphs

1st Symposium on Algorithmic Foundations of Dynamic networks, SAND 2022

作者： Adamson, Duncan Gusev, Vladimir V. Malyshev, Dmitriy Zamaraev, Viktor Department of Computer Science Reykjavik University Iceland Materials Innovation Factory University of Liverpool United Kingdom Laboratory of Algorithms and Technologies for Network Analysis HSE University Russia Nizhny Novgorod Russia Department of Computer Science University of Liverpool United Kingdom

ISBN: (纸本)9783959772242

A temporal graph G = (G1, G2,..., GT ) is a graph represented by a sequence of T graphs over a common set of vertices, such that at the ith time step only the edge set Ei is active. The temporal graph exploration problem asks for a shortest temporal walk on some temporal graph visiting every vertex. We show that temporal graphs with n vertices can be explored in O(kn1.5 log n) days if the underlying graph has treewidth k and in O(n1.75 log n) days if the underlying graph is planar. Furthermore, we show that any temporal graph whose underlying graph is a cycle with k chords can be explored in at most 6kn days. Finally, we demonstrate that there are temporal realisations of sub cubic planar graphs that cannot be explored faster than in Ω(n log n) days. All these improve best known results in the literature. © Duncan Adamson, Vladimir V. Gusev, Dmitriy Malyshev, and Viktor Zamaraev;licensed under Creative Commons License CC-BY 4.0

关键词： Graphic methods

HSEmotion Team at the 6th ABAW Competition: Facial Expressions, Valence-Arousal and Emotion Intensity Prediction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Savchenko, Andrey V. Sber AI Lab Moscow Russia HSE University Laboratory of Algorithms and Technologies for Network Analysis Nizhny Novgorod Russia

This article presents our results for the sixth Affective Behavior analysis in-the-wild (ABAW) competition. To improve the trustworthiness of facial analysis, we study the possibility of using pre-trained deep models that extract reliable emotional features without the need to fine-tune the neural networks for a downstream task. In particular, we introduce several lightweight models based on MobileViT, MobileFaceNet, EfficientNet, and DDAMFN architectures trained in multi-task scenarios to recognize facial expressions, valence, and arousal on static photos. These neural networks extract frame-level features fed into a simple classifier, e.g., linear feed-forward neural network, to predict emotion intensity, compound expressions, action units, facial expressions, and valence/arousal. Experimental results for five tasks from the sixth ABAW challenge demonstrate that our approach lets us significantly improve quality metrics on validation sets compared to existing non-ensemble *** Codes 68T10 Copyright © 2024, The Authors. All rights reserved.

关键词： Feedforward neural networks

HSEmotion Team at the 7th ABAW Challenge: Multi-Task Learning and Compound Facial Expression Recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Savchenko, Andrey V. Sber AI Lab Moscow Russia HSE University Laboratory of Algorithms and Technologies for Network Analysis Nizhny Novgorod Russia

In this paper, we describe the results of the HSEmotion team in two tasks of the seventh Affective Behavior analysis in-the-wild (ABAW) competition, namely, multi-task learning for simultaneous prediction of facial expression, valence, arousal, and detection of action units, and compound expression recognition. We propose an efficient pipeline based on frame-level facial feature extractors pre-trained in multi-task settings to estimate valence-arousal and basic facial expressions given a facial photo. We ensure the privacy-awareness of our techniques by using the lightweight architectures of neural networks, such as MT-EmotiDDAMFN, MT-EmotiEffNet, and MT-EmotiMobileFaceNet, that can run even on a mobile device without the need to send facial video to a remote server. It was demonstrated that a significant step in improving the overall accuracy is the smoothing of neural network output scores using Gaussian or box filters. It was experimentally demonstrated that such a simple post-processing of predictions from simple blending of two top visual models improves the F1-score of facial expression recognition up to 7%. At the same time, the mean Concordance Correlation Coefficient (CCC) of valence and arousal is increased by up to 1.25 times compared to each model’s frame-level predictions. As a result, our final performance score on the validation set from the multi-task learning challenge is 4.5 times higher than the baseline (1.494 vs 0.32).MSC Codes 68T10 Copyright © 2024, The Authors. All rights reserved.

关键词： Multi-task learning

Leveraging Pre-trained Multi-task Deep Models for Trustworthy Facial analysis in Affective Behaviour analysis in-the-Wild

学校读者我要写书评

暂无评论

Leveraging Pre-trained Multi-task Deep Models for Trustworth...

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

作者： Andrey V. Savchenko Sber AI Lab Moscow Russia Laboratory of Algorithms and Technologies for Network Analysis HSE University Nizhny Novgorod Russia

ISBN: (数字)9798350365474

ISBN: (纸本)9798350365481

This article presents our results for the sixth Affective Behavior analysis in-the-wild (ABAW) competition. To improve the trustworthiness of facial analysis, we study the possibility of using pre-trained deep models that extract reliable emotional features without the need to fine-tune the neural networks for a downstream task. In particular, we introduce several lightweight models based on MobileViT, MobileFaceNet, EfficientNet, and DDAMFN architectures trained in multi-task scenarios to recognize facial expressions, valence, and arousal on static photos. These neural networks extract frame-level features fed into a simple classifier, e.g., linear feed-forward neural network, to predict emotion intensity, compound expressions, and valence/arousal. Experimental results for three tasks from the sixth ABAW challenge demonstrate that our approach lets us significantly improve quality metrics on validation sets compared to existing non-ensemble techniques. As a result, our solutions took second place in the compound expression recognition competition.

关键词： Training Analytical models Correlation Electromagnetic interference Estimation Predictive models Feature extraction

EmotiEffNet Facial Features in Uni-task Emotion Recognition in Video at ABAW-5 competition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Savchenko, Andrey V. Sber AI Lab Moscow Russia HSE University Laboratory of Algorithms and Technologies for Network Analysis Nizhny Novgorod Russia

In this article, the results of our team for the fifth Affective Behavior analysis in-the-wild (ABAW) competition are presented. The usage of the pre-trained convolutional networks from the EmotiEffNet family for frame-level feature extraction is studied. In particular, we propose an ensemble of a multi-layered perceptron and the LightAutoML-based classifier. The post-processing by smoothing the results for sequential frames is implemented. Experimental results for the large-scale Aff-Wild2 database demonstrate that our model achieves a much greater macro-averaged F1-score for facial expression recognition and action unit detection and concordance correlation coefficients for valence/arousal estimation when compared to *** Codes 68T10 Copyright © 2023, The Authors. All rights reserved.

关键词： Emotion Recognition

Gender domain adaptation for automatic speech recognition

学校读者我要写书评

暂无评论

Gender domain adaptation for automatic speech recognition

International Symposium on Applied Machine Intelligence and Informatics (SAMI)

作者： Artem Sokolov Andrey V. Savchenko HSE University Nizhny Novgorod Russia Laboratory of Algorithms and Technologies for Network Analysis HSE University Nizhny Novgorod Russia

ISBN: (数字)9781728180533

ISBN: (纸本)9781728180540

This paper is focused on the finetuning of acoustic models for speaker adaptation goals on a given gender. We pretrained the Transformer baseline model on Librispeech-960 and conducted experiments with finetuning on the gender-specific test subsets. The obtained word error rate (WER) relatively to the baseline is up to 5% and 3% lower on male and female subsets, respectively, if the layers in the encoder and decoder are not frozen, and the tuning is started from the last checkpoints. Moreover, we adapted our base model on the complete L2 Arctic dataset of accented speech and finetuned it for particular speakers and male and female genders separately. The models trained on the gender subsets obtained 1-2% lower WER when compared to the model tuned on the whole L2 Arctic dataset. Finally, it was experimentally confirmed that the concatenation of the pretrained voice embeddings (x-vector) and embeddings from a conventional encoder cannot significantly improve the speech recognition accuracy.

关键词： Training Adaptation models Filter banks Acoustics Decoding Arctic Tuning