检索结果-内蒙古大学图书馆

Adaptation of Tacotron2-based text-to-speech for articulatory-to-acoustic mapping using ultrasound tongue imaging

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zainkó, Csaba Tóth, László Shandiz, Amin Honarmandi Gosztolya, Gábor Markó, Alexandra Németh, Géza Csapó, Tamás Gábor Department of Telecommunications and Media Informatics Budapest University of Technology and Economics Budapest Hungary Institute of Informatics University of Szeged Hungary MTA-SZTE Research Group on Artificial Intelligence Szeged Hungary Department of Applied Linguistics and Phonetics Eötvös Loránd University Budapest Hungary MTA-ELTE Lendület Lingual Articulation Research Group Budapest Hungary

For articulatory-to-acoustic mapping, typically only limited parallel training data is available, making it impossible to apply fully end-to-end solutions like Tacotron2. In this paper, we experimented with transfer learning and adaptation of a Tacotron2 text-to-speech model to improve the final synthesis quality of ultrasound-based articulatory-to-acoustic mapping with a limited database. We use a multi-speaker pre-trained Tacotron2 TTS model and a pre-trained WaveGlow neural vocoder. The articulatory-to-acoustic conversion contains three steps: 1) from a sequence of ultrasound tongue image recordings, a 3D convolutional neural network predicts the inputs of the pre-trained Tacotron2 model, 2) the Tacotron2 model converts this intermediate representation to an 80-dimensional mel-spectrogram, and 3) the WaveGlow model is applied for final inference. This generated speech contains the timing of the original articulatory data from the ultrasound recording, but the F0 contour and the spectral information is predicted by the Tacotron2 model. The F0 values are independent of the original ultrasound images, but represent the target speaker, as they are inferred from the pre-trained Tacotron2 model. In our experiments, we demonstrated that the synthesized speech quality is more natural with the proposed solutions than with our earlier model. © 2021, CC BY.

关键词： Ultrasonics

Speech synthesis from text and ultrasound tongue image-based articulatory input

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Csapó, Tamás Gábor Tóth, László Gosztolya, Gábor Markó, Alexandra Department of Telecommunications and Media Informatics Budapest University of Technology and Economics Budapest Hungary MTA-ELTE Lendület Lingual Articulation Research Group Budapest Hungary Institute of Informatics University of Szeged Hungary MTA-SZTE Research Group on Artificial Intelligence Szeged Hungary Department of Applied Linguistics and Phonetics Eötvös Loránd University Budapest Hungary

Articulatory information has been shown to be effective in improving the performance of HMM-based and DNN-based text-to-speech synthesis. Speech synthesis research focuses traditionally on text-to-speech conversion, when the input is text or an estimated linguistic representation, and the target is synthesized speech. However, a research field that has risen in the last decade is articulation-to-speech synthesis (with a target application of a Silent Speech Interface, SSI), when the goal is to synthesize speech from some representation of the movement of the articulatory organs. In this paper, we extend traditional (vocoder-based) DNN-TTS with articulatory input, estimated from ultrasound tongue images. We compare text-only, ultrasound-only, and combined inputs. Using data from eight speakers, we show that that the combined text and articulatory input can have advantages in limited-data scenarios, namely, it may increase the naturalness of synthesized speech compared to single text input. Besides, we analyze the ultrasound tongue recordings of several speakers, and show that misalignments in the ultrasound transducer positioning can have a negative effect on the final synthesis performance. © 2021, CC BY.

关键词： Speech synthesis

Neural speaker embeddings for ultrasound-based silent speech interfaces

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Shandiz, Amin Honarmandi Tóth, László Gosztolya, Gábor Markó, Alexandra Csapó, Tamás Gábor Institute of Informatics University of Szeged Hungary MTA-SZTE Research Group on Artificial Intelligence Szeged Hungary Department of Applied Linguistics and Phonetics Eötvös Loránd University Budapest Hungary MTA-ELTE Lendület Lingual Articulation Research Group Budapest Hungary Department of Telecommunications and Media Informatics Budapest University of Technology and Economics Budapest Hungary

Articulatory-to-acoustic mapping seeks to reconstruct speech from a recording of the articulatory movements, for example, an ultrasound video. Just like speech signals, these recordings represent not only the linguistic content, but are also highly specific to the actual speaker. Hence, due to the lack of multi-speaker data sets, researchers have so far concentrated on speaker-dependent modeling. Here, we present multi-speaker experiments using the recently published TaL80 corpus. To model speaker characteristics, we adjusted the x-vector framework popular in speech processing to operate with ultrasound tongue videos. Next, we performed speaker recognition experiments using 50 speakers from the corpus. Then, we created speaker embedding vectors and evaluated them on the remaining speakers. Finally, we examined how the embedding vector influences the accuracy of our ultrasound-to-speech conversion network in a multi-speaker scenario. In the experiments we attained speaker recognition error rates below 3%, and we also found that the embedding vectors generalize nicely to unseen speakers. Our first attempt to apply them in a multi-speaker silent speech framework brought about a marginal reduction in the error rate of the spectral estimation step. © 2021, CC BY.

关键词： Vectors

Initial concept of an oracle-structured stream compression protocol for arbitrary network flows 25

学校读者我要写书评

暂无评论

Initial concept of an oracle-structured stream compression p...

25th European Wireless Conference 2019: Wireless Futures in the Era of Network Programmability, EW 2019

作者： Tömösközi, Máté Luo, Mingchuan Fitzek, Frank H.P. Ekler, Péter Communication Networks Group Technische Universität Dresden Germany Department of Automation and Applied Informatics Budapest University of Technology and Economics Hungary

ISBN: (纸本)9783800749492

Next generation use-cases of wireless networks require a great deal of flexibility in order to adopt to the constantly changing state of innovation and to accommodate future application requirements. Header compression has been an ever present solution since the advent of wireless networks and the most current version of it, Robust Header Compression (RoHC), has seen a widespread adoption in Long Term Evolution (LTE) cellular networks. Recent research has mostly focused on the integration and enhancement of RoHC, instead of advancing the core concept of the compression. In this paper we present for the first time a novel design that can tackle the compression of arbitrary packet streams regardless of the employed protocols and the transmitted data. We present our initial findings for an error-free scenario with various simulated and real-life packet streams and show that even with the absence of design time knowledge about the packet structures, one can compress a significant part of the streams, yielding compression gains for IP packets up to 90 %, as an example. © VDE VERLAG GMBH.

关键词： Wireless networks

Vision Transformer for Parkinson’s Disease Classification using Multilingual Sustained Vowel Recordings

学校读者我要写书评

暂无评论

Vision Transformer for Parkinson’s Disease Classification u...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Daria Hemmerling Marek Wodzinski Juan Rafael Orozco-Arroyave David Sztaho Mateusz Daniol Pawel Jemiolo Magdalena Wojcik-Pedziwiatr Daria Hemmerling Marek Wodzinski Mateusz Daniol and Pawel Jemiolo are With the AGH University of Science and Technology Faculty of Electrical Engineering Automatics Computer Science and Biomedical Engineering Krakow Poland Information Systems Institute University of Applied Sciences Western Switzerland (HES-SO Valais) Sierre Switzerland Universidad de Antioquia Medellin Colombia and With the Pattern Recognition Lab at the University of Erlangen Erlangen Germany Department of Telecommunications and Media Informatics Budapest University of Technology and Economics Hungary Department of Neurology Andrzej Frycz Modrzewski Krakow University Krakow Poland

Parkinson’s disease (PD) is the 2 nd most prevalent neurodegenerative disease in the world. Thus, the early detection of PD has recently been the subject of several scientific and commercial studies. In this paper, we propose a pipeline using Vision Transformer applied to mel-spectrograms for PD classification using multilingual sustained vowel recordings. Furthermore, our proposed transformed-based model shows a great potential to use voice as a single modality biomarker for automatic PD detection without language restrictions, a wide range of vowels, with an F1-score equal to 0.78. The results of our study fall within the range of the estimated prevalence of voice and speech disorders in Parkinson’s disease, which ranges from 70-90%. Our study demonstrates a high potential for adaptation in clinical decision-making, allowing for increasingly systematic and fast diagnosis of PD with the potential for use in *** relevance— There is an urgent need to develop non invasive biomarker of Parkinson’s disease effective enough to detect the onset of the disease to introduce neuroprotective treatment at the earliest stage possible and to follow the results of that intervention. Voice disorders in PD are very frequent and are expected to be utilized as an early diagnostic biomarker. The voice analysis using deep neural networks open new opportunities to assess neurodegenerative diseases’ symptoms, for fast diagnosis-making, to guide treatment initiation, and risk prediction. The detection accuracy for voice biomarkers according to our method reached close to the maximum achievable value.

关键词：

3D head-mounted projection display using retro-reflective screen

学校读者我要写书评

暂无评论

3D head-mounted projection display using retro-reflective sc...

Digital Holography and Three-Dimensional Imaging, DH 2019

作者： Sulyok, Ábel Héricz, Dalma Sarkadi, Tamás Kovács, Viktor Koppa, Pál Department of Atomic Physics Budapest University of Technology and Economics Budafoki út 8 BudapestH-1111 Hungary Department of Automation and Applied Informatics Budapest University of Technology and Economics Bldg Q Magyar tudósok krt. 2. BudapestH-1111 Hungary

We propose a compact 3D display using two head-mounted picoprojectors projecting images on a retro-reflective screen that reflects left and right images to the appropriate eye. Key system properties - brightness and c... 详细信息

ISBN: (纸本)9781943580590

关键词： Three dimensional displays

Uniform Dispersal of Cheap Flying Robots

学校读者我要写书评

暂无评论

Uniform Dispersal of Cheap Flying Robots

International Conference on Cognitive Infocommunications (CogInfoCom)

作者： Attila Hideg László Blázovics Bertalan Forstner Department of Automation and Applied Informatics Budapest University of Technology and Economics Budapest Hungary

ISBN: (数字)9781728147932

ISBN: (纸本)9781728147949

In the uniform dispersal problem (or filling problem) robots enter from one or multiple an entry point called the Doors, and have to disperse to cover the whole area. The robots are injected one-by-one into an unknown area. The robots are homogeneous, anonymous, autonomous, have limited visibility radius, limited persistent memory, and cannot use explicit communication. In a previous paper the authors investigated how 'weak' those robots can be in terms of hardware requirements and still be able to solve the problem in general areas. As the algorithm did not need any special requirement, in this paper the possibilities of usage in three-dimensional areas is investigated. The final goal is to solve the uniform dispersal with flying robots, e.g. drones or quadcopters, with constant amount of memory, minimal visibility, and no communication with linear runtime. In this paper an algorithm is presented which satisfies that criteria. Then it is also shown how it can be used in case of multiple entry points.

关键词： Drones Robot kinematics Robot sensing systems Dispersion Computational modeling Observers

Improving neural silent speech interface models by adversarial training

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Shandiz, Amin Honarmandi Tóth, László Gosztolya, Gábor Markó, Alexandra Csapó, Tamás Gábor Institute of Informatics University of Szeged Hungary MTA-SZTE Research Group on Artificial Intelligence Hungary Department of Telecommunications and Media Informatics Budapest University of Technology and Economics Hungary MTA-ELTE Lendulet Lingual Articulation Research Group Hungary Dept. of Applied Linguistics and Phonetics Eötvös Loránd University Hungary

Besides the well-known classification task, these days neural networks are frequently being applied to generate or transform data, such as images and audio signals. In such tasks, the conventional loss functions like the mean squared error (MSE) may not give satisfactory results. To improve the perceptual quality of the generated signals, one possibility is to increase their similarity to real signals, where the similarity is evaluated via a discriminator network. The combination of the generator and discriminator nets is called a Generative Adversarial Network (GAN). Here, we evaluate this adversarial training framework in the articulatory-to-acoustic mapping task, where the goal is to reconstruct the speech signal from a recording of the movement of articulatory organs. As the generator, we apply a 3D convolutional network that gave us good results in an earlier study. To turn it into a GAN, we extend the conventional MSE training loss with an adversarial loss component provided by a discriminator network. As for the evaluation, we report various objective speech quality metrics such as the Perceptual Evaluation of Speech Quality (PESQ), and the Mel-Cepstral Distortion (MCD). Our results indicate that the application of the adversarial training loss brings about a slight, but consistent improvement in all these metrics. © 2021, CC BY.

关键词： Generative adversarial networks

Deep learning based chatbot models scientific students’ associations report

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Csáky, Richárd Krisztián BUDAPEST UNIVERSITY OF TECHNOLOGY AND ECONOMICS FACULTY OF ELECTRICAL ENGINEERING AND INFORMATICS DEPARTMENT OF AUTOMATION AND APPLIED INFORMATICS

A conversational agent (chatbot) is a piece of software that is able to communicate with humans using natural language. Modeling conversation is an important task in natural language processing and artificial intelligence (AI). Indeed, ever since the birth of AI, creating a good chatbot remains one of the field’s hardest challenges. While chatbots can be used for various tasks, in general they have to understand users’ utterances and provide responses that are relevant to the problem at hand. In the past, methods for constructing chatbot architectures have relied on hand-written rules and templates or simple statistical methods. With the rise of deep learning these models were quickly replaced by end-to-end trainable neural networks around 2015. More specifically, the recurrent encoder-decoder model [Cho et al., 2014] dominates the task of conversational modeling. This architecture was adapted from the neural machine translation domain, where it performs extremely well. Since then a multitude of variations [Serban et al., 2016] and features were presented that augment the quality of the conversation that chatbots are capable of. In my work, I conduct an in-depth survey of recent literature, examining over 70 publications related to chatbots published in the last 3 years. Then I proceed to make the argument that the very nature of the general conversation domain demands approaches that are different from current state-of-the-art architectures. Based on several examples from the literature I show why current chatbot models fail to take into account enough priors when generating responses and how this affects the quality of the conversation. In the case of chatbots these priors can be outside sources of information that the conversation is conditioned on like the persona [Li et al., 2016a] or mood of the conversers. In addition to presenting the reasons behind this problem, I propose several ideas on how it could be remedied. The next section of my paper focuses on ada

关键词： Modeling languages