检索结果-内蒙古大学图书馆

Combination of Translation and Rotation in Dual Quaternion Space for Temporal Knowledge Graph Completion

学校读者我要写书评

暂无评论

Combination of Translation and Rotation in Dual Quaternion S...

International Joint Conference on Neural Networks (IJCNN)

作者： Ruiguo Yu Tao Liu Jian Yu Wenbin Zhang Yue Zhao Ming Yang Mankun Zhao Jiujiang Guo College of Intelligence and Computing Tianjin University Tianjin China Tianjin Key Laboratory of Advanced Networking(TANK Lab) Tianjin China Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China Information and Network Center Tianjin University Tianjin China College of Computing and Software Engineering Kennesaw State University Marietta GA USA

Compared with static knowledge graphs (KGs) temporal KGs record the dynamic relations between entities over time, therefore, research on temporal Knowledge Graph Completion (KGC) attracts much attention. Temporal KGs exhibit complex temporal relation patterns, such as multiple relations. However, existing methods can hardly model all the relation patterns and apply to the temporal KGs. In this paper, we propose a novel temporal KGC method that Combining Translation and Rotation (ComTR) in Dual Quaternion Space for temporal KGC. Specifically, we use dual-quaternion-based multiplication to model timestamps and relations as the combination of translation and rotation operations. We analyze the relation patterns of temporal KGs in detail and demonstrate that our method can model all the relation patterns in temporal KGs. Empirically, we show that ComTR can achieve the state-of-the-art performances over four temporal KGC benchmarks datasets.

关键词：

MASZSL: A Multi-Block Attention-Based Description Generative Adversarial Network for Knowledge Graph Zero-Shot Relational Learning

学校读者我要写书评

暂无评论

MASZSL: A Multi-Block Attention-Based Description Generative...

International Joint Conference on Neural Networks (IJCNN)

作者： Mei Yu Pengtao Fan Mankun Zhao Wenbin Zhang Yue Zhao Ming Yang Jian Yu College of Intelligence and Computing Tianjin University Tianjin China Tianjin Key Laboratory of Advanced Networking(TANK Lab) Tianjin China Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China Information and Network Center Tianjin University Tianjin China College of Computing and Software Engineering Kennesaw State University Marietta GA USA

In the real world, the Knowledge Graph(KG) is dynamic and new entities are added at any time. Therefore, open-world Knowledge Graph Completion(KGC) was proposed to approach new-added entities, but previous approaches often introduced too much noise when introducing external text resources for new entities. To alleviate this problem, knowledge graph zero-shot relational learning (KGZSL) has recently attracted more attention. Generative Adversarial Networks (GANs) are frequently used in KGZSL to connect existing relation descriptions to the domain of knowledge graphs. However, these methods ignore the impact of existing entities on embeddings for unidentified relational representations and in-stead concentrate on examining the connection between relational textual texts and knowledge network structures. In this work, we propose a multi-block attention framework using relation Description Generative Adversarial Networks (desGAN) jointing KG and text representation and address model collapse and training stability problems in previous studies. The core idea of our method is to obtain the background knowledge graph information and the relation representation through and multi-block attention layer and the desGAN, then a connection between the structured KG semantic space and the unstructured text semantic space of the new entity is established, forcing the entity pair to be closer to their real relation. Experimental results on the knowledge graph zeroshot relational learning dataset demonstrate that our MASZSL has a faster convergence speed and achieves state-of-the-art performance on this task.

关键词：

Characteristic-Specific Partial Fine-Tuning for Efficient Emotion and Speaker Adaptation in Codec Language Text-to-Speech Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Tianrui Ge, Meng Gong, Cheng Qiang, Chunyu Wang, Haoyu Huang, Zikang Jiang, Yu Wang, Xiaobao Chen, Xie Wang, Longbiao Dang, Jianwu Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Guangdong Laboratory of Artificial Intelligence and Digital Economy Guangdong China China Telecom Beijing China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China Co. Ltd Tianjin China Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong China

Recently, emotional speech generation and speaker cloning have garnered significant interest in text-to-speech (TTS). With the open-sourcing of codec language TTS models trained on massive datasets with large-scale parameters, adapting these general pre-trained TTS models to generate speech with specific emotional expressions and target speaker characteristics has become a topic of great attention. Common approaches, such as full and adapter-based fine-tuning, often overlook the specific contributions of model parameters to emotion and speaker control. Treating all parameters uniformly during fine-tuning, especially when the target data has limited content diversity compared to the pre-training corpus, results in slow training speed and an increased risk of catastrophic forgetting. To address these challenges, we propose a characteristic-specific partial fine-tuning strategy, short as CSP-FT. First, we use a weighted-sum approach to analyze the contributions of different Transformer layers in a pre-trained codec language TTS model for emotion and speaker control in the generated speech. We then selectively fine-tune the layers with the highest and lowest characteristic-specific contributions to generate speech with target emotional expression and speaker identity. Experimental results demonstrate that our method achieves performance comparable to, or even surpassing, full fine-tuning in generating speech with specific emotional expressions and speaker identities. Additionally, CSP-FT delivers approximately 2× faster training speeds, fine-tunes only around 8% of parameters, and significantly reduces catastrophic forgetting. Furthermore, we show that codec language TTS models perform competitively with self-supervised models in speaker identification and emotion classification tasks, offering valuable insights for developing universal speech processing models. © 2025, CC BY.

关键词： C (programming language)

Progressive Residual Extraction based Pre-training for Speech Representation Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Tianrui Li, Jin Ma, Ziyang Cao, Rui Chen, Xie Wang, Longbiao Ge, Meng Wang, Xiaobao Wang, Yuguang Dang, Jianwu Tashi, Nyima The Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin300350 China Company Ltd. Tianjin300350 China Saw Swee Hock School of Public Health National University of Singapore Singapore MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China Co. Ltd. Tianjin China The Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong China The School of Information Science and Technology Tibet University Lhasa China

Self-supervised learning (SSL) has garnered significant attention in speech processing, excelling in linguistic tasks such as speech recognition. However, jointly improving the performance of pre-trained models on various downstream tasks, each requiring different speech information, poses significant challenges. To this purpose, we propose a progressive residual extraction based self-supervised learning method, named PROGRE. Specifically, we introduce two lightweight and specialized task modules into an encoder-style SSL backbone to enhance its ability to extract pitch variation and speaker information from speech. Furthermore, to prevent the interference of reinforced pitch variation and speaker information with irrelevant content information learning, we residually remove the information extracted by these two modules from the main branch. The main branch is then trained using HuBERT's speech masking prediction to ensure the performance of the Transformer's deep-layer features on content tasks. In this way, we can progressively extract pitch variation, speaker, and content representations from the input speech. Finally, we can combine multiple representations with diverse speech information using different layer weights to obtain task-specific representations for various downstream tasks. Experimental results indicate that our proposed method achieves joint performance improvements on various tasks, such as speaker identification, speech recognition, emotion recognition, speech enhancement, and voice conversion, compared to excellent SSL methods such as wav2vec2.0, HuBERT, and WavLM. © 2024, CC BY.

关键词： Self-supervised learning

Frequency-specific Brain Network Dynamics during Perceiving Real Words and Pseudowords

学校读者我要写书评

暂无评论

Frequency-specific Brain Network Dynamics during Perceiving ...

International Symposium on Chinese Spoken Language Processing

作者： Taiyang Guo Jianwu Dang Gaoyan Zhang Bin Zhao Masashi Unoki Japan Advanced Institute of Science and Technology Japan Tianjin key lab of cognitive computing and application Tianjin University China

ISBN: (数字)9781728169941

ISBN: (纸本)9781728169958

Many studies used EEG to investigate the brain mechanism of semantic processing and the dynamic brain connectivity at the word level. However, it is requiring more detailed dynamic analysis within a word to clarify the onset of dynamic brain network activity and priming effect. For this reason, this study focused on syllab.e level within a word to investigate semantic processing brain network dynamics for perceiving spoken real words and pseudowords using EEG data with the constraint of fMRI-based network templates. Results illustrated that real words can activate speech perception brain network rapidly, then finished very soon. When perceiving pseudowords, the onset of the perception brain network was slower, but activity lasted longer. If the first syllab.e of a real word has clear categorical features, the semantic categorization brain network would respond to it quickly as the priming effect. The frequency-specific analysis showed that the theta, alpha, and beta brain rhythms contribute more to the semantic processing of Chinese real words than the gamma.

关键词： Semantics Electroencephalography Speech processing

Information sieve: Content leakage reduction in end-to-end prosody transfer for expressive speech synthesis

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Dai, Xudong Gong, Cheng Wang, Longbiao Zhang, Kaili Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Tianjin University-Huiyan Technology Joint AI Lab Tianjin University Tianjin China

Expressive neural text-to-speech (TTS) systems incorporate a style encoder to learn a latent embedding as the style information. However, this embedding process may encode redundant textual information. This phenomenon is called content leakage. Researchers have attempted to resolve this problem by adding an ASR or other auxiliary supervision loss functions. In this study, we propose an unsupervised method called the "information sieve" to reduce the effect of content leakage in prosody transfer. The rationale of this approach is that the style encoder can be forced to focus on style information rather than on textual information contained in the reference speech by a well-designed downsample-upsample filter, i.e., the extracted style embeddings can be downsampled at a certain interval and then upsampled by duplication. Furthermore, we used instance normalization in convolution layers to help the system learn a better latent style space. Objective metrics such as the significantly lower word error rate (WER) demonstrate the effectiveness of this model in mitigating content leakage . Listening tests indicate that the model retains its prosody transferability compared with the baseline models such as the original GST-Tacotron and ASR-guided Tacotron. Copyright © 2021, The Authors. All rights reserved.

关键词： Sieves

A Hybrid Continuity Loss to Reduce Over-Suppression for Time-domain Target Speaker Extraction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Pan, Zexu Ge, Meng Li, Haizhou Institute of Data Science National University of Singapore Singapore Key Laboratory of Cognitive Computing and Application Tianjin University China The Chinese University of Hong Kong Shenzhen China Machine Listening Lab University of Bremen Germany

The speaker extraction algorithm extracts the target speech from a mixture speech containing interference speech and background noise. The extraction process sometimes over-suppresses the extracted target speech, which not only creates artifacts during listening but also harms the performance of downstream automatic speech recognition algorithms. We propose a hybrid continuity loss function for time-domain speaker extraction algorithms to settle the over-suppression problem. On top of the waveform-level loss used for superior signal quality, i.e., SI-SDR, we introduce a multi-resolution delta spectrum loss in the frequency-domain, to ensure the continuity of an extracted speech signal, thus alleviating the over-suppression. We examine the hybrid continuity loss function using a time-domain audio-visual speaker extraction algorithm on the YouTube LRS2-BBC dataset. Experimental results show that the proposed loss function reduces the over-suppression and improves the word error rate of speech recognition on both clean and noisy two-speakers mixtures, without harming the reconstructed speech quality. Copyright © 2022, The Authors. All rights reserved.

关键词： Extraction

Decoding Music Genres Based on High Resolution Brain Activity Information

学校读者我要写书评

暂无评论

Decoding Music Genres Based on High Resolution Brain Activit...

Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA)

作者： Qinhan Hou Gaoyan Zhang College of Intelligence and Computing Tianjin key lab of cognitive computing and application Tianjin University China

ISBN: (数字)9789881476883

ISBN: (纸本)9781728181301

Decoding stimuli from the brain is important for understanding the brain's processing mechanism of external information, which can also promote the development of brain-computer interface. Most of the existing researches focused on the decoding of audiovisual information. Few studies investigated the decoding of music stimuli, and the decoding accuracy is also not satisfactory. This paper uses a public 7-Tesla fMRI image dataset, which collects the high resolution blood oxygen dependent level (BOLD) signals when 20 subjects listen to 5 music genres. After fMRI data preprocessing, two feature selection methods were used. One is based on a prior template (called MASK) including the Heschl's gyrus (HG), the anterior superior temporal gyrus (aSTG), and the posterior superior temporal gyrus (pSTG). The other one is whole brain analysis of variance (ANOVA). Then, the gradient boosting decision tree (GBDT) algorithm is used to train the decoding models to discriminate different music genres. Results showed that among the five genres, ambient music is easier than the other four categories to be decoded. Compared with the previous study that used the same dataset and the same prior template but combined with the classifier of support vector machine[1], the GBDT algorithm improved the accuracies in most genres from around 45% to around 65%. Compared with the MASK method, the ANOVA method improved the decoding accuracy to larger than 74% for all genres. Analysis of contributive regions in the ANOVA method shows that insular and parietal regions are additionally recruited in decoding of music genres, which may be related to music understanding and emotion expression. In summary, the findings in this paper help to understand the brain mechanism of music processing in depth. Meanwhile, the decoding model proposed in this study can also be used to classify other fMRI stimuli.

关键词： Decoding Functional magnetic resonance imaging Feature extraction Analysis of variance Music Brain modeling Encoding

SpEx+: A complete time domain speaker extraction network 21

学校读者我要写书评

暂无评论

SpEx+: A complete time domain speaker extraction network

21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020

作者： Ge, Meng Xu, Chenglin Wang, Longbiao Chng, Eng Siong Dang, Jianwu Li, Haizhou Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China School of Computer Science and Engineering Nanyang Technological University Singapore Japan Advanced Institute of Science and Technology Ishikawa Japan Department of Electrical and Computer Engineering National University of Singapore Singapore Machine Listening Lab University of Bremen Germany

ISBN: (纸本)9781713820697

Speaker extraction aims to extract the target speech signal from a multi-talker environment given a target speaker's reference speech. We recently proposed a time-domain solution, SpEx, that avoids the phase estimation in frequency-domain approaches. Unfortunately, SpEx is not fully a time-domain solution since it performs time-domain speech encoding for speaker extraction, while taking frequency-domain speaker embedding as the reference. The size of the analysis window for time-domain and the size for frequency-domain input are also different. Such mismatch has an adverse effect on the system performance. To eliminate such mismatch, we propose a complete time-domain speaker extraction solution, that is called SpEx+. Specifically, we tie the weights of two identical speech encoder networks, one for the encoder-extractor-decoder pipeline, another as part of the speaker encoder. Experiments show that the SpEx+ achieves 0.8dB and 2.1dB SDR improvement over the state-of-the-art SpEx baseline, under different and same gender conditions on WSJ0-2mix-extr database respectively. Copyright © 2020 ISCA

关键词： Extraction