检索结果-内蒙古大学图书馆

VoiceStyle: Voice-Based Face Generation via Cross-Modal Prototype Contrastive Learning

ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 2024年第9期20卷 1-23页

作者： Chen, Wuyang Zhu, Boqing Xu, Kele Dou, Yong Feng, Dawei Natl Key Lab Parallel & Distributed Proc Changsha Peoples R China

Can we predict a person's appearance solely based on their voice? This article explores this question by focusing on generating a face from an unheard voice segment. Our proposed method, VoiceStyle, combines cross-modal representation learning with generation modeling, enabling us to incorporate voice semantic cues into the generated face. In the first stage, we introduce cross-modal prototype contrastive (CMPC) learning to establish the association between voice and face. Recognizing the presence of false negative and deviate positive instances in real-world unlabeled data, we not only use voice-face pairs in the same video but also construct additional semantic positive pairs through unsupervised clustering, enhancing the learning process. Moreover, we recalibrate instances based on their similarity to cluster centers in the other modality. In the second stage, we harness the powerful generative capabilities of StyleGAN to produce faces. We optimize the latent code in StyleGAN's latent space, guided by the learned voice-face alignment. To address the importance of selecting an appropriate starting point for optimization, we aim to automatically find an optimal starting point by utilizing the face prototype derived from the voice input. The entire pipeline can be implemented in a self-supervised manner, eliminating the need for manually labeled annotations. Through extensive experiments, we demonstrate the effectiveness and performance of our VoiceStyle method in both cross-modal representation learning and voice-based face generation.

关键词： Cross-modal representation learning contrastive learning StyleGAN face generation

来源：评论

学校读者我要写书评

暂无评论

Leaders and Collaborators: Addressing Sparse Reward Challenges in Multi-Agent Reinforcement Learning

引用

IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE 2025年第2期9卷 1976-1989页

作者： Sun, Shaoqi Liu, Hui Xu, Kele Ding, Bo Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha 410003 Peoples R China

Cooperative multi-agent reinforcement learning (MARL) has emerged as an effective tool for addressing complex control tasks. However, sparse team rewards present significant challenges for MARL, leading to low exploration efficiency, slow learning speed, and homogenized behaviors among agents. To address these issues, we propose a novel Leader-Collaborator (LC) MARL framework inspired by human social collaboration. The LC framework introduces parallel online knowledge distillation for policy networks (KDPN). KDPN extracts knowledge from two policy networks with different training objectives: one aims to maximize individual rewards, while the other aims to maximize team rewards. The extracted knowledge is utilized to construct team leaders and collaborators. By effectively balancing individual and team rewards, our approach enhances exploration efficiency and promotes behavioral diversity among agents. This addresses the issue of low learning efficiency caused by the lack of objectives early in the agent's learning process and facilitates the development of more effective and differentiated team interaction policies. Additionally, we present the Self-Repairing Strategy (SRS) and Self-Augmenting Strategy (SAS) to facilitate team policies learning while preserving the initial team goal. We evaluate the effectiveness of the LC framework by conducting extensive experiments on the Multi-Agent Particle Environment (MPE), the Google Research Football (GRF), and StarCraft Multi-Agent Challenge (SMAC) with varying levels of difficulty. Our experimental results demonstrate that LC significantly improves the efficiency of the agent's exploration, achieves state-of-the-art performance, and accelerates the learning of the optimal policy. Specifically, in the SMAC scenarios, our method increases the winning rate by 21.9%, increases the average cumulative reward by 12%, and reduces the training time by 57% to achieve optimal performance.

关键词： Computational modeling Training Reinforcement learning Sports Knowledge engineering Computational intelligence Teamwork Sun Standards Predictive models Multi-agent reinforcement learning sparse rewards online knowledge distillation exploration efficiency

来源：评论

学校读者我要写书评

暂无评论

Automated Data Augmentation for Audio Classification

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE procESSING 2024年 32卷 2716-2728页

作者： Sun, Yanjie Xu, Kele Liu, Chaorun Dou, Yong Wang, Huaimin Ding, Bo Pan, Qinghua Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha 410073 Peoples R China iFlytek Co Ltd Hefei 230088 Peoples R China

Audio classification is a challenging task that requires categorizing audio data based on its content or characteristics. Existing approaches for audio classification rely either on supervised learning or fine-tuning based on self-supervised learning, both of which require manually labeled data. However, manually labeling audio datasets is a time-consuming and expensive process that limits the dataset's size. Moreover, the diversity of sound categories and class imbalances can further impede classification performance. To overcome these challenges, researchers have proposed various audio data augmentation methods. However, most of these methods focus less on augmentations combination and design and rely solely on waveform-based or spectrogram-based approaches. This paper presents an Automated Audio Augmentation (AAA) method for audio classification, which generates learnable and composable augmentation policies suitable for the audio classification task and can be employed in a plug-and-play manner. This method leverages both waveform-level and spectrogram-level augmentation, and a Bayesian optimization algorithm is proposed to search for composed augmentation policies. To the best of our knowledge, this is the first attempt to propose an automatic data augmentation method for audio classification tasks. Through large-scale empirical studies, we demonstrate that the proposed method outperforms previous competitive methods by a significant margin. We improve the average performance of multiple datasets by 6.421% and by 7.330% on few-shot scenarios, respectively.

关键词： Audio classification automated augmentation audio data augmentation

来源：评论

学校读者我要写书评

暂无评论

A systematic review of the application of machine learning techniques to ultrasound tongue imaging analysis

引用

JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2024年第3期156卷 1796-1819页

作者： Xia, Zhen Yuan, Ruicheng Cao, Yuan Sun, Tao Xiong, Yunsheng Xu, Kele Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha Hunan Peoples R China Hunan Univ Coll Comp Sci & Elect Engn Changsha Hunan Peoples R China

B-mode ultrasound has emerged as a prevalent tool for observing tongue motion in speech production, gaining traction in speech therapy applications. However, the effective analysis of ultrasound tongue image frame sequences (UTIFs) encounters many challenges, such as the presence of high levels of speckle noise and obscured views. Recently, the application of machine learning, especially deep learning techniques, to UTIF interpretation has shown promise in overcoming these hurdles. This paper presents a thorough examination of the existing literature, focusing on UTIF analysis. The scope of our work encompasses four key areas: a foundational introduction to deep learning principles, an exploration of motion tracking methodologies, a discussion of feature extraction techniques, and an examination of cross-modality mapping. The paper concludes with a detailed discussion of insights gleaned from the comprehensive literature review, outlining potential trends and challenges that lie ahead in the field.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Temporal Inconsistency-Based Intrinsic Reward for Multi-Agent Reinforcement Learning

Temporal Inconsistency-Based Intrinsic Reward for Multi-Agen...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Sun, Shaoqi Xu, Kele Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha Peoples R China

ISBN: (纸本)9781665488679

Multi-agent reinforcement learning (MARL) has shown promising results in many challenging sequential decision-making tasks. Recently, deep neural networks have dominated this field. However, the policy networks of agent's may fall into local optimum during the training phase, which severely constrains the performance of exploration. To address this issue, we propose a novel MARL learning framework named PSAM, which contains a new temporal inconsistency-based intrinsic reward and a diversity control strategy. Specifically, we save the parameters of the deep models along the optimization path of the agent's policy network, which can be denoted as snapshots. Through measuring the difference between snapshots, we can employ the difference as an intrinsic reward. Moreover, we propose a diversity control strategy to improve the performance further. Finally, to verify the effectiveness of the proposed method, we conduct extensive experiments in several widely used MARL environments. The results show that in many environments, PSAM can not only achieve state-of-the-art performance and prevent the policy network from getting stuck in local minima but also accelerate the agent's learning of the policy. It is worth noting that the proposed regularizer can be used using a plug-and-play manner without introducing any additional hyper-parameters and training costs.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

A Survey on Auto-parallelism of Large-Scale Deep Learning Training

引用

IEEE TRANSACTIONS ON parallel AND distributed SYSTEMS 2023年第8期34卷 2377-2390页

作者： Liang, Peng Tang, Yu Zhang, Xiaoda Bai, Youhui Su, Teng Lai, Zhiquan Qiao, Linbo Li, Dongsheng Natl Univ Def Technol State Key Lab Parallel & Distributed Proc Changsha 410073 Peoples R China Huawei Technol Co Ltd Shenzhen 518100 Peoples R China

Deep learning (DL) has gained great success in recent years, leading to state-of-the-art performance in research community and industrial fields like computer vision and natural language processing. One of the reasons for this success is the huge amount parameters adopted in DL models. However, it is impractical to train a moderately large model with a large number of parameters on a typical single device. Thus, It is necessary to train DL models in clusters with distributed training algorithms. However, traditional distributed training algorithms are usually sub-optimal and highly customized, which owns the drawbacks to train large-scale DL models in varying computing clusters. To handle the above problem, researchers propose auto-parallelism, which is promising to train large-scale DL models efficiently and practically in various computing clusters. In this survey, we perform a broad and thorough investigation on challenges, basis, and strategy searching methods of auto-parallelism in DL training. First, we abstract basic parallelism schemes with their communication cost and memory consumption in DL training. Further, we analyze and compare a series of current auto-parallelism works and investigate strategies and searching methods which are commonly used in practice. At last, we discuss several trends in auto-parallelism which are promising in further research.

关键词： Auto-parallelism large-scale deep learning model training technique parallel and distributed training

来源：评论

学校读者我要写书评

暂无评论

Large Pretrained Foundation Model for key Performance Indicator Multivariate Time Series Anomaly Detection

IEEE OPEN JOURNAL OF THE COMPUTER SOCIETY

引用

IEEE OPEN JOURNAL OF THE COMPUTER SOCIETY 2025年第1期6卷 176-187页

作者： Wang, Xu Xu, Qisheng Xu, Kele Yu, Ting Ding, Bo Feng, Dawei Dou, Yong Natl Univ Def Technol Coll Comp Sci & Technol Changsha 410073 Peoples R China Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha 410073 Peoples R China

In the realm of key Performance Indicator (KPI) anomaly detection, deep learning has emerged as a pivotal technology. Yet, the development of effective deep learning models is hindered by several challenges: scarce and complex labeled data, noise interference from data handling, the necessity to capture temporal dependencies in time series KPI data, and the complexity of multivariate data analysis. Despite recent progress in large models that show potential for handling complex, multidimensional tasks, the lack of extensive, high-quality datasets presents a significant barrier for directly training these models in KPI anomaly detection. This scarcity limits the models' ability to learn and generalize effectively within this specific domain. To overcome this, we propose an innovative approach to adapt fully pretrained large models from other domains to KPI anomaly detection, thereby mitigating data constraints and enhancing detection precision. Our approach involves adapting large models to anomaly detection tasks using patch operations and fine-tuning techniques, which significantly enhances the model's temporal dependency capture capabilities. Furthermore, to address the multivariate challenge, we introduce a novel feature extraction method based on channel independence to optimize information processing across multidimensional features. Additionally, we leverage frequency domain information to design a feature enhancement method, further boosting the model's detection accuracy. By integrating these innovative techniques, we have developed a large-scale KPI anomaly detection model named ViTSD. Empirical evidence from experiments on five benchmark datasets and two additional datasets demonstrates ViTSD's superior performance, outperforming existing models across various evaluation metrics.

关键词： Feature extraction Anomaly detection Time series analysis key performance indicator Adaptation models Data models Analytical models Time-frequency analysis Transformers Computational modeling large foundation model patch operation key Performance Indicator (KPI) anomaly detection

来源：评论

学校读者我要写书评

暂无评论

Multi-representation knowledge distillation for audio classification

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2022年第4期81卷 5089-5112页

作者： Gao, Liang Xu, Kele Wang, Huaimin Peng, Yuxing Natl Univ Def Technol Coll Comp Natl Key Lab Parallel & Distributed Proc Changsha 410073 Peoples R China

Audio classification aims to discriminate between different audio signal types, and it has received intensive attention due to its wide applications. In deep learning-based audio classification methods, researchers usually transform the raw signal of audios into different feature representations (such as Short Time Fourier Transform and Mel Frequency Cepstral Coefficients) as the inputs of networks. However, selecting the feature representation requires expert knowledge and extensive experimental verification. Besides, using a single type of feature representation may cause suboptimal results as the information implied in different kinds of feature representations may be complementary. Previous works show that ensembling the networks trained on different representations can greatly boost classification performance. However, making inferences using multiple networks is cumbersome and computation expensive. In this paper, we propose a novel end-to-end collaborative training framework for the audio classification task. The framework takes multiple representations as inputs to train the networks jointly with a knowledge distillation method. Consequently, our framework significantly promotes the performance of networks without increasing the computational overhead in the inference stage. Extensive experimental results demonstrate that the proposed approach improves classification performance and achieves competitive results on both acoustic scene classification tasks and general audio tagging tasks.

关键词： Neural networks Multiple representations Acoustic classification Knowledge distillation

来源：评论

学校读者我要写书评

暂无评论

Self-supervised learning-based underwater acoustical signal classification via mask modeling

引用

JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2023年第1期154卷 5-15页

作者： Xu, Kele Xu, Qisheng You, Kang Zhu, Boqing Feng, Ming Feng, Dawei Liu, Bo Natl Key Lab Parallel & Distributed Proc Changsha 410073 Peoples R China Tongji Univ Shanghai Peoples R China Natl Univ Def Technol Coll Comp Changsha 410073 Peoples R China

The classification of underwater acoustic signals has garnered a great deal of attention in recent years due to its potential applications in military and civilian contexts. While deep neural networks have emerged as the preferred method for this task, the representation of the signals plays a crucial role in determining the performance of the classification. However, the representation of underwater acoustic signals remains an under-explored area. In addition, the annotation of large-scale datasets for the training of deep networks is a challenging and expensive task. To tackle these challenges, we propose a novel self-supervised representation learning method for underwater acoustic signal classification. Our approach consists of two stages: a pretext learning stage using unlabeled data and a downstream fine-tuning stage using a small amount of labeled data. The pretext learning stage involves randomly masking the log Mel spectrogram and reconstructing the masked part using the Swin Transformer architecture. This allows us to learn a general representation of the acoustic signal. Our method achieves a classification accuracy of 80.22% on the DeepShip dataset, outperforming or matching previous competitive methods. Furthermore, our classification method demonstrates good performance in low signal-to-noise ratio or few-shot settings.

关键词： Classification (of information)

来源：评论

学校读者我要写书评

暂无评论

CNA: A Dataset for Parsing Discourse Structure on Chinese News Articles 34

CNA: A Dataset for Parsing Discourse Structure on Chinese Ne...

引用

34th IEEE International Conference on Tools with Artificial Intelligence (ICTAI)

作者： Guo, Zhenliang Huang, Zhen Dou, Yong Yu, Xiubin Wang, Sijie Chen, Zhongwu Su, Xinxin Liu, Xiaohang Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha Peoples R China

ISBN: (纸本)9798350397444

Discourse structure analysis has shown to be useful for many artificial intelligence (AI) tasks such as text summarization and text categorization. However, for the Chinese news domain, the discourse structure analysis system is still immature due to the limitation of the lack of expert-annotated datasets. In this paper, we present CNA, a Chinese news corpus containing 1155 news articles annotated by human experts, which covers four domains and four news media sources. Next, we implement several text classification methods as baselines. Experimental results demonstrate that document-level method can achieve a better performance, and we further propose a document-level neural network model with multiple sentence features which achieves the state-of-the-art performance. In the end, we analyze the content type distribution of each sentence in CNA and the prediction errors of our model that occurred on the test set. The codes and dataset will be open-sourced at https://***/gzl98/Chinese Discourse Profiling.

关键词： discourse structure analysis Chinese dataset main event multiple features

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：