检索结果-内蒙古大学图书馆

Speech Emotion Recognition: A Brief Review of Multi-modal Multi-task Learning Approaches

学校读者我要写书评

暂无评论

Speech Emotion Recognition: A Brief Review of Multi-modal Mu...

International Conference on Advanced engineering Theory and Applications, AETA 2022

作者： Pham, Nhat Truong Tran, Anh-Tuan Pham, Bich Ngoc Hong Dang-Ngoc, Hanh Nguyen, Sy Dzung Dang, Duc Ngoc Minh Division of Computational Mechatronics Institute for Computational Science Ton Duc Thang University Ho Chi Minh City Viet Nam Faculty of Electrical and Electronics Engineering Ton Duc Thang University Ho Chi Minh City Viet Nam Faculty of Information Technology Ho Chi Minh City Open University Ho Chi Minh City Viet Nam Department of Telecommunications Engineering Ho Chi Minh City University of Technology Ho Chi Minh City Viet Nam Laboratory for Computational Mechatronics Institute for Computational Science and Artificial Intelligence Van Lang University Ho Chi Minh City Viet Nam Faculty of Mechanical–Electrical and Computer Engineering Van Lang University Ho Chi Minh City Viet Nam Computing Fundamental Department FPT University Ho Chi Minh Viet Nam

ISBN: (纸本)9789819987023

Speech emotion recognition (SER) has become an attention-grabbing topic in recent years thanks to the development of deep learning in the field of speech processing. However, it is difficult to recognize an accurate emotional state from only speech signals. Therefore, researchers have investigated multi-modalities such as speech, visual, and text inputs to improve the emotional recognition rate of speech. In addition, to enhance the generalized deep learning models for SER, multi-task learning (MTL) strategies have also been applied in the past decade. In this paper, a brief and comprehensive review of multi-modal multi-task learning (3MTL) approaches for recognizing emotional states from speech signals is presented, including multi-modal SER, multi-task learning SER, and multi-modal multi-task learning SER. This paper also discusses about some problems that still need to be solved in 3MTL SER and gives some suggestions for the future. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

关键词： Emotion Recognition

Task Delay and Energy Consumption Minimization for Low-altitude MEC via Evolutionary Multi-objective Deep Reinforcement Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sun, Geng Ma, Weilong Li, Jiahui Sun, Zemin Wang, Jiacheng Niyato, Dusit Mao, Shiwen College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China College of Computing and Data Science Nanyang Technological University Singapore639798 Singapore College of Software Jilin University Changchun130012 China Department of Electrical and Computer Engineering Auburn University AuburnAL36849-5201 United States

The low-altitude economy (LAE), driven by unmanned aerial vehicles (UAVs) and other aircraft, has revolutionized fields such as transportation, agriculture, and environmental monitoring. In the upcoming six-generation (6G) era, UAV-assisted mobile edge computing (MEC) is particularly crucial in challenging environments such as mountainous or disaster-stricken areas. The computation task offloading problem is one of the key issues in UAV-assisted MEC, primarily addressing the trade-off between minimizing the task delay and the energy consumption of the UAV. In this paper, we consider a UAV-assisted MEC system where the UAV carries the edge servers to facilitate task offloading for ground devices (GDs), and formulate a calculation delay and energy consumption multi-objective optimization problem (CDECMOP) to simultaneously improve the performance and reduce the cost of the system. Then, by modeling the formulated problem as a multi-objective Markov decision process (MOMDP), we propose a multi-objective deep reinforcement learning (DRL) algorithm within an evolutionary framework to dynamically adjust the weights and obtain non-dominated policies. Moreover, to ensure stable convergence and improve performance, we incorporate a target distribution learning (TDL) algorithm. Simulation results demonstrate that the proposed algorithm can better balance multiple optimization objectives and obtain superior non-dominated solutions compared to other methods. Copyright © 2025, The Authors. All rights reserved.

关键词： Deep reinforcement learning

H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture

学校读者我要写书评

暂无评论

H-GCN: A Graph Convolutional Network Accelerator on Versal A...

International Conference on Field Programmable Logic and Applications

作者： Chengming Zhang Tong Geng Anqi Guo Jiannan Tian Martin Herbordt Ang Lit Dingwen Tao School of Electrical Engineering and Computer Science Washington State University Pullman WA USA School of Informatics Computing and Engineering Indiana University Bloomington IN USA Mathematics and Computer Science Division Pacific Northwest National Laboratory Richland WA USA Department of Electrical and Computer Engineering University of Rochester Rochester NY USA Department of Electrical and Computer Engineering Boston University Boston MA USA

Graph Neural Networks (GNNs) have drawn tremendous attention due to their unique capability to extend Machine Learning (ML) approaches to applications broadly-defined as having unstructured data, especially graphs. Compared with other Machine Learning (ML) modalities, the acceleration of Graph Neural Networks (GNNs) is more challenging due to the irregularity and heterogeneity derived from graph typologies. Existing efforts, however, have focused mainly on handling graphs' irregularity and have not studied their heterogeneity. To this end we propose H-GCN, a PL (Programmable Logic) and AIE (AI Engine) based hybrid accelerator that leverages the emerging heterogeneity of Xilinx Versal Adaptive Compute Acceleration Platforms (ACAPs) to achieve high-performance GNN inference. In particular, H-GCN partitions each graph into three subgraphs based on its inherent heterogeneity, and processes them using PL and AIE, respectively. To further improve performance, we explore the sparsity support of AIE and develop an efficient density-aware method to automatically map tiles of sparse matrix-matrix multiplication (SpMM) onto the systolic tensor array. Compared with state-of-the-art GCN accelerators, H-GCN achieves, on average, speedups of 1.1~2.3x.

关键词： Tensors Adaptive systems Machine learning computer architecture Systolic arrays Graph neural networks Sparse matrices

Energy Efficient Collaborative Federated Learning Design: A Graph Neural Network based Approach

学校读者我要写书评

暂无评论

Energy Efficient Collaborative Federated Learning Design: A ...

IEEE Conference on Global Communications (GLOBECOM)

作者： Nuocheng Yang Sihua Wang Mingzhe Chen Christopher G. Brinton Changchuan Yin Beijing Laboratory of Advanced Information Network Beijing University of Posts and Telecommunications Beijing China State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications Beijing China Department of Electrical and Computer Engineering Institute for Data Science and Computing University of Miami Coral Gables FL USA School of Electrical and Computer Engineering Purdue University West Lafayette IN USA

In this paper, we consider the design of an energy efficient collaborative federated learning (CFL) methodology where devices exchange their local FL parameters with a subset of their neighbors without reliance on a parameter server. In the considered model, mobile devices implement the designed CFL to train their local FL models using their own datasets over a realistic wireless network. Due to the limited wireless resources and user movements, each device may not be able to transmit its FL parameters with all neighboring devices. Therefore, each device must select a subset of devices to share its FL parameters and optimize the transmit power. This problem is formulated as an optimization problem, whose goal is to minimize CFL training energy consumption while satisfying the delay and CFL training loss requirements. To solve this problem, a two-stage solution is proposed. At the first stage, a graph neural network (GNN) based algorithm is proposed, which enables each device to individually determine the subset of devices to transmit FL parameters using its neighboring devices' location and connection information. Compared to standard iterative algorithms that need to iteratively optimize device connections and transmit power, the proposed GNN based method can directly obtain the optimal device connections without iterative optimization. Given the optimal device connections, at the second stage, each device can directly obtain the optimal transmit power. Simulation results show that the proposed algorithm can decrease energy consumption by up to 46% compared to the algorithm where each device will directly connect to its first and second nearest neighbors.

关键词：

Dual UAV Cluster-Assisted Maritime Physical Layer Secure Communications via Collaborative Beamforming

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Jiawei Wang, Aimin Sun, Geng Li, Jiahui Wang, Jiacheng Du, Hongyang Niyato, Dusit College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China College of Computing and Data Science Nanyang Technological University Singapore639798 Singapore Department of Electrical and Electronic Engineering Hong Kong University 999077 Hong Kong

Unmanned aerial vehicles (UAVs) can be utilized as relay platforms to assist maritime wireless communications. However, complex channels and multipath effects at sea can adversely affect the quality of UAV transmitted signals. Collaborative beamforming (CB) can enhance the signal strength and range to assist the UAV relay for remote maritime communications. However, due to the open nature of UAV channels, security issue requires special consideration. This paper proposes a dual UAV cluster-assisted system via CB to achieve physical layer security in maritime wireless communications. Specifically, one UAV cluster forms a maritime UAV-enabled virtual antenna array (MUVAA) relay to forward data signals to the remote legitimate vessel, and the other UAV cluster forms an MUVAA jammer to send jamming signals to the remote eavesdropper. In this system, we formulate a secure and energy-efficient maritime communication multi-objective optimization problem (SEMCMOP) to maximize the signal-to-interference-plus-noise ratio (SINR) of the legitimate vessel, minimize the SINR of the eavesdropping vessel and minimize the total flight energy consumption of UAVs. Since the SEMCMOP is an NP-hard and large-scale optimization problem, we propose an improved swarm intelligence optimization algorithm with chaotic solution initialization and hybrid solution update strategies to solve the problem. Simulation results indicate that the proposed algorithm outperforms other comparison algorithms, and it can achieve more efficient signal transmission by using the CB-based method. Copyright © 2024, The Authors. All rights reserved.

关键词： Multiobjective optimization

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Pengxiang Gao, Zhi Zhang, Bofei Yuan, Tao Wu, Yuwei Harandi, Mehrtash Jia, Yunde Zhu, Song-Chun Li, Qing Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China State Key Laboratory of General Artificial Intelligence BIGAI China State Key Laboratory of General Artificial Intelligence Peking University China Department of Electrical and Computer System Engineering Monash University Australia Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China Department of Automation Tsinghua University China

Vision language models (VLMs) have achieved impressive progress in diverse applications, becoming a prevalent research direction. In this paper, we build FIRE, a feedback-refinement dataset, consisting of 1.1M multi-turn conversations that are derived from 27 source datasets, empowering VLMs to spontaneously refine their responses based on user feedback across diverse tasks. To scale up the data collection, FIRE is collected in two components: FIRE-100K and FIRE-1M, where FIRE-100K is generated by GPT-4V, and FIRE-1M is freely generated via models trained on FIRE-100K. Then, we build FIRE-Bench, a benchmark to comprehensively evaluate the feedback-refining capability of VLMs, which contains 11K feedback-refinement conversations as the test data, two evaluation settings, and a model to provide feedback for VLMs. We develop the FIRE-LLaVA model by fine-tuning LLaVA on FIRE-100K and FIRE-1M, which shows remarkable feedback-refining capability on FIRE-Bench and outperforms untrained VLMs by 50%, making more efficient user-agent interactions and underscoring the significance of the FIRE dataset. Copyright © 2024, The Authors. All rights reserved.

关键词： Fireclay

Rethinking the visual cues in audio-visual speaker extraction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Junjie Ge, Meng Pan, Zexu Cao, Rui Wang, Longbiao Dang, Jianwu Zhang, Shiliang Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Department of Electrical and Computer Engineering National University of Singapore Singapore Institute of Data Science National University of Singapore Singapore Shenzhen Research Institute of Big Data Shenzhen China

The Audio-Visual Speaker Extraction (AVSE) algorithm employs parallel video recording to leverage two visual cues, namely speaker identity and synchronization, to enhance performance compared to audio-only algorithms. However, the visual front-end in AVSE is often derived from a pre-trained model or end-to-end trained, making it unclear which visual cue contributes more to the speaker extraction performance. This raises the question of how to better utilize visual cues. To address this issue, we propose two training strategies that decouple the learning of the two visual cues. Our experimental results demonstrate that both visual cues are useful, with the synchronization cue having a higher impact. We introduce a more explainable model, the Decoupled Audio-Visual Speaker Extraction (DAVSE) model, which leverages both visual cues. Copyright © 2023, The Authors. All rights reserved.

关键词： Extraction

Large Scale Model Enabled Semantic Communications Based on Robust Knowledge Distillation

学校读者我要写书评

暂无评论

Large Scale Model Enabled Semantic Communications Based on R...

IEEE Conference on Global Communications (GLOBECOM)

作者： Kuiyuan Ding Fangfang Liu Yang Yang Mingzhe Chen Caili Guo Beijing Key Laboratory of Network System Architecture and Convergence School of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing China Department of Electrical and Computer Engineering and the Institute for Data Science and Computing University of Miami Coral Gables FL USA Beijing Laboratory of Advanced Information Networks School of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing China

ISBN: (数字)9798350351255

ISBN: (纸本)9798350351262

Large scale artificial intelligence (AI) models possess excellent capabilities in semantic representation and understanding, making them particularly well-suited for semantic encoding and decoding. However, the substantial scale of these AI models imposes unacceptable computational resources and communication delays. To address this issue, we propose a semantic communication scheme based on robust knowledge distillation (RKD-SC) for large scale model enabled semantic communications. In the considered system, a transmitter extracts the features of the source image for robust transmission and accurate image classification at the receiver. To effectively utilize the superior capability of large scale model while make the cost affordable, we first transfer knowledge from a large scale model to a smaller scale model to serve as the semantic encoder. Then, to enhance the robustness of the system against channel noise, we propose a channel-aware autoencoder (CAA) based on the Transformer architecture. Experimental results show that the encoder of proposed RKD-SC system can achieve over 93.3% of the performance of a large scale model while compressing 96.67% number of parameters. Code: https://***/echojayne/RKD-SC.

关键词： Transmitters Computational modeling Noise Autoencoders Receivers Semantic communication Transformers Feature extraction Robustness Decoding

Deep Robust Reversible Watermarking

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Jiale Wang, Wei Shi, Chongyang Dong, Li Li, Yuanman Hu, Xiping School of Computer Science Beijing Institute of Technology Beijing100081 China Guangdong-Hong Kong-Macao Joint Laboratory for Emotional Intelligence and Pervasive Computing Artificial Intelligence Research Institute Shenzhen MSU-BIT University Shenzhen518172 China School of Medical Technology Beijing Institute of Technology Beijing100081 China College of Electronics and Information Engineering Shenzhen University Shenzhen518060 China Department of Computer Science Faculty of Electrical Engineering and Computer Science Ningbo University Ningbo315211 China

Robust Reversible Watermarking (RRW) enables perfect recovery of cover images and watermarks in lossless channels while ensuring robust watermark extraction under lossy channels. However, existing RRW methods, mostly non-deep learning-based, suffer from complex designs, high computational costs, and poor robustness limiting their practical applications. To address these issues, this paper proposes Deep Robust Reversible Watermarking (DRRW), a deep learning-based RRW scheme. DRRW introduces an Integer Invertible Watermark Network (iIWN) to achieve an invertible mapping between integer data distributions, fundamentally addressing the limitations of conventional RRW approaches. Unlike traditional RRW methods requiring task-specific designs for different distortions, DRRW adopts an encoder-noise layer-decoder framework, enabling adaptive robustness against various distortions through end-to-end training. During inference, the cover image and watermark are mapped into an overflowed stego image and latent variables. Arithmetic coding efficiently compresses these into a compact bitstream, which is embedded via reversible data hiding to ensure lossless recovery of both the image and watermark. To reduce pixel overflow, we introduce an overflow penalty loss, significantly shortening the auxiliary bitstream while improving both robustness and stego image quality. Additionally, we propose an adaptive weight adjustment strategy that eliminates the need to manually preset the watermark loss weight, ensuring improved training stability and performance. Experiments on multiple datasets demonstrate that DRRW achieves notable performance advantages. Compared to state-of-the-art RRW methods, DRRW improves robustness and reduces embedding, extraction, and recovery complexities by 55.14×, 5.95×, and 3.57×, respectively. The auxiliary bitstream is shortened by 43.86×, and reversible embedding succeeds on 16,762 images in the PASCAL VOC 2012 dataset, marking a significant step toward pra

关键词： Image enhancement