检索结果-内蒙古大学图书馆

Attention mechanism based multimodal feature fusion network for human action recognition

Journal of Visual Communication and Image Representation 2025年 110卷

作者： Zhao, Xu Tang, Chao Hu, Huosheng Wang, Wenjian Qiao, Shuo Tong, Anyang School of Artificial Intelligence and Big Data Hefei University Hefei China School of Computer Science and Electronic Engineering University of Essex England ColchesterCO4 3SQ United Kingdom School of Computer and Information Technology Shanxi University Taiyuan China School of Computer Science and Information Engineering Hefei University of Technology Hefei China

Current human action recognition (HAR) methods focus on integrating multiple data modalities, such as skeleton data and RGB data. However, they struggle to exploit motion correlation information in skeleton data and rely on spatial representations from RGB modalities. This paper proposes a novel Attention-based Multimodal Feature Integration Network (AMFI-Net) designed to enhance modal fusion and improve recognition accuracy. First, RGB and skeleton data undergo multi-level preprocessing to obtain differential movement representations, which are then input into a heterogeneous network for separate multimodal feature extraction. Next, an adaptive fusion strategy is employed to enhance the integration of these multimodal features. Finally, the network assesses the confidence level of weighted skeleton information to determine the extent and type of appearance information to be used in the final feature integration. Experiments conducted on the NTU-RGB + D dataset demonstrate that the proposed method is feasible, leading to significant improvements in human action recognition accuracy. © 2025 Elsevier Inc.

关键词： Attention Mechanism Feature Extraction and Fusion Human Action Recognition Multimodal Features

来源：评论

学校读者我要写书评

暂无评论

KAN-Face: Efficient Resource Usage and Precision Lip-Sync in Talking Head Generation

KAN-Face: Efficient Resource Usage and Precision Lip-Sync in...

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Feng, Guanwen Jin, Siyu Qian, Zhihao Li, Yunan Miao, Qiguang School of Computer Science and Technology Xidian University Shaanxi Xi'an710071 China Xi'an Key Laboratory of Big Data and Intelligent Vision Xidian University Shaanxi Xi'an710071 China Key Laboratory of Collaborative Intelligence Systems Ministry of Education Xidian University Xi'an710071 China

ISBN: (纸本)9798350368741

Despite significant progress in NeRF-based talking head generation, problems like poor lip synchronization and inefficient resource usage remain. To solve these, we propose KAN-Face, a lightweight framework. In preprocessing, we introduce a Lip-Sync Enhancement Module that uses Wav2Lip to extract high-resolution audio features and map them to an explicit intermediate representation, ensuring precise lip movement alignment with the speaker's identity. These predicted lip-sync features are combined with fundamental audio-extracted lip features and injected into the rendering module to improve synchronization. For rendering, we introduce FastKAN to map spatial points to color values. As a variant of KAN, FastKAN's sensitivity to 3D scenes and efficient structure enable precise, fast color prediction. Our framework reduces resource consumption while enhancing lip-sync accuracy and facial reconstruction, making it ideal for talking head generation tasks in resource-limited settings. Project: https://***/KAN-Face/. © 2025 IEEE.

关键词： Face Animation KAN Network Low Resource Neural Radiance Fields Talking Head Generation

来源：评论

学校读者我要写书评

暂无评论

Robust Multi-Object Tracking Using Vision Sensor with Fine-Grained Cues in Occluded and Dynamic Scenes

引用

IEEE Sensors Journal 2025年

作者： Hu, Yaoqi Sun, Jinqiu Jin, Hao Niu, Axi Yan, Qingsen Zhu, Yu Zhang, Yanning School of Computer Science China School of Astronautics China National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology Northwestern Polytechnical University Dongxiang Road Shaanxi Xi'an710129 China

Multi-object tracking (MOT) using vision sensors remains a challenging problem, particularly in dynamic backgrounds and severe occlusions. Existing methods, relying on holistic appearance or spatial cues, fail to capture detailed information within object regions and the background, resulting in inaccurate and inconsistent tracking. To address these issues, we propose a novel Detail-Driven Multi-Object Tracking (DD-MOT) method that revisits and leverages fine-grained cues to discover the dynamics of both object regions and the background, facilitating the recovery and association of trajectories. Specifically, the proposed method consists of three key modules: (i) Points Trajectories Generator, (ii) Camera Motion and Occlusion Compensation, and (iii) Fine- and Coarsegrained Association. The Points Trajectories Generator is responsible for generating fine-grained cues by sampling an initial set of points, generating point trajectories, and refining the initial set of points. The Camera Motion and Occlusion Compensation module utilizes the background and object point trajectories to correct background motion and recover occluded object bounding boxes. Finally, the Fine- and Coarse-grained Association module leverages point trajectory cues to assist in establishing a more effective association strategy, combining both fine-grained and coarse-grained spatial cues from the object bounding boxes. We evaluate our DD-MOT method on several benchmark datasets, including MOT17, MOT20, demonstrating that it consistently outperforms state-of-the-art (SOTA) methods in key metrics such as HOTA (64.2, 62.6), MOTA (80.8, 78.1), and IDF1 (78.8, 77.1). © 2025 IEEE.

关键词： Motion compensation

来源：评论

学校读者我要写书评

暂无评论

FFCI: A Camera and IMU Sensors Based Multi-modal Neural Network for Activity Recognition in Smart Factory

引用

IEEE Transactions on Consumer Electronics 2025年

作者： Wang, Yujue Niu, Xin Lv, Xianwei Yu, Chen Huazhong University of Science and Technology National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Wuhan430074 China

Worker activity recognition is an important aspect of the construction of smart factory. The development of deep neural networks and the widespread distribution of sensors in the smart factory have brought opportunities for the recognition of workers' activities. The existing methods based on camera and IMU sensors respectively have problems of visual occlusion and difficulty in recognition of similar activities, which result in the reduction of recognition accuracy. Therefore, we propose a feature-level fusion based on camera and IMU sensor (FFCI) for activity recognition in smart factory. The FFCI employs an optimized multi-modal fusion strategy to integrate the visual information and the action information of workers' activities. In order to verify the effectiveness of FFCI, we define a simple yet fine-grained assembly task and collect six common operations. Furthermore, we add visual occlusion and similar activity data to get closer to the smart factory. Finally, we evaluate the FFCI on the collected dataset and achieve a recognition accuracy of 98.17%, demonstrating its effectiveness in accurately classifying workers' activities. © 1975-2011 IEEE.

关键词： Factory automation

来源：评论

学校读者我要写书评

暂无评论

Impulsive Control of Nonlinear Multi-Agent Systems: A Hybrid Fuzzy Adaptive and Event-Triggered Strategy

引用

IEEE Transactions on Fuzzy Systems 2025年

作者： Han, Fang Jin, Hai Huazhong University of Science and Technology National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Laboratory Cluster and Grid Computing Laboratory School of Computer Science and Technology Wuhan430074 China

This paper presents a hybrid control approach that integrates an adaptive fuzzy mechanism with an event-triggered impulse strategy to address consensus control challenges in nonlinear Multi-Agent Systems (MASs) with uncertain information, both in leader and leaderless scenarios. Firstly, a fuzzy logic-based adaptive control mechanism is proposed to handle system nonlinearity and uncertainty. This mechanism dynamically adjusts fuzzy control rules in real time, enabling the system to adapt to changing conditions and enhancing its robustness against nonlinear disturbances. Secondly, to alleviate communication overhead and reduce control frequency, an event-triggered impulse strategy is implemented. This strategy activates impulse control only under specific conditions, effectively minimizing control actions while maintaining system consensus. Using Lyapunov stability theory, sufficient conditions for achieving system consensus are established, demonstrating the global consensus and robustness of the MASs under certain criteria. Finally, numerical simulations and experimental evaluations validate the proposed method's effectiveness and its practical advantages. © 2025 IEEE.

关键词： Multi agent systems

来源：评论

学校读者我要写书评

暂无评论

SPViT: Accelerate Vision Transformer Inference on Mobile Devices via Adaptive Splitting and Offloading

引用

IEEE Transactions on Mobile Computing 2025年

作者： Zhao, Sifan Liu, Tongtong Jin, Hai Yao, Dezhong Huazhong University of Science and Technology National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Wuhan430074 China

The Vision Transformer (ViT), which benefits from utilizing self-attention mechanisms, has demonstrated superior accuracy compared to CNNs. However, due to the expensive computational costs, deploying and inferring ViTs on resource-constrained mobile devices has become a challenge. To resolve this challenge, we conducted an empirical analysis to identify performance bottlenecks in deploying ViTs on mobile devices and explored viable solutions. In this paper, we propose SPViT, an adaptive split and offloading method that accelerates ViT inference on mobile devices. SPViT executes collaborative inference of ViT across available edge devices. We introduce a fine-grained splitting technique for the vision transformer structure. Furthermore, we propose an algorithm based on the Auto Regression model to predict partition latency and adaptive offload partitions. Finally, we design offline and online optimization methods to minimize the computational and communication overhead on each device. Based on real-world prototype experiments, SPViT effectively reduces inference latency by 2.2x to 3.3x across four state-of-the-art models. © 2002-2012 IEEE.

关键词： Regression analysis

来源：评论

学校读者我要写书评

暂无评论

Channel Gain Map Estimation for Wireless Networks Based on Scatterer Model

引用

IEEE Transactions on Wireless Communications 2025年

作者： Sun, He Zhu, Lipeng Zhang, Rui National University of Singapore Department of Electrical and Computer Engineering 117583 Singapore The Chinese University of Hong Kong School of Science and Engineering Shenzhen Research Institute of Big Data Guangdong Shenzhen518172 China

Channel gain map (CGM) contains crucial large-scale fading information regarding wireless channels at specific frequency bands in wireless networks. Traditional CGM construction methods require either detailed propagation environment information or numerous channel measurements, rendering them practically cumbersome to implement. To overcome such difficulties, we propose in this paper a novel scatterer-based CGM construction framework to characterize the large-scale channel fading in a spatial region by estimating the scatterer responses from a few channel gain measurements at designated locations in the region. In particular, the region is divided into equally-spaced grids so as to smooth out the small-scale fading by averaging the channel power gain within each grid. As the average channel power gain within each grid can be expressed as a function of large-scale multi-path channel parameters such as the scatterer response coefficients and path-loss coefficients, an iterative algorithm is proposed to estimate these parameters based on only a sufficient number of channel gain measurements taken at random locations around each scatterer. Specifically, the proposed algorithm decomposes the parameter estimation problem into a set of univariate or linear estimation subproblems, which can be efficiently solved by one-dimensional (1D) line search and least-squares methods. Based on the estimated channel parameters, the CGM over the whole region can be estimated by calculating the average channel power gain for each grid. Simulation results under both the two-dimensional (2D) multipath channel model and the practical three-dimensional (3D) ray-tracing channel model verify that the proposed scatterer-based channel gain characterization can accurately capture the large-scale channel gain spatial distribution within a region, and the proposed CGM estimation framework outperforms other channel-measurement-based benchmarks in terms of estimation accuracy, while requiring a sign

关键词： Least squares approximations

来源：评论

学校读者我要写书评

暂无评论

PFedLAH: Personalized Federated Learning with Lookahead for Adaptive Cross-modal Hashing

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Chen, Yunfei Lin, Hongyu Yang, Zhan Long, Jun Central South University Big Data Institute School of Computer Science and Engineering ChangSha410000 China Central South University Dundee International Institute of Central South University Changsha410000 China

Cross-modal hashing enables efficient cross-modal retrieval by compressing multi-modal data into compact binary codes, but traditional methods primarily rely on centralized training, which is limited when handling large-scale distributed datasets. Federated learning presents a scalable alternative, yet existing federated frameworks for cross-modal hashing face challenges like data heterogeneity and imbalance, such as non-IID data distribution across clients. To address these challenges, we propose Personalized Federated learning with Lookahead for Adaptive cross-modal Hashing (PFedLAH) method, which combines Feature Adaptive Personalized Learning (FAPL) and Weight-aware Lookahead Adaptive Selection (WLAS) mechanism together. Initially, the FAPL module is designed for the client, enabling personalized learning to mitigate the effect of divergence between server and client resulting from non-IID data distribution, while the local optimization constraint mechanism is also integrated to avoid local optimization shift and ensure better alignment with global convergence. On the server side, WLAS module combines weight-aware adaptive client selection and gradient momentum lookahead to form a dynamic and intelligent client selection scheme, while enhancing the overall convergence and consistency through lookahead gradient prediction. Comprehensive experiments on widely used datasets, including MIRFlickr-25K, MS COCO, and NUS-WIDE, comparing state-of-the-art federated hashing methods, demonstrate the superior retrieval performance, robustness, and scalability of the PFedLAH method. © 2025 IEEE. All rights reserved.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

Optimization-Inspired Graph Neural Network for Cellular Network Optimization

引用

IEEE Transactions on Mobile Computing 2025年

作者： He, Pengcheng Tang, Yijia Xu, Fan Shi, Qingjiang Tongji University School of Computer Science and Technology Shanghai China Shenzhen Research Institute of Big Data Shenzhen China Tongji University School of Software Engineering Shanghai China Tongji University School of Electronic and Information Engineering Shanghai China

The rapid development of wireless communications have driven the need for careful optimization of network parameters to improve network performance and reduce operational cost. Traditional methods, however, struggle with the vast number of tunable parameters and lack scalability in diverse network scenarios. To address these challenges, this paper introduces an optimization-inspired bipartite graph neural network (Bi-GNN) approach for scalable network optimization. Our approach leverages the bipartite structure of network topologies, and incorporates a message-passing mechanism by unfolding the Zeroth-Order Block Coordinate Projected Gradient Descent (ZO-BCPGD) algorithm, which ensures not only high-performance optimization but also manageable computational demand. We demonstrate the permutation and dimensionality equivariance property of the Bi-GNN, which significantly enhances the model's generalizability across various network structures and sizes. Furthermore, we theoretically analyze the expressive power and generalization ability of the Bi-GNN, demonstrating its adeptness at complex network optimization tasks. The training process, parallel execution, and practical implementation techniques are also discussed to ensure the model's applicability in real-world scenarios. Numerical results verify that the Bi-GNN outperforms existing methods in both coverage ratios and computational cost. Furthermore, our approach exhibits robust scalability across various network scenarios, making it a versatile tool for optimizing a wide range of wireless networks. © 2002-2012 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Wireless Semantic Communication Based on Probability Distribution: An Initial Work

Wireless Semantic Communication Based on Probability Distrib...

引用

IEEE Conference on Wireless Communications and Networking

作者： Qingxiang Luo Yashuang Guo Aoran Zheng Zhitong Ni F. Richard Yu Victor C.M. Leung School of Computer Science and Technology Beijing Jiaotong University Big Data Technologies Centre University of Technology Sydney Department of Systems and Computer Engineering Carleton University Department of Electrical and Computer Engineering The University of British Columbia

ISBN: (数字)9798350368369

ISBN: (纸本)9798350368376

In the paper, we consider the general semantic transmission in wireless networks based on probability distribution. Firstly, we extract a multidimensional semantic probability distribution function, independent of any a specific wireless channel model, by using the variational inference technique. Secondly, we propose a new semantic similarity metric for measuring the difference between the received semantics and the expected semantics based on Kullback-Leibler divergence. Then, we formulate the semantic transmission problem as an optimization problem of transmission symbol adjustment with the aim to maximize the semantic similarity. Finally, we develop an optimal semantic transformation and transmission (STT) algorithm to obtain the optimal transmission symbol adjustment decision. This decision makes the closed-form expression of semantic transmission symbol available, which can realize lossless semantic transmission with energy constraint. Simulation results verify the effectiveness and robustness of the proposed STT algorithm.

关键词： Measurement Wireless networks Simulation Semantics Symbols Semantic communication Propagation losses Probability distribution Robustness Optimization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：