检索结果-内蒙古大学图书馆

CapST: Leveraging Capsule networks and Temporal Attention for Accurate Model Attribution in Deep-fake Videos

ACM Transactions on Multimedia computing, Communications and Applications 2025年第4期21卷

作者： Ahmad, Wasim Peng, Yan-Tsung Chang, Yuan-Hao Ganfure, Gaddisa Olani Khan, Sarwar Social Networks and Human-Centered Computing Taiwan International Graduate Program Institute of Information Science Academia Sinica Taipei Taiwan Department of Computer Science National Chengchi University Taipei Taiwan Institute of Information Science Academia Sinica Taipei Taiwan Department of Computer Science Dire Dawa University Dire Dawa Ethiopia Social Networks and Human-Centered Computing Taiwan International Graduate Program Research Center for Information Technology Innovation Academia Sinica Taipei Taiwan

Deep-fake videos, generated through AI face-swapping techniques, have garnered considerable attention due to their potential for impactful impersonation attacks. While existing research primarily distinguishes real from fake videos, attributing a deep-fake to its specific generation model or encoder is crucial for forensic investigation, enabling precise source tracing and tailored countermeasures. This approach not only enhances detection accuracy by leveraging unique model-specific artifacts but also provides insights essential for developing proactive defenses against evolving deep-fake techniques. Addressing this gap, this article investigates the model attribution problem for deep-fake videos using two datasets-Deepfakes from Different Models (DFDM) and GANGen-Detection, which comprise deep-fake videos and images generated by GAN models. We select only fake images from the GANGen-Detection dataset to align with the DFDM dataset, which specifies the goal of this study, focusing on model attribution rather than real/fake classification. This study formulates deep-fake model attribution as a multiclass classification task, introducing a novel Capsule-Spatial-Temporal (CapST) model that effectively integrates a modified VGG19 (utilizing only the first 26 out of 52 layers) for feature extraction, combined with Capsule networks and a Spatio-Temporal attention mechanism. The Capsule module captures intricate feature hierarchies, enabling robust identification of deep-fake attributes, while a video-level fusion technique leverages temporal attention mechanisms to process concatenated feature vectors and capture temporal dependencies in deep-fake videos. By aggregating insights across frames, our model achieves a comprehensive understanding of video content, resulting in more precise predictions. Experimental results on the DFDM and GANGen-Detection datasets demonstrate the efficacy of CapST, achieving substantial improvements in accurately categorizing deep-fake videos

关键词： Computer forensics

来源：评论

学校读者我要写书评

暂无评论

AVTENet: A human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection

引用

IEEE Transactions on Cognitive and Developmental Systems 2025年

作者： Hashmi, Ammarah Shahzad, Sahibzada Adil Lin, Chia-Wen Tsao, Yu Wang, Hsin-Min Taiwan International Graduate Program Academia Sinica Social Networks and Human-Centered Computing Program Taipei11529 Taiwan National Tsing Hua University Institute of Information Systems and Applications Hsinchu30013 Taiwan National Chengchi University Department of Computer Science Taipei11605 Taiwan National Tsing Hua University Department of Electrical Engineering Institute of Communications Engineering Hsinchu300044 Taiwan Academia Sinica Research Center for Information Technology Innovation Taipei11529 Taiwan

The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous studies on detecting artificial intelligence-generated fake videos only utilize visual modality or audio modality. While some methods exploit audio and visual modalities to detect forged videos, they have not been comprehensively evaluated on multimodal datasets of deepfake videos involving acoustic and visual manipulations, and are mostly based on convolutional neural networks with low detection accuracy. Considering that human cognition instinctively integrates multisensory information including audio and visual cues to perceive and interpret content and the success of transformer in various fields, this study introduces the audio-visual transformer-based ensemble network (AVTENet). This innovative framework tackles the complexities of deepfake technology by integrating both acoustic and visual manipulations to enhance the accuracy of video forgery detection. Specifically, the proposed model integrates several purely transformer-based variants that capture video, audio, and audio-visual salient cues to reach a consensus in prediction. For evaluation, we use the recently released benchmark multimodal audio-video FakeAVCeleb dataset. For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset. Experimental results show that the proposed model outperforms all existing methods and achieves state-of-the-art performance on Testset-I and Testset-II of the FakeAVCeleb dataset. We also compare AVTENet against humans in detecting video forgery. The results show that AVTENet significantly outperforms humans. © 2016 IEEE.

关键词： audio-visual AVTENet Deepfake detection

来源：评论

学校读者我要写书评

暂无评论

HAAQI-Net: A Non-Intrusive Neural Music Audio Quality Assessment Model for Hearing Aids

引用

IEEE Transactions on Audio, Speech and Language Processing 2025年 33卷 1877-1892页

作者： Dyah A. M. G. Wisnu Stefano Rini Ryandhimas E. Zezario Hsin-Min Wang Yu Tsao Taiwan International Graduate Program—Social Network and Human-Centered Computing Institute of Information Science Academia Sinica Taipei Taiwan College of Informatics National Chengchi University Taipei Taiwan Institute of Communications Engineering National Yang Ming Chiao Tung University Hsinchu Taiwan Research Center for Information Technology Innovation Academia Sinica Taipei Taiwan Institute of Information Science Academia Sinica Taipei Taiwan Department of Electrical Engineering Chung Yuan Christian University Taiwan

This paper introduces HAAQI-Net, a non-intrusive music audio quality assessment model for hearing aid users. Unlike traditional methods such as Hearing Aid Audio Quality Index (HAAQI), which requires intrusive reference signal comparisons, HAAQI-Net offers a more accessible and computationally efficient alternative. Leveraging a bidirectional long short-term memory architecture with attention mechanisms and features extracted from a pre-trained BEATs model, it can predict HAAQI scores directly from music audio clips and hearing loss patterns. The experimental results demonstrate that, compared to the traditional HAAQI as the reference, HAAQI-Net achieves a linear correlation coefficient (LCC) of 0.9368, a Spearman's rank correlation coefficient (SRCC) of 0.9486, and a mean squared error (MSE) of 0.0064, while significantly reducing the inference time from 62.52 seconds to 2.54 seconds. Furthermore, a knowledge distillation strategy was applied, reducing the parameters by 75.85% and inference time by 96.46%, while maintaining strong performance (LCC: 0.9071, SRCC: 0.9307, MSE: 0.0091). To expand its capabilities, HAAQI-Net was adapted to predict subjective human scores, mean opinion score (MOS), by fine-tuning. This adaptation significantly improved the prediction accuracy. Furthermore, the robustness of HAAQI-Net was evaluated under varying sound pressure level (SPL) conditions, revealing optimal performance at a reference SPL of 65 dB, with the accuracy gradually decreasing as SPL deviated from this point. The advancements in subjective score prediction, SPL robustness, and computational efficiency position HAAQI-Net as a reliable solution for music audio quality assessment, significantly contributing to the development of efficient and accurate models in audio signal processing and hearing aid technology.

关键词： Hearing aids Quality assessment Training Feature extraction Speech processing Auditory system Computational modeling Adaptation models Transformers Multiple signal classification

来源：评论

学校读者我要写书评

暂无评论

An empirical study of PM2.5 forecasting using neural network

An empirical study of PM2.5 forecasting using neural network

引用

2017 IEEE SmartWorld Ubiquitous Intelligence and computing, Advanced and Trusted Computed, Scalable computing and Communications, Cloud and Big Data computing, Internet of People and Smart City Innovation, SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI 2017

作者： Mahajan, Sachit Chen, Ling-Jyh Tsai, Tzu-Chieh Social Networks and Human Centered Computing Program Taiwan International Graduate Program Taiwan Institute of Information Science Academia Sinica Taiwan Department of Computer Science National Chengchi University Taiwan

ISBN: (纸本)9781538604342

In the recent years, a lot of efforts have been made to regulate air pollutant levels in most of the developed and developing countries. Fine particulate matter (PM2.5) is considered to be one of the major reasons behind deteriorating public health and a lot of efforts are being made to keep a check on PM2.5 levels. Accurately forecasting PM2.5 level is a challenging task and has been highly dependent on model based approaches. In this paper, we explore new possibilities to hourly forecast PM2.5. Choosing the right forecasting model becomes a very important aspect when it comes to improvement in prediction accuracy. We used Neural Network Autoregression (NNAR) method for the prediction task. The paper also provides a comparative analysis of prediction performance for additive version of Holt-Winters method, autoregressive integrated moving average (ARIMA) model and NNAR model. The experimentation and evaluation is done using real world measurement data from Airbox Project, which shows that our proposed method accurately does the prediction with significantly low error. © 2017 IEEE.

关键词： Neural networks

来源：评论

学校读者我要写书评

暂无评论

Audio-Driven Facial Landmark Generation in Violin Performance using 3DCNN Network with Self Attention Model 48

Audio-Driven Facial Landmark Generation in Violin Performanc...

引用

48th IEEE international Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Lin, Ting-Wei Liu, Chao-Lin Su, Li Taiwan International Graduate Program Social Networks and Human-Centered Computing Taiwan Academia Sinica Institute of Information Science Taipei Taiwan National Chengchi University Department of Computer Science Taipei Taiwan

ISBN: (纸本)9781728163277

In a music scenario, both auditory and visual elements are essential to achieve an outstanding performance. Recent research has focused on the generation of body movements or fingering from audio in music performance. The audio-driven face generation technique in music performance is still deficient. In this paper, we compile a violin soundtrack and facial expression dataset (VSFE) for modeling facial expressions in violin performance. To our knowledge, this is the first dataset mapping the relationship between violin performance audio and musicians' facial expressions. We then propose a 3DCNN network with self-attention and residual blocks for audio-driven facial expression generation. In the experiments, we compare our methods with three baselines on talking face generation. The codes and dataset are available on the Github (https://***/kevinlin91/icassp-music2face). © 2023 IEEE.

关键词： Music

来源：评论

学校读者我要写书评

暂无评论

Demo: CAR: The cleanest air routing algorithm for path navigation with minimal PM2.5 exposure on the move 16

Demo: CAR: The cleanest air routing algorithm for path navig...

引用

16th ACM international Conference on Mobile Systems, Applications, and Services,MobiSys 2018

作者： Mahajan, Sachit Tang, Yu-Siou Wu, Dong-Yi Tsai, Tzu-Chieh Chen, Ling-Jyh Social Networks and Human Centered Computing Program TIGP Taiwan Institute of Information Science Academia Sinica Taiwan National Chengchi University Taiwan

来源：评论

学校读者我要写书评

暂无评论

Text and Speech Adjustable Bimodal Emotion Recognition in Conversational Interface 1

Text and Speech Adjustable Bimodal Emotion Recognition in Co...

引用

1st IEEE international Symposium on Consumer Technology, ISCT 2024

作者： Rinartha, Komang Ku, Lun-Wei Social Networks and Human Centered Computing Program Taiwan International Graduate Program Institute of Information Science 128 Academia Road Sec. 2. Nangang Taipei115 Taiwan Institute of Information Science Academia Sinica Taipei Taiwan

ISBN: (纸本)9798350365191

Speech emotion recognition (SER) is a Machine Learning (ML) topic that is now receiving a lot of research attention. This can be attributed to its growing capacity, improvements in algorithms, and utilization in practical situations. Furthermore, speech contains not just auditory information but also lexical information. Both speech-based emotion recognition and text-based emotion recognition, called bimodal emotion recognition, can be highly beneficial for human interaction. Take an example, maintaining the communication between elder parents who are left behind after their children leave for education, employment, and family dynamics who can be greatly affected, both physically and emotionally, by feelings of loneliness and social isolation. Bimodal emotion recognition employs both text-based and audio-based methods to identify emotions. Each algorithm was trained separately and then combined using a late fusion technique. RoBERTa text-based emotion recognition can be combined with CNN audio-based emotion recognition by using late fusion to create a bimodal emotion recognition that can obtain more than 50% of accuracy by combining text and audio in a similar proportion or relying on text more than audio. By adjusting the proportion of text or audio emotion recognition, we can get better result of recognition and it can be customized by the user's preference. We elaborate on creating bimodal emotion recognition, conversational interface, a framework for emotion monitoring system, and give the example of a combined algorithm to perform emotion recognition. © 2024 IEEE.

关键词： Emotion Recognition

来源：评论

学校读者我要写书评

暂无评论

LOCI: A Mobile QA System with Multimodal Motivation Scheme for Local Intent Questions in Dynamic social networks 91

LOCI: A Mobile QA System with Multimodal Motivation Scheme f...

引用

91st IEEE Vehicular Technology Conference, VTC Spring 2020

作者： Ali, Imad Chang, Ronald Y. Hsu, Cheng-Hsin Lee, Chi-Han Institute of Information Science Academia Sinica Taiwan International Graduate Program Social Networks and Human-Centered Computing Taipei Taiwan Research Center for Information Technology Innovation Academia Sinica Taipei Taiwan National Tsing Hua University Department of Computer Science Hsinchu Taiwan

ISBN: (纸本)9781728152073

Recent research has shown that people are frequently searching for information with local intent. The existing systems fail to answer this class of questions as they are not mainly designed to handle it. In this paper, we propose LOCI, a mobile question answer system with a multimodal motivation scheme for local intent questions where users provide timely and accurate answers in a dynamic social network. LOCI mines users' shared and contextual information to identify the most relevant users for each question. The proposed multimodal motivation scheme exploits users' social ties and monetary rewards to raise their basic motivation levels. LOCI assigns questions to the best answerers via our three proposed algorithms: (i) LOCI-RT, which immediately assigns newly arrived questions/users to available users/questions, (ii) LOCI-LO, which uses short-time batches for locally optimal questions-users assignment, and (iii) LOCI-AD, which assigns short- and long-deadline questions immediately and via batching, respectively. We conduct: (i) a survey to support our assumptions regarding the proposed multimodal motivation scheme, and (ii) trace-driven experiments to evaluate the performance of the three proposed LOCI's algorithms. © 2020 IEEE.

关键词： Motivation

来源：评论

学校读者我要写书评

暂无评论

On Efficient Processing of Queries for Live Multi-Streaming Soiree Organization

引用

IEEE Transactions on Services computing 2023年第4期16卷 2812-2826页

作者： Kankeu Fotsing, C.P. Shen, Chih-Ya Huang, Liang-Hao Chen, Yi-Shin Lee, Wang-Chien Yang, De-Nian International Graduate Program Academia Sinica Social Networks and Human-Centered Computing Program Taipei115 Taiwan National Tsing Hua University Institute of Information Systems and Applications Hsinchu300 Taiwan National Tsing Hua University Department of Computer Science Hsinchu300 Taiwan Academia Sinica Institute of Information Science Taipei115 Taiwan Pennsylvania State University Department of Computer Science and Engineering State CollegePA16801 United States

Real-time social interactions and multi-streaming are two critical features of live streaming services. In this paper, we formulate a new fundamental service query, social-aware Diverse and Preferred Organization Query (SDSQ), that jointly selects a set of diverse and preferred live streaming channels and a group of socially tight viewers for organization of a live multi-streaming soiree. We prove that SDSQ is NP-hard and inapproximable within any factor, and design SDSSel, a 2-approximation algorithm with a guaranteed error bound. Moreover, we study SDSQ-T, a special case of SDSQ, where the social graph is a threshold graph, and propose TDSSel, a 2-approximation algorithm without any error to SDSQ-T. We propose two pruning strategies, PCP and CDP to boost SDSSel and TDSSel. We further propose a more challenging but practical service query, Generalized social-aware Maximum Preferred and Diverse Query (GSPQ), a generalization of SDSQ. We design GPDSel, a 4-approximation algorithm for GSPQ with a guaranteed error bound. We propose a strategy to improve the approximation ratios of the proposed algorithms. A user study on Twitch validates SDSQ, and the large-scale experiments on real datasets demonstrate the superiority of the proposed algorithms over several baselines for live-streaming services. © 2008-2012 IEEE.

关键词： Approximation algorithms

来源：评论

学校读者我要写书评

暂无评论

TERMS: textual emotion recognition in multidimensional space

引用

Applied Intelligence 2023年第3期53卷 2673-2693页

作者： Ghafoor, Yusra Jinping, Shi Calderon, Fernando H. Huang, Yen-Hao Chen, Kuan-Ta Chen, Yi-Shin Social Networks and Human-Centered Computing Taiwan International Graduate Program Institute of Information Science Academia Sinica. Institute of Information Systems and Applications National Tsing Hua University Hsinchu City Taiwan Institute of Information Systems and Applications National Tsing Hua University Hsinchu City Taiwan Institute of Information Science Academia Sinica Taipei Taiwan Department of Computer Science National Tsing Hua University Hsinchu City Taiwan

Microblogs generate a vast amount of data in which users express their emotions regarding almost all aspects of everyday life. Capturing affective content from these context-dependent and subjective texts is a challenging task. We propose an intelligent probabilistic model for textual emotion recognition in multidimensional space (TERMS) that captures the subjective emotional boundaries and contextual information embedded in a text for robust emotion recognition. It is implausible with discrete label assignment;therefore, the model employs a soft assignment by mapping varying emotional perceptions in a multidimensional space and generates them as distributions via the Gaussian mixture model (GMM). To strengthen emotion distributions, TERMS integrates a probabilistic emotion classifier that captures the contextual and linguistic information from texts. The integration of these aspects, the context-aware emotion classifier and the learned GMM parameters provide a complete coverage for accurate emotion recognition. The large-scale experimentation shows that compared to baseline and state-of-the-art models, TERMS achieved better performance in terms of distinguishability, prediction, and classification performance. In addition, TERMS provide insights on emotion classes, the annotation patterns, and the models application in different scenarios. © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

关键词： Emotion Recognition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：