检索结果-内蒙古大学图书馆

Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors and Perceptual Insights

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hashmi, Ammarah Shahzad, Sahibzada Adil Lin, Chia-Wen Tsao, Yu Wang, Hsin-Min The Social Networks and Human-Centered Computing Program Taiwan International Graduate Program Institute of Information Science Academia Sinica Taipei11529 Taiwan The Institute of Information Systems and Applications National Tsing Hua University Hsinchu300044 Taiwan The Department of Computer Science National Chengchi University Taipei11529 Taiwan The Department of Electrical Engineering The Institute of Communications Engineering National Tsing Hua University Hsinchu300044 Taiwan The Research Center for Information Technology Innovation Academia Sinica Taipei11529 Taiwan The Institute of Information Science Academia Sinica Taipei11529 Taiwan

Deep Learning has been successfully applied in diverse fields, and its impact on deepfake detection is no exception. Deepfakes are fake yet realistic synthetic content that can be used deceitfully for political impersonation, phishing, slandering, or spreading misinformation. Despite extensive research on unimodal deepfake detection, identifying complex deepfakes through joint analysis of audio and visual streams remains relatively unexplored. To fill this gap, this survey first provides an overview of audiovisual deepfake generation techniques, applications, and their consequences, and then provides a comprehensive review of state-of-the-art methods that combine audio and visual modalities to enhance detection accuracy, summarizing and critically analyzing their strengths and limitations. Furthermore, we discuss existing open source datasets for a deeper understanding, which can contribute to the research community and provide necessary information to beginners who want to analyze deep learning-based audiovisual methods for video forensics. By bridging the gap between unimodal and multimodal approaches, this paper aims to improve the effectiveness of deepfake detection strategies and guide future research in cybersecurity and media integrity. © 2024, CC BY.

关键词： Audiovisual

Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hashmi, Ammarah Shahzad, Sahibzada Adil Lin, Chia-Wen Tsao, Yu Wang, Hsin-Min Social Networks and Human-Centered Computing Program Taiwan International Graduate Program Institute of Information Science Academia Sinica Taipei11529 Taiwan Institute of Information Systems and Applications National Tsing Hua University Hsinchu300044 Taiwan Department of Computer Science National Chengchi University Taipei11529 Taiwan Department of Electrical Engineering Institute of Communications Engineering National Tsing Hua University Hsinchu300044 Taiwan Research Center for Information Technology Innovation Academia Sinica Taipei11529 Taiwan Institute of Information Science Academia Sinica Taipei11529 Taiwan

The emergence of contemporary deepfakes has attracted significant attention in machine learning research, as artificial intelligence (AI) generated synthetic media increases the incidence of misinterpretation and is difficult to distinguish from genuine content. Currently, machine learning techniques have been extensively studied for automatically detecting deepfakes. However, human perception has been less explored. Malicious deepfakes could ultimately cause public and social problems. Can we humans correctly perceive the authenticity of the content of the videos we watch? The answer is obviously uncertain;therefore, this paper aims to evaluate the human ability to discern deepfake videos through a subjective study. We present our findings by comparing human observers to five state-of-the-art deepfake detection models. To this end, we used gamification concepts to provide 110 participants (55 native English speakers and 55 non-native English speakers) with a web-based platform where they could access a series of 40 videos (20 real and 20 fake) to determine their authenticity. Each participant performed the experiment twice with the same 40 videos in different random orders. The videos are manually selected from the FakeAVCeleb dataset. We found that all AI models performed significantly better than humans when evaluated on the same 40 videos. The study also reveals that while deception is not impossible, humans tend to overestimate their detection capabilities. Our experimental results may help benchmark human versus machine performance, advance forensics analysis, and enable adaptive countermeasures. © 2024, CC BY.

关键词： Audiovisual

Adaptive Contextual Caching for Mobile Edge Large Language Model Service

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Liu, Guangyuan Liu, Yinqiu Wang, Jiacheng Du, Hongyang Niyato, Dusit Kang, Jiawen Xiong, Zehui College of Computing and Data Science The Energy Research Institute @ NTU Interdisciplinary Graduate Program Nanyang Technological University Singapore College of Computing and Data Science Nanyang Technological University Singapore Department of Electrical and Electronic Engineering University of Hong Kong Hong Kong School of Automation Guangdong University of Technology China Pillar of Information Systems Technology and Design Singapore University of Technology and Design Singapore

Mobile edge Large Language Model (LLM) deployments face inherent constraints, such as limited computational resources and network bandwidth. Although Retrieval-Augmented Generation (RAG) mitigates some challenges by integrating external knowledge bases, inefficient cache management can still result in high retrieval latency and frequent cache updates. To address these issues, we propose an Adaptive Contextual Caching (ACC) framework that anticipates user needs by proactively caching semantically relevant data for mobile-edge LLMs. ACC utilizes a deep reinforcement learning (DRL) module to refine cache replacement policies, balancing user context, document similarity, and the overhead associated with cache misses. Experimental results demonstrate that ACC increases cache hit rates to over 80% after only 11 training episodes, outperforming FIFO, LRU, and semantic-only caching while reducing retrieval latency by up to 40%. In particular, ACC also reduces local caching overhead (i.e., the cost of updating the cache when a miss occurs) by as much as 55%, enabling scalable, low-latency LLM services in resource-constrained edge environments. © 2025, CC BY.

关键词： Deep reinforcement learning

AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection

学校读者我要写书评

暂无评论

IEEE Transactions on Cognitive and Developmental Systems 2025年

作者： Hashmi, Ammarah Shahzad, Sahibzada Adil Lin, Chia-Wen Tsao, Yu Wang, Hsin-Min Taiwan International Graduate Program Academia Sinica Social Networks and Human-Centered Computing Program Taipei11529 Taiwan National Tsing Hua University Institute of Information Systems and Applications Hsinchu30013 Taiwan National Chengchi University Department of Computer Science Taipei11605 Taiwan National Tsing Hua University Department of Electrical Engineering Institute of Communications Engineering Hsinchu300044 Taiwan Academia Sinica Research Center for Information Technology Innovation Taipei11529 Taiwan

The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous studies on detecting artificial intelligence-generated fake videos only utilize visual modality or audio modality. While some methods exploit audio and visual modalities to detect forged videos, they have not been comprehensively evaluated on multimodal datasets of deepfake videos involving acoustic and visual manipulations, and are mostly based on convolutional neural networks with low detection accuracy. Considering that human cognition instinctively integrates multisensory information including audio and visual cues to perceive and interpret content and the success of transformer in various fields, this study introduces the audio-visual transformer-based ensemble network (AVTENet). This innovative framework tackles the complexities of deepfake technology by integrating both acoustic and visual manipulations to enhance the accuracy of video forgery detection. Specifically, the proposed model integrates several purely transformer-based variants that capture video, audio, and audio-visual salient cues to reach a consensus in prediction. For evaluation, we use the recently released benchmark multimodal audio-video FakeAVCeleb dataset. For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset. Experimental results show that the proposed model outperforms all existing methods and achieves state-of-the-art performance on Testset-I and Testset-II of the FakeAVCeleb dataset. We also compare AVTENet against humans in detecting video forgery. The results show that AVTENet significantly outperforms humans. © 2016 IEEE.

关键词： audio-visual AVTENet Deepfake detection

A Hybrid Learning Approach for Automatic Data Labelling and Anomaly Detection in IoT Networks

学校读者我要写书评

暂无评论

A Hybrid Learning Approach for Automatic Data Labelling and ...

Artificial Intelligence (ICAI), International Conference on

作者： Rimsha Kanwal Umara Noor Zahid Rashid Department of Computer Science Faculty of Computing and Information Technology International Islamic University Islamabad Pakistan Technology Management Economics and Policy Program College of Engineering Seoul National University Seoul South Korea

Internet of Things (IoT) is an environment in which digital equipment is augmented with sensors to share and receive data through network. When devices share data it can be effected by anomalies or any attack due to corrupted data or by any other uncertainty and ambiguity in data. The data can also be corrupted through a damage in device. These attacks or anomalies damage the working of the IoT networks. The anomalous data can be detected through detection techniques however most anomaly detection techniques depend upon labelled data but for IoT datasets, usually class labels are not available. Labeling is performed by a manual process which is time consuming and also costly. As data in IoT increases day by day so there is a need to label and classify data for future unseen data. In this paper a hybrid algorithm is proposed in which both clustering and classification techniques are applied for automatic labeling and classifying on IoT dataset. The model contains two function. In the first phase k-means clustering is employed for labelling dataset instances as normal or anomalous. In the second phase labelled dataset is used to train Random Forest model to detect anomalies in IoT networks. The results show that the proposed model is detecting anomalies in IoT networks with an accuracy 98%, precision 98 %, recall 98%, and F-meausre 0.98%.

关键词：

Predicting User Preferences with Neural Network Learning to Rank Method 5

学校读者我要写书评

暂无评论

Predicting User Preferences with Neural Network Learning to ...

5th IEEE International Conference on information Technology, information Systems and Electrical Engineering, ICITISEE 2021

作者： Qomariyah, Nunung Nurul Fajar, Ahmad Nurul Bina Nusantara University Faculty Of Computing And Media Computer Science Department Jakarta11480 Indonesia Bina Nusantara University BINUS Graduate Program-Master Of Information Systems Management Information Systems Management Department Jakarta11480 Indonesia

ISBN: (纸本)9781665401968

User preference learning has been around for many years. This is a common problem arise in e-commerce system, where the companies need to understand their customers in order to sell the correct products to their target customers. The user preference is also needed in the movie domain area, as the film companies who produce the movie also need to understand how the customers value their movie products, whether the market like it or not. This can help them to predict the best components for the future movie. In order to produce the recommendation to the customer, we can show a list of item in a correct order, sorted from the most relevant/the most liked one to the most irrelevant item. In this paper, we utilize the Learning to Rank (LTR) approach from information retrieval domain area to learn the user preference in recommender system domain. We applied LTR with pairwise approach from the two most popular algorithms in this area, i.e. RankNet and LambdaRank, on 25, 345 user ratings from 5, 000 movie dataset and evaluate both algorithms. We used a Neural Network architecture as the learning mechanism. We also measuring the speed performance of the two algorithms. © 2021 IEEE.

关键词： Sales

HAAQI-Net: A Non-intrusive Neural Music Audio Quality Assessment Model for Hearing Aids

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wisnu, Dyah A.M.G. Rini, Stefano Zezario, Ryandhimas E. Wang, Hsin-Min Tsao, Yu Taiwan International Graduate Program—Social Network and Human Centered Computing Institute of Information Science Academia Sinica Taipei Taiwan College of Informatics National Chengchi University Taipei Taiwan Institute of Communications Engineering National Yang Ming Chiao Tung University Hsinchu Taiwan Research Center for Information Technology Innovation Academia Sinica Taipei Taiwan Institute of Information Science Academia Sinica Taipei Taiwan

This paper introduces HAAQI-Net, a non-intrusive deep learning-based music audio quality assessment model for hearing aid users. Unlike traditional methods like the Hearing Aid Audio Quality Index (HAAQI) that require intrusive reference signal comparisons, HAAQI-Net offers a more accessible and computationally efficient alternative. By utilizing a Bidirectional Long Short-Term Memory (BLSTM) architecture with attention mechanisms and features extracted from the pre-trained BEATs model, it can predict HAAQI scores directly from music audio clips and hearing loss patterns. Experimental results demonstrate HAAQI-Net’s effectiveness, achieving a Linear Correlation Coefficient (LCC) of 0.9368, a Spearman’s Rank Correlation Coefficient (SRCC) of 0.9486, and a Mean Squared Error (MSE) of 0.0064 and inference time significantly reduces from 62.52 to 2.54 seconds. To address computational overhead, a knowledge distillation strategy was applied, reducing parameters by 75.85% and inference time by 96.46%, while maintaining strong performance (LCC: 0.9071, SRCC: 0.9307, MSE: 0.0091). To expand its capabilities, HAAQI-Net was adapted to predict subjective human scores like the Mean Opinion Score (MOS) through fine-tuning. This adaptation significantly improved prediction accuracy, validated through statistical analysis. Furthermore, the robustness of HAAQI-Net was evaluated under varying Sound Pressure Level (SPL) conditions, revealing optimal performance at a reference SPL of 65 dB, with accuracy gradually decreasing as SPL deviated from this point. The advancements in subjective score prediction, SPL robustness, and computational efficiency position HAAQI-Net as a scalable solution for music audio quality assessment in hearing aid applications, contributing to efficient and accurate models in audio signal processing and hearing aid technology. Copyright © 2024, The Authors. All rights reserved.

关键词： Mean square error

A BCLC Staging System for Hepatocellular Carcinoma using Swin Transformer and CT Imaging 46

学校读者我要写书评

暂无评论

A BCLC Staging System for Hepatocellular Carcinoma using Swi...

46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2024

作者： Chang, Shun-Cheng Wang, Pochuang Wang, Weichung Su, Tung-Hung Kao, Jia-Horng Lin, Che Taipei10617 Taiwan Department of Computer Science and Information Engineering Ntu Taipei10617 Taiwan Institute of Applied Mathematical Sciences Ntu Taipei10617 Taiwan Division of Gastroenterology and Hepatology Department of Internal Medicine National Taiwan University Hospital Taipei10048 Taiwan Department of Electrical Engineering Ntu Taipei10617 Taiwan Center for Advanced Computing and Imaging in Biomedicine Ntu Taipei10617 Taiwan Smart Medicine and Health Informatics Program Ntu Taipei10617 Taiwan

ISBN: (纸本)9798350371499

The Barcelona Clinic Liver Cancer (BCLC) staging system plays a crucial role in clinical planning, offering valuable insights for effectively managing hepatocellular carcinoma. Accurate prediction of BCLC stages can significantly ease the workload on radiologists. However, few datasets are explicitly designed for discerning BCLC stages. Despite the common practice of appending BCLC labels to clinical data within datasets, the inherent imbalance in BCLC distribution is further amplified by the diverse purposes for which datasets are curated. In this study, we aim to develop a BCLC staging system using the advanced Swin Transformer model. Additionally, we explore the integration of two datasets, each originally intended for separate objectives, highlighting the critical challenge of preserving class distribution in practical study designs. This exploration is pivotal for ensuring the applicability of our developed staging system in the designed clinical settings. Our resulting BCLC staging system demonstrates an accuracy of 55.81% (±7.8%), contributing to advancing medical image-based research for predicting BCLC stages. © 2024 IEEE.

关键词： Distribution transformers

ReViT: A Hybrid Approach for BCLC Staging of Hepatocellular Carcinoma Using 3D CT with Multiple Instance Learning

学校读者我要写书评

暂无评论

ReViT: A Hybrid Approach for BCLC Staging of Hepatocellular ...

2024 IEEE-EMBS International Conference on Biomedical and Health Informatics, BHI 2024

作者： Chang, Shun-Cheng Yu, Hsin-Pei Hsieh, Yi-Hsien Wang, Pochuang Wang, Weichung Su, Tung-Hung Kao, Jia-Horng Lin, Che Taipei10617 Taiwan Department of Computer Science and Information Engineering Ntu Taipei10617 Taiwan Institute of Applied Mathematical Sciences Ntu Taipei10617 Taiwan National Taiwan University Hospital Division of Gastroenterology and Hepatology Department of Internal Medicine Taipei10048 Taiwan Department of Electrical Engineering Ntu Taipei10617 Taiwan Center for Advanced Computing and Imaging in Biomedicine Ntu Taipei10617 Taiwan Smart Medicine and Health Informatics Program Ntu Taipei10617 Taiwan

ISBN: (纸本)9798350351552

Deep learning has revolutionized medical imaging, offering advanced methods for accurate diagnosis and treatment planning. The BCLC staging system is crucial for staging Hepatocellular Carcinoma (HCC), a high-mortality cancer. An automated BCLC staging system could significantly enhance diagnosis and treatment planning efficiency. However, we found that BCLC staging, which is directly related to the size and number of liver tumors, aligns well with the principles of the Multiple Instance Learning (MIL) framework. To effectively achieve this, we proposed a new preprocessing technique called Masked Cropping and Padding(MCP), which addresses the variability in liver volumes and ensures consistent input sizes. This technique preserves the structural integrity of the liver, facilitating more effective learning. Furthermore, we introduced Re ViT, a novel hybrid model that integrates the local feature extraction capabilities of Convolutional Neural Networks (CNNs) with the global context modeling of Vision Transformers (ViTs). Re ViT leverages the strengths of both architectures within the MIL framework, enabling a robust and accurate approach for BCLC staging. We will further explore the trade-off between performance and interpretability by employing TopK Pooling strategies, as our model focuses on the most informative instances within each bag. © 2024 IEEE.

关键词： Convolutional neural networks