检索结果-内蒙古大学图书馆

AVTENet: A human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble network for Video Deepfake Detection

IEEE Transactions on Cognitive and Developmental Systems 2025年

作者： Hashmi, Ammarah Shahzad, Sahibzada Adil Lin, Chia-Wen Tsao, Yu Wang, Hsin-Min Taiwan International Graduate Program Academia Sinica Social Networks and Human-Centered Computing Program Taipei11529 Taiwan National Tsing Hua University Institute of Information Systems and Applications Hsinchu30013 Taiwan National Chengchi University Department of Computer Science Taipei11605 Taiwan National Tsing Hua University Department of Electrical Engineering Institute of Communications Engineering Hsinchu300044 Taiwan Academia Sinica Research Center for Information Technology Innovation Taipei11529 Taiwan

The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous studies on detecting artificial intelligence-generated fake videos only utilize visual modality or audio modality. While some methods exploit audio and visual modalities to detect forged videos, they have not been comprehensively evaluated on multimodal datasets of deepfake videos involving acoustic and visual manipulations, and are mostly based on convolutional neural networks with low detection accuracy. Considering that human cognition instinctively integrates multisensory information including audio and visual cues to perceive and interpret content and the success of transformer in various fields, this study introduces the audio-visual transformer-based ensemble network (AVTENet). This innovative framework tackles the complexities of deepfake technology by integrating both acoustic and visual manipulations to enhance the accuracy of video forgery detection. Specifically, the proposed model integrates several purely transformer-based variants that capture video, audio, and audio-visual salient cues to reach a consensus in prediction. For evaluation, we use the recently released benchmark multimodal audio-video FakeAVCeleb dataset. For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset. Experimental results show that the proposed model outperforms all existing methods and achieves state-of-the-art performance on Testset-I and Testset-II of the FakeAVCeleb dataset. We also compare AVTENet against humans in detecting video forgery. The results show that AVTENet significantly outperforms humans. © 2016 IEEE.

关键词： audio-visual AVTENet Deepfake detection

来源：评论

学校读者我要写书评

暂无评论

An empirical study of PM2.5 forecasting using neural network

An empirical study of PM2.5 forecasting using neural network

引用

2017 IEEE SmartWorld Ubiquitous Intelligence and computing, Advanced and Trusted Computed, Scalable computing and Communications, Cloud and Big Data computing, Internet of People and Smart City Innovation, SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI 2017

作者： Mahajan, Sachit Chen, Ling-Jyh Tsai, Tzu-Chieh Social Networks and Human Centered Computing Program Taiwan International Graduate Program Taiwan Institute of Information Science Academia Sinica Taiwan Department of Computer Science National Chengchi University Taiwan

ISBN: (纸本)9781538604342

In the recent years, a lot of efforts have been made to regulate air pollutant levels in most of the developed and developing countries. Fine particulate matter (PM2.5) is considered to be one of the major reasons behind deteriorating public health and a lot of efforts are being made to keep a check on PM2.5 levels. Accurately forecasting PM2.5 level is a challenging task and has been highly dependent on model based approaches. In this paper, we explore new possibilities to hourly forecast PM2.5. Choosing the right forecasting model becomes a very important aspect when it comes to improvement in prediction accuracy. We used Neural network Autoregression (NNAR) method for the prediction task. The paper also provides a comparative analysis of prediction performance for additive version of Holt-Winters method, autoregressive integrated moving average (ARIMA) model and NNAR model. The experimentation and evaluation is done using real world measurement data from Airbox Project, which shows that our proposed method accurately does the prediction with significantly low error. © 2017 IEEE.

关键词： Neural networks

来源：评论

学校读者我要写书评

暂无评论

Audio-Driven Facial Landmark Generation in Violin Performance using 3DCNN network with Self Attention Model 48

Audio-Driven Facial Landmark Generation in Violin Performanc...

引用

48th IEEE international Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Lin, Ting-Wei Liu, Chao-Lin Su, Li Taiwan International Graduate Program Social Networks and Human-Centered Computing Taiwan Academia Sinica Institute of Information Science Taipei Taiwan National Chengchi University Department of Computer Science Taipei Taiwan

ISBN: (纸本)9781728163277

In a music scenario, both auditory and visual elements are essential to achieve an outstanding performance. Recent research has focused on the generation of body movements or fingering from audio in music performance. The audio-driven face generation technique in music performance is still deficient. In this paper, we compile a violin soundtrack and facial expression dataset (VSFE) for modeling facial expressions in violin performance. To our knowledge, this is the first dataset mapping the relationship between violin performance audio and musicians' facial expressions. We then propose a 3DCNN network with self-attention and residual blocks for audio-driven facial expression generation. In the experiments, we compare our methods with three baselines on talking face generation. The codes and dataset are available on the Github (https://***/kevinlin91/icassp-music2face). © 2023 IEEE.

关键词： Music

来源：评论

学校读者我要写书评

暂无评论

Text and Speech Adjustable Bimodal Emotion Recognition in Conversational Interface 1

Text and Speech Adjustable Bimodal Emotion Recognition in Co...

引用

1st IEEE international Symposium on Consumer Technology, ISCT 2024

作者： Rinartha, Komang Ku, Lun-Wei Social Networks and Human Centered Computing Program Taiwan International Graduate Program Institute of Information Science 128 Academia Road Sec. 2. Nangang Taipei115 Taiwan Institute of Information Science Academia Sinica Taipei Taiwan

ISBN: (纸本)9798350365191

Speech emotion recognition (SER) is a Machine Learning (ML) topic that is now receiving a lot of research attention. This can be attributed to its growing capacity, improvements in algorithms, and utilization in practical situations. Furthermore, speech contains not just auditory information but also lexical information. Both speech-based emotion recognition and text-based emotion recognition, called bimodal emotion recognition, can be highly beneficial for human interaction. Take an example, maintaining the communication between elder parents who are left behind after their children leave for education, employment, and family dynamics who can be greatly affected, both physically and emotionally, by feelings of loneliness and social isolation. Bimodal emotion recognition employs both text-based and audio-based methods to identify emotions. Each algorithm was trained separately and then combined using a late fusion technique. RoBERTa text-based emotion recognition can be combined with CNN audio-based emotion recognition by using late fusion to create a bimodal emotion recognition that can obtain more than 50% of accuracy by combining text and audio in a similar proportion or relying on text more than audio. By adjusting the proportion of text or audio emotion recognition, we can get better result of recognition and it can be customized by the user's preference. We elaborate on creating bimodal emotion recognition, conversational interface, a framework for emotion monitoring system, and give the example of a combined algorithm to perform emotion recognition. © 2024 IEEE.

关键词： Emotion Recognition

来源：评论

学校读者我要写书评

暂无评论

LOCI: A Mobile QA System with Multimodal Motivation Scheme for Local Intent Questions in Dynamic social networks 91

LOCI: A Mobile QA System with Multimodal Motivation Scheme f...

引用

91st IEEE Vehicular Technology Conference, VTC Spring 2020

作者： Ali, Imad Chang, Ronald Y. Hsu, Cheng-Hsin Lee, Chi-Han Institute of Information Science Academia Sinica Taiwan International Graduate Program Social Networks and Human-Centered Computing Taipei Taiwan Research Center for Information Technology Innovation Academia Sinica Taipei Taiwan National Tsing Hua University Department of Computer Science Hsinchu Taiwan

ISBN: (纸本)9781728152073

Recent research has shown that people are frequently searching for information with local intent. The existing systems fail to answer this class of questions as they are not mainly designed to handle it. In this paper, we propose LOCI, a mobile question answer system with a multimodal motivation scheme for local intent questions where users provide timely and accurate answers in a dynamic social network. LOCI mines users' shared and contextual information to identify the most relevant users for each question. The proposed multimodal motivation scheme exploits users' social ties and monetary rewards to raise their basic motivation levels. LOCI assigns questions to the best answerers via our three proposed algorithms: (i) LOCI-RT, which immediately assigns newly arrived questions/users to available users/questions, (ii) LOCI-LO, which uses short-time batches for locally optimal questions-users assignment, and (iii) LOCI-AD, which assigns short- and long-deadline questions immediately and via batching, respectively. We conduct: (i) a survey to support our assumptions regarding the proposed multimodal motivation scheme, and (ii) trace-driven experiments to evaluate the performance of the three proposed LOCI's algorithms. © 2020 IEEE.

关键词： Motivation

来源：评论

学校读者我要写书评

暂无评论

On Efficient Processing of Queries for Live Multi-Streaming Soiree Organization

引用

IEEE Transactions on Services computing 2023年第4期16卷 2812-2826页

作者： Kankeu Fotsing, C.P. Shen, Chih-Ya Huang, Liang-Hao Chen, Yi-Shin Lee, Wang-Chien Yang, De-Nian International Graduate Program Academia Sinica Social Networks and Human-Centered Computing Program Taipei115 Taiwan National Tsing Hua University Institute of Information Systems and Applications Hsinchu300 Taiwan National Tsing Hua University Department of Computer Science Hsinchu300 Taiwan Academia Sinica Institute of Information Science Taipei115 Taiwan Pennsylvania State University Department of Computer Science and Engineering State CollegePA16801 United States

Real-time social interactions and multi-streaming are two critical features of live streaming services. In this paper, we formulate a new fundamental service query, social-aware Diverse and Preferred Organization Query (SDSQ), that jointly selects a set of diverse and preferred live streaming channels and a group of socially tight viewers for organization of a live multi-streaming soiree. We prove that SDSQ is NP-hard and inapproximable within any factor, and design SDSSel, a 2-approximation algorithm with a guaranteed error bound. Moreover, we study SDSQ-T, a special case of SDSQ, where the social graph is a threshold graph, and propose TDSSel, a 2-approximation algorithm without any error to SDSQ-T. We propose two pruning strategies, PCP and CDP to boost SDSSel and TDSSel. We further propose a more challenging but practical service query, Generalized social-aware Maximum Preferred and Diverse Query (GSPQ), a generalization of SDSQ. We design GPDSel, a 4-approximation algorithm for GSPQ with a guaranteed error bound. We propose a strategy to improve the approximation ratios of the proposed algorithms. A user study on Twitch validates SDSQ, and the large-scale experiments on real datasets demonstrate the superiority of the proposed algorithms over several baselines for live-streaming services. © 2008-2012 IEEE.

关键词： Approximation algorithms

来源：评论

学校读者我要写书评

暂无评论

TERMS: textual emotion recognition in multidimensional space

引用

Applied Intelligence 2023年第3期53卷 2673-2693页

作者： Ghafoor, Yusra Jinping, Shi Calderon, Fernando H. Huang, Yen-Hao Chen, Kuan-Ta Chen, Yi-Shin Social Networks and Human-Centered Computing Taiwan International Graduate Program Institute of Information Science Academia Sinica. Institute of Information Systems and Applications National Tsing Hua University Hsinchu City Taiwan Institute of Information Systems and Applications National Tsing Hua University Hsinchu City Taiwan Institute of Information Science Academia Sinica Taipei Taiwan Department of Computer Science National Tsing Hua University Hsinchu City Taiwan

Microblogs generate a vast amount of data in which users express their emotions regarding almost all aspects of everyday life. Capturing affective content from these context-dependent and subjective texts is a challenging task. We propose an intelligent probabilistic model for textual emotion recognition in multidimensional space (TERMS) that captures the subjective emotional boundaries and contextual information embedded in a text for robust emotion recognition. It is implausible with discrete label assignment;therefore, the model employs a soft assignment by mapping varying emotional perceptions in a multidimensional space and generates them as distributions via the Gaussian mixture model (GMM). To strengthen emotion distributions, TERMS integrates a probabilistic emotion classifier that captures the contextual and linguistic information from texts. The integration of these aspects, the context-aware emotion classifier and the learned GMM parameters provide a complete coverage for accurate emotion recognition. The large-scale experimentation shows that compared to baseline and state-of-the-art models, TERMS achieved better performance in terms of distinguishability, prediction, and classification performance. In addition, TERMS provide insights on emotion classes, the annotation patterns, and the models application in different scenarios. © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

关键词： Emotion Recognition

来源：评论

学校读者我要写书评

暂无评论

Text and Speech Adjustable Bimodal Emotion Recognition in Conversational Interface

Text and Speech Adjustable Bimodal Emotion Recognition in Co...

引用

IEEE international Symposium on Consumer Electronics (ISCE)

作者： Komang Rinartha Lun-Wei Ku Social Networks and Human Centered Computing Program Taiwan International Graduate Program Institute of Information Science Nangang Taipei Taiwan Institute of Information Science Academia Sinica Taipei Taiwan

ISBN: (数字)9798350365191

ISBN: (纸本)9798350365207

关键词： Emotion recognition Accuracy Machine learning algorithms Text recognition Computational modeling Heuristic algorithms Speech recognition Machine learning Real-time systems Monitoring

来源：评论

学校读者我要写书评

暂无评论

Massive Machine Type Communication in Cellular System: A Distributed Queue Approach

Massive Machine Type Communication in Cellular System: A Dis...

引用

Ad-Hoc and Sensor networks Symposium

作者： Fredrick Mzee Awuor Chih-Yu Wang Taiwan International Graduate Program Social Networks and Human-Centered Computing Program Institute of Information Science Research Center for Information Technology Innovation Academia Sinica

ISBN: (纸本)9781479966653

Massive machine type communication (MTC) brings new challenges to next generation wide-area networks. Random Access Channel procedure (RACH) used in LTE-Advanced system for request delivery, for instance, is inefficient in handling massive number of concurrent access requests. The S-Aloha approach in RACH will lead to unsustainable delays and resource wastage due to collisions from concurrent access in massive MTC networks. To address this issue, we propose a distributed queue based MTC communication protocol that exploits multiple packet reception (MPR) capability of MTC devices to self-organize themselves into a logical queue in a distributed way such that the devices connect to BS depending on their positions in the queue. We derive the algorithm and present analysis of its performance in regard to delay and throughput. The algorithm behaves like S-Aloha when MTC traffic load is low and seamlessly and smoothly switches to a reservation based protocol during heavy traffic. From the simulation results, the proposed scheme exhibits better performance compared to conventional RACH procedure as it increases number of successful transmissions by 2.7 times and reduces both the average access delay and transmission failure probability by 20%.

关键词： Concurrent Access Queue Minesite Technical Committees cellular system S-ALOHA

来源：评论

学校读者我要写书评

暂无评论

Learning Adaptation and Generalization from human-Inspired Meta-Reinforcement Learning Using Bayesian Knowledge and Analysis 6

Learning Adaptation and Generalization from Human-Inspired M...

引用

6th IEEE international Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2023

作者： Ho, Joshua Wang, Chien-Min King, Chung-Ta You, Yi-Hsin Feng, Chi-Wei Chen, Yen-Min Kuo, Bo-Yi Institute of Information Science Academia Sinica Taipei11529 Taiwan Social Networks and Human-Centered Computing Program Taiwan International Graduate Program Taiwan Institute of Information Systems and Applications National Tsing Hua University Hsinchu30013 Taiwan Department of Computer Science National Tsing Hua University Hsinchu30013 Taiwan Department of Computer Science National Taiwan University Taipei10631 Taiwan Center of Intelligent Healthcare National Taiwan University Hospital Taipei10022 Taiwan

ISBN: (纸本)9798350331288

Over the last decades, there has been growing interest in research in multiple and interdisciplinary fields of human-AI computing. In particular, approaches integrating the intersecting design with reinforcement learning (RL) have received more attention. However, the current research on RL may need to consider its enhancement from a humaninspired approach further. In the present work, we focus on enabling a meta-reinforcement learning (meta-RL) agent to achieve adaptation and generalization according to modeling Markov decision processes using Bayesian knowledge and analysis. By introducing a novel framework called human-inspired meta-RL (HMRL), we incorporate the agent performing resilient actions to leverage the dynamic dense reward based on the knowledge and prediction of a Bayesian analysis. The proposed framework can make the agent learn generalization and prevent the agent from failing catastrophically. The experimental results show that our approach helps the agent reduce computational costs with learning adaptation. Finally, we conclude and anticipate that integrating human-inspired meta-RL can enable learning more formulations relating to robustness and scalability, leading to promising directions and more complex AI goals in the future. © 2023 IEEE.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：