检索结果-内蒙古大学图书馆

MOCAS: A multimodal dataset for Objective Cognitive Workload Assessment on Simultaneous Tasks

IEEE TRANSACTIONS ON AFFECTIVE COMPUTING 2025年第1期16卷 116-132页

作者： Jo, Wonse Wang, Ruiqi Cha, Go-Eum Sun, Su Senthilkumaran, Revanth Krishna Foti, Daniel Min, Byung-Cheol Purdue Univ Dept Comp & Informat Technol SMART Lab W Lafayette IN 47907 USA Purdue Univ Dept Psychol Sci W Lafayette IN 47907 USA

This paper presents MOCAS, a multimodal dataset dedicated for human cognitive workload (CWL) assessment. In contrast to existing datasets based on virtual game stimuli, the data in MOCAS was collected from realistic closed-circuit television (CCTV) monitoring tasks, increasing its applicability for real-world scenarios. To build MOCAS, two off-the-shelf wearable sensors and one webcam were utilized to collect physiological signals and behavioral features from 21 human subjects. After each task, participants reported their CWL by completing the NASA-Task Load Index (NASA-TLX) and Instantaneous Self-Assessment (ISA). Personal background (e.g., personality and prior experience) was surveyed using demographic and Big Five Factor personality questionnaires, and two domains of subjective emotion information (i.e., arousal and valence) were obtained from the Self-Assessment Manikin (SAM), which could serve as potential indicators for improving CWL recognition performance. Technical validation was conducted to demonstrate that target CWL levels were elicited during simultaneous CCTV monitoring tasks;its results support the high quality of the collected multimodal signals.

关键词： Task analysis Monitoring Biomedical monitoring Behavioral sciences Sensors Affective computing Human-machine systems cognitive workload assessment human-machine systems human-robot teams multimodal dataset

来源：评论

学校读者我要写书评

暂无评论

Interactions for Socially Shared Regulation in Collaborative Learning: An Interdisciplinary multimodal dataset

引用

ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS 2024年第3期14卷 1-34页

作者： Li, Yante Liu, Yang Nguyen, Andy Shi, Henglin Vuorenmaa, Eija Jarvela, Sanna Zhao, Guoying Univ Oulu Oulu Finland

Socially shared regulation plays a pivotal role in the success of collaborative learning. However, evaluating socially shared regulation of learning (SSRL) proves challenging due to the dynamic and infrequent cognitive and socio-emotional interactions, which constitute the focal point of SSRL. To address this challenge, this article gathers interdisciplinary researchers to establish a multimodal dataset with cognitive and socioemotional interactions for SSRL study. Firstly, to induce cognitive and socio-emotional interactions, learning science researchers designed a special collaborative learning task with regulatory trigger events among triadic people for the SSRL study. Secondly, this dataset includes various modalities like video, Kinect data, audio, and physiological data (accelerometer, EDA, heart rate) from 81 high school students in 28 groups, offering a comprehensive view of the SSRL process. Thirdly, three-level verbal interaction annotations and nonverbal interactions including facial expression, eye gaze, gesture, and posture are provided, which could further contribute to interdisciplinary fields such as computer science, sociology, and education. In addition, comprehensive analysis verifies the dataset's effectiveness. As far as we know, this is the first multimodal dataset for studying SSRL among triadic group members.

关键词： multimodal dataset human interaction facial expression gesture physio- logical signal collaborative learning

来源：评论

学校读者我要写书评

暂无评论

Structure and design of multimodal dataset for automatic regex synthesis methods in Roman Urdu

引用

INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS 2024年 1-15页

作者： Tariq, Sadia Rana, Toqir A. Univ Lahore Dept CS & IT Lahore Pakistan

Automatic regex synthesis involves generation of regular expressions from user-written natural language descriptions, example strings or both. Daily, countless regex generation queries are posted on online Q&A platforms such as StackOverflow (https://***) and Quora (https://***). Existing automatic regex synthesis methods demand concretely designed, multimodal datasets for optimal performance. Unfortunately, publicly available datasets even for resource-rich languages like English are often model-specific and incomplete, potentially hindering the efficiency and accurateness of regex synthesis methods. This issue is worsened for resource-poor languages such as Standard Urdu and Roman Urdu. In this paper, we present a novel, benchmark Roman Urdu dataset with 900 words and a novel Roman Urdu lexicon of 4225 words, annotated and labeled, to address the unmet needs of regex synthesis methods. Equipping these methods with a proficient dataset can lead to more fruitful regex generation.

关键词： multimodal dataset Auto regex synthesis dataset for Roman Urdu auto regex synthesis Roman Urdu dataset Query to regex Strings to regex

来源：评论

学校读者我要写书评

暂无评论

Collecting Mementos: A multimodal dataset for Context-Sensitive Modeling of Affect and Memory Processing in Responses to Videos

引用

IEEE TRANSACTIONS ON AFFECTIVE COMPUTING 2023年第2期14卷 1249-1266页

作者： Dudzik, Bernd Hung, Hayley Neerincx, Mark Broekens, Joost Delft Univ Technol Dept Intelligent Syst INSY NL-2628 XE Delft Netherlands Leiden Univ Leiden Inst Adv Comp Sci LIACS NL-2333 CA Leiden Netherlands

In this article we introduce Mementos: the first multimodal corpus for computational modeling of affect and memory processing in response to video content. It was collected online via crowdsourcing and captures 1995 individual responses collected from 297 unique viewers responding to 42 different segments of music videos. Apart from webcam recordings of their upper-body behavior (totaling 2012 minutes) and self-reports of their emotional experience, it contains detailed descriptions of the occurrence and content of 989 personal memories triggered by the video content. Finally, the dataset includes self-report measures related to individual differences in participants' background and situation (Demographics, Personality, and Mood), thereby facilitating the exploration of important contextual factors in research using the dataset. We describe 1) the construction and contents of the corpus itself, 2) analyse the validity of its content by investigating biases and consistency with existing research on affect and memory processing, 3) review previously published work that demonstrates the usefulness of the multimodal data in the corpus for research on automated detection and prediction tasks, and 4) provide suggestions for how the dataset can be used in future research on modeling Video-Induced Emotions, Memory-Associated Affect, and Memory Evocation.

关键词： Videos Media Computational modeling Films Particle measurements Mood Atmospheric measurements multimodal dataset personal memory video-induced emotion memory evocation memory-associated affect affect detection video affective content analysis context-sensitivity personalization

来源：评论

学校读者我要写书评

暂无评论

HEADSET: Human Emotion Awareness under Partial Occlusions multimodal dataset

引用

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023年第11期29卷 4686-4696页

作者： Lohesara, Fatemeh Ghorbani Freitas, Davi Rabbouni Guillemot, Christine Eguiazarian, Karen Knorr, Sebastian Tech Univ Berlin Commun Syst Grp Berlin Germany INRIA Paris France Tampere Univ Computat Imaging Grp Tampere Finland Ernst Abbe Univ Appl Sci Jena Jena Germany

The volumetric representation of human interactions is one of the fundamental domains in the development of immersive media productions and telecommunication applications. Particularly in the context of the rapid advancement of Extended Reality (XR) applications, this volumetric data has proven to be an essential technology for future XR elaboration. In this work, we present a new multimodal database to help advance the development of immersive technologies. Our proposed database provides ethically compliant and diverse volumetric data, in particular 27 participants displaying posed facial expressions and subtle body movements while speaking, plus 11 participants wearing head-mounted displays (HMDs). The recording system consists of a volumetric capture (VoCap) studio, including 31 synchronized modules with 62 RGB cameras and 31 depth cameras. In addition to textured meshes, point clouds, and multi-view RGB-D data, we use one Lytro Illum camera for providing light field (LF) data simultaneously. Finally, we also provide an evaluation of our dataset employment with regard to the tasks of facial expression classification, HMDs removal, and point cloud reconstruction. The dataset can be helpful in the evaluation and performance testing of various XR algorithms, including but not limited to facial expression recognition and reconstruction, facial reenactment, and volumetric video. HEADSET and its all associated raw data and license agreement will be publicly available for research purposes.

关键词： Extended reality multimodal dataset virtual reality volumetric video light field

来源：评论

学校读者我要写书评

暂无评论

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Wu, Xinda Wang, Jiaming Yu, Jiaxing Zhang, Tieyao Zhang, Kejun Zhejiang Univ Coll Comp Sci & Technol Hangzhou Peoples R China

ISBN: (纸本)9798350379822;9798350379815

The Internet is rich in unimodal music data, available in either symbolic or audio representations. However, there is a notable scarcity of multimodal music datasets that offer aligned modal information and comprehensive annotations for music understanding and generation. In this paper, we introduce Popular Hooks: a publicly accessible multimodal music dataset comprising 38,694 popular musical hooks (i.e., memorable sections of songs) with synchronized MIDI, music video, audio, and lyrics. Furthermore, the dataset provides detailed annotations of high-level musical attributes such as tonality, structure, genre, emotion, and region. Specifically, we leverage a pre-trained multimodal music emotion recognition framework for automatic emotion labeling and conduct a user study to assess its accuracy. Finally, we explore emotion-conditioned music generation baselines using this dataset, demonstrating its potential to advance the field.

关键词： multimodal dataset multimodal emotion recognition music generation music understanding

来源：评论

学校读者我要写书评

暂无评论

HOLOGESTURE: A multimodal dataset FOR HAND GESTURE RECOGNITION ROBUST TO HAND TEXTURES ON HEAD-MOUNTED MIXED-REALITY DEVICES 31

HOLOGESTURE: A MULTIMODAL DATASET FOR HAND GESTURE RECOGNITI...

引用

2024 International Conference on Image Processing

作者： Park, Jeongwoo Hong, Je Hyeong Hanyang Univ Dept Elect Engn Seoul South Korea

ISBN: (纸本)9798350349405;9798350349399

While the recent development of high performance mixed-reality (MR) devices is enabling its use in medical and industrial domains, this requires hand gesture recognition to be robust to different textures inflicted by gloves often worn for hygiene and safety purposes. Unfortunately, most existing hand gesture datasets are not captured using recent commercial MR devices, and none addresses the issue of wearing gloves in gesture recognition. We aim to fill these gaps by introducing a new dataset called HoloGesture, which comprises gesture clips acquired with and without latex gloves using Microsoft HoloLens 2. To leverage the multimodal nature of the latest MR device, we go beyond simply stacking RGB and depth frames and provide spatially aligned depth and RGB images. Experimental results show that i) incorporating gloves for training enhances robustness of gesture recognition to different hand textures and ii) spatial alignment of RGB and depth images enhances the recognition accuracy. Our code and dataset can be found at https://***/hellojpark/hologesture.

关键词： hand gesture recognition mixed-reality multimodal dataset gloves depth

来源：评论

学校读者我要写书评

暂无评论

A multimodal dataset for Robot Learning to Imitate Social Human-Human Interaction 23

A Multimodal Dataset for Robot Learning to Imitate Social Hu...

引用

18th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI)

作者： Tuyen, Nguyen Tan Viet Georgescu, Alexandra L. Di Giulio, Irene Celiktutan, Oya Kings Coll London Dept Engn London England Kings Coll London Dept Psychol London England Kings Coll London Ctr Human & Appl Physiol Sci London England

ISBN: (纸本)9781450399708

Humans tend to use various nonverbal signals to communicate their messages to their interaction partners. Previous studies utilised this channel as an essential clue to develop automatic approaches for understanding, modelling and synthesizing individual behaviours in human-human interaction and human-robot interaction settings. On the other hand, in small-group interactions, an essential aspect of communication is the dynamic exchange of social signals among interlocutors. This paper introduces LISI-HHI - Learning to Imitate Social Human-Human Interaction, a dataset of dyadic human interactions recorded in a wide range of communication scenarios. The dataset contains multiple modalities simultaneously captured by high-accuracy sensors, including motion capture, RGB-D cameras, eye trackers, and microphones. LISI-HHI is designed to be a benchmark for HRI and multimodal learning research for modelling intra- and interpersonal nonverbal signals in social interaction contexts and investigating how to transfer such models to social robots.

关键词： human-human interaction multimodal dataset nonverbal behaviour analysis and synthesis

来源：评论

学校读者我要写书评

暂无评论

Unlocking Human-Robot Dynamics: Introducing SenseCobot, a Novel multimodal dataset on Industry 4.0 24

Unlocking Human-Robot Dynamics: Introducing SenseCobot, a No...

引用

19th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI)

作者： Borghi, Simone Zucchi, Federica Prati, Elisa Ruo, Andrea Villani, Valeria Sabattini, Lorenzo Peruzzini, Margherita Univ Modena & Reggio Emilia Dept Engn Enzo Ferrari Modena Italy Univ Modena & Reggio Emilia Dept Sci & Methods Engn Reggio Emilia Italy Univ Bologna Dept Ind Engn Bologna Italy

ISBN: (数字)9798400703225

ISBN: (纸本)9798400703225

In the era of Industry 4.0, the importance of human-robot collaboration (HRC) in the advancement of modern manufacturing and automation is paramount. Understanding the intricate physiological responses of the operator when they interact with a cobot is essential, especially during programming tasks. To this aim, wearable sensors have become vital for real-time monitoring of worker well-being, stress, and cognitive load. This article presents an innovative dataset (SenseCobot) of physiological signals recorded during several collaborative robotics programming tasks. This dataset includes various measures like ElectroCardioGram (ECG), Galvanic Skin Response (GSR), ElectroDermal Activity (EDA), body temperature, accelerometer, ElectroEncephaloGram (EEG), Blood Volume Pulse (BVP), emotions and subjective responses from NASA-TLX questionnaires for a total of 21 participants. By sharing dataset details, collection methods, and task designs, this article aims to drive research in HRC advancing understanding of the User eXperience (UX) and fostering efficient, intuitive robotic systems. This could promote safer and more productive HRC amid technological shifts and help decipher intricate physiological signals in different scenarios.

关键词： Human-Robot Collaboration Human-Robot Interaction multimodal dataset Psychophysiological Signals Stress Evaluation

来源：评论

学校读者我要写书评

暂无评论

MMASD: A multimodal dataset for Autism Intervention Analysis 23

MMASD: A Multimodal Dataset for Autism Intervention Analysis

引用

25th International Conference on multimodal Interaction (ICMI)

作者： Li, Jicheng Chheang, Vuthea Kullu, Pinar Brignac, Eli Guo, Zhang Barner, Kenneth E. Bhat, Anjana Barmaki, Roghayeh Leila Univ Delaware Newark DE 19716 USA

ISBN: (纸本)9798400700552

Autism spectrum disorder (ASD) is a developmental disorder characterized by significant impairments in social communication and difficulties perceiving and presenting communication signals. Machine learning techniques have been widely used to facilitate autism studies and assessments. However, computational models are primarily concentrated on very specific analysis and validated on private, non-public datasets in the autism community, which limits comparisons across models due to privacy-preserving data-sharing complications. This work presents a novel open source privacy-preserving dataset, MMASD as a multimodal ASD benchmark dataset, collected from play therapy interventions for children with autism. The MMASD includes data from 32 children with ASD, and 1,315 data samples segmented from more than 100 hours of intervention recordings. To promote the privacy of children while offering public access, each sample consists of four privacy-preserving modalities, some of which are derived from original videos: (1) optical flow, (2) 2D skeleton, (3) 3D skeleton, and (4) clinician ASD evaluation scores of children. MMASD aims to assist researchers and therapists in understanding children's cognitive status, monitoring their progress during therapy, and customizing the treatment plan accordingly. It also inspires downstream social tasks such as action quality assessment and interpersonal synchrony estimation. The dataset is publicly accessible via the MMASD project website.

关键词： multimodal dataset machine learning deep learning autism spectrum disorder human activity recognition 2D/ 3D skeleton privacy-preserving data sharing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：