This study aimed to identify electroencephalography (EEG)-based neurophysiological biomarkers for monitoring mental fatigue during motor imagery (MI) tasks. We used EEG data recorded from 29 healthy adults during MI t...
详细信息
Human–computer interaction (HCI) focuses on designing efficient and intuitive interactions between humans and computer systems. Recent advancements have utilized multimodal approaches, such as electroencephalography ...
详细信息
Human–computer interaction (HCI) focuses on designing efficient and intuitive interactions between humans and computer systems. Recent advancements have utilized multimodal approaches, such as electroencephalography (EEG)-based systems combined with other biosignals, along with deep learning to enhance performance and reliability. However, no systematic review has consolidated findings on EEG-based multimodal HCI systems. This review examined 124 studies published from 2016 to 2024, retrieved from the Web of Science database, focusing on hybrid EEG-based multimodal HCI systems employing deep learning. The keywords used for evaluation were as follows: ‘Deep Learning’ AND ‘EEG’ AND (‘fNIRS’ OR ‘NIRS’ OR ‘MEG’ OR ‘fMRI’ OR ‘EOG’ OR ‘EMG’ OR ‘ECG’ OR ‘PPG’ OR ‘GSR’). Main topics explored are: (1) types of biosignals used with EEG, (2) neural network architectures, (3) fusion strategies, (4) system performance, and (5) target applications. Frequently paired signals, such as EOG, EMG, and fNIRS, effectively complement EEG by addressing its limitations. Convolutional neural networks are extensively employed for spatio-temporal-spectral feature extraction, with early and intermediate fusion strategies being the most commonly used. Applications, such as sleep stage classification, emotion recognition, and mental state decoding, have shown notable performance improvements. Despite these advancements, challenges remain, including the lack of real-time online systems, difficulties in signal synchronization, limited data availability, and insufficient explainable AI (XAI) methods to interpret signal interactions. Emerging solutions, such as portable systems, lightweight deep learning models, and data augmentation techniques, offer promising pathways to address these issues. This review highlights the potential of EEG-based multimodal HCI systems and emphasizes the need for advancements in real-time interaction, fusion algorithms, and XAI to enhance their adaptability, interpreta
This study aimed to identify electroencephalography (EEG)-based neurophysiological biomarkers for monitoring mental fatigue during motor imagery (MI) tasks. We used EEG data recorded from 29 healthy adults during MI t...
详细信息
ISBN:
(数字)9798331521929
ISBN:
(纸本)9798331521936
This study aimed to identify electroencephalography (EEG)-based neurophysiological biomarkers for monitoring mental fatigue during motor imagery (MI) tasks. We used EEG data recorded from 29 healthy adults during MI tasks. For the preprocessing of the EEG data, we sequentially performed down-sampling into 200 Hz, independent component analysis (ICA), band-pass filtering between 0.5 and 50 Hz, and common average reference (CAR). The preprocessed EEG data were analyzed using a fast Fourier transform (FFT) to calculate spectral power in the δ (0.5-3 Hz), θ (4–7 Hz), $a$ (8–13 Hz), and β (13–30 Hz) frequency bands. Subsequently, seventeen fatigue-related metrics were calculated using individual spectral powers and their combinations. These fatigue-related metrics were averaged for the frontal lobe (F3 and F4), central lobe (FCC3h and FCC4h), parietal lobe (P3 and P4), and occipital lobe (POO1 and POO2). The statistical significance of the average fatigue metrics was evaluated through Pearson correlation analysis with the sequence of trials. As a result, we observed strong positive correlations (r < 0.9, p < 0.001) between beta power in the frontal and central regions and the sequence of trials. These findings suggest that central and frontal beta activity can serve as reliable biomarkers for estimating fatigue during MI tasks, supporting their potential for real-time fatigue monitoring.
Understanding the morphology of amyloid fibrils is crucial for comprehending the aggregation and degradation mechanisms of abnormal proteins implicated in various diseases, such as Alzheimer's disease, Parkinson&#...
详细信息
The goal of financial QA is to generate solution equations by solving problems about financial reports. Current financial QA models can suffer from two issues: expression fragmentation and number redundancy. We conduc...
详细信息
ISBN:
(数字)9798331529024
ISBN:
(纸本)9798331529031
The goal of financial QA is to generate solution equations by solving problems about financial reports. Current financial QA models can suffer from two issues: expression fragmentation and number redundancy. We conduct experiments to examine the impact of the two issues. The experimental results show that addressing the expression fragmentation issue in financial QA using EPT improves the execution accuracy by 0.09, and alleviating the number redundancy issue by removing redundant numbers in the input of the FinQANet generator improves the execution accuracy by 0.07.
Multimodal Emotion Recognition in Conversation (ERC) is a task of predicting the emotion of each utterance in a conversation by utilizing both verbal and non-verbal modalities. However, existing approaches often strug...
详细信息
ISBN:
(数字)9798331529024
ISBN:
(纸本)9798331529031
Multimodal Emotion Recognition in Conversation (ERC) is a task of predicting the emotion of each utterance in a conversation by utilizing both verbal and non-verbal modalities. However, existing approaches often struggle to bridge cross-modal gaps, resulting in misaligned features and frequent misclassification of minority emotions into semantically similar majority emotions. To address these challenges, we propose MERNet, a framework that employs cross-modal knowledge distillation and contrastive learning to align multimodal features and effectively distinguish subtle emotions in conversations. Our framework consists of two stages: 1) guiding non-verbal modalities with the text modality to transfer knowledge and align their features, and 2) applying contrastive learning with emotion labels as anchors to distinguish subtle differences between similar emotions and address the class imbalance problem. Experiments conducted on two benchmark datasets, IEMOCAP and MELD, demonstrate that our MERNet outperforms existing state-of-the-art models.
The COVID-19 outbreak has highlighted the importance of mathematical epidemic models like the Susceptible-Infected-Recovered (SIR) model, for understanding disease spread dynamics. However, enhancing their predictive ...
详细信息
TLR2 is a key component of the innate immune system, responsible for recognizing Gram-positive bacterial components and initiating inflammatory signaling cascades that activate defense responses. However, little is kn...
详细信息
The development and use of a cost-effective, eco-friendly catalyst for the efficient removal of Cr (VI) from wastewater are critical. In response, we have created a Pani/MoS2 composite for adsorption, utilizing a bina...
详细信息
暂无评论