检索结果-内蒙古大学图书馆

AV-CrossNet: An Audiovisual Complex Spectral Mapping Network for Speech Separation by Leveraging Narrow- and Cross-Band Modeling

引用

IEEE Journal on Selected Topics in Signal Processing 2025年

作者： Kalkhorani, Vahid Ahmadi Yu, Cheng Kumar, Anurag Tan, Ke Xu, Buye Wang, DeLiang Ohio State University Department of Computer Science and Engineering ColumbusOH43210 United States Meta Reality Labs RedmondWA20004 United States Ohio State University Department of Computer Science and Engineering Center for Cognitive and Brain Sciences ColumbusOH43210 United States

Adding visual cues to audio-based speech separation can improve separation performance. This paper introduces AV-CrossNet, an audiovisual (AV) system for speech enhancement, target speaker extraction, and multi-talker speaker separation. AV-CrossNet is extended from the TF-CrossNet architecture, which is a recently proposed network that performs complex spectral mapping for speech separation by leveraging global attention and positional encoding. To effectively utilize visual cues, the proposed system incorporates pre-extracted visual embeddings and employs a visual encoder comprising temporal convolutional layers. Audio and visual features are fused in an early fusion layer before feeding to AV-CrossNet blocks. We evaluate AV-CrossNet on multiple datasets, including LRS, VoxCeleb, TCD-TIMIT, and COG-MHEAR challenge, in terms of the performance metrics of PESQ, STOI, SNR and SDR. Evaluation results demonstrate that AV-CrossNet advances the state-of-the-art performance in all audiovisual tasks, even on untrained and mismatched datasets. © 2007-2012 IEEE.

关键词： Speech enhancement

来源：评论

学校读者我要写书评

暂无评论

CrossNet: Leveraging Global, Cross-Band, Narrow-Band, and Positional Encoding for Single- and Multi-Channel Speaker Separation

arXiv

引用

arXiv 2024年

作者： Kalkhorani, Vahid Ahmadi Wang, DeLiang Department of Computer Science and Engineering The Ohio State University ColumbusOH43210 United States Department of Computer Science and Engineering The Center for Cognitive and Brain Sciences The Ohio State University ColumbusOH43210 United States

We introduce CrossNet, a complex spectral mapping approach to speaker separation and enhancement in reverberant and noisy conditions. The proposed architecture comprises an encoder layer, a global multi-head self-attention module, a cross-band module, a narrow-band module, and an output layer. CrossNet captures global, cross-band, and narrow-band correlations in the time-frequency domain. To address performance degradation in long utterances, we introduce a random chunk positional encoding. Experimental results on multiple datasets demonstrate the effectiveness and robustness of CrossNet, achieving state-of-the-art performance in tasks including reverberant and noisy-reverberant speaker separation. Furthermore, CrossNet exhibits faster and more stable training in comparison to recent baselines. Additionally, CrossNet’s high performance extends to multi-microphone conditions, demonstrating its versatility in various acoustic scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Frequency domain analysis

来源：评论

学校读者我要写书评

暂无评论

Prediction of autism spectrum disorder using epigenetic, brain, and sensory behavioral factors

Prediction of autism spectrum disorder using epigenetic, bra...

引用

International Winter Workshop on brain-computer Interface (BCI)

作者： Yong Jeon Cheong Jihyun Bae Seonkyoung Lee Ro Ji Hyeong Hirotaka Kosaka Ilwoo Lyu Minyoung Jung Cognitive Science Research Group Korea Brain Research Institute Daegu Republic of Korea Department of Neuropsychiatry University of Fukui Fukui Japan Computer Science and Engineering Ulsan National Institute of Science and Technology Ulsan Republic of Korea

Autism spectrum disorder (ASD) is characterized by clinical and etiological heterogeneity. Although there is broad agreement on the necessity of integrating reliable biomarkers of ASD, little is known about how epigenetic and/or brain factors contribute to ASD. To probe epigenetic factors, we obtained the participants’ saliva samples to obtain DNA methylation values of the oxytocin receptor and arginine vasopressin genes. Given the atypical sensory processing associated with ASD and the role of thalamus as the sensory relay station, we acquired neuroimaging data to create a thalamocortical resting state functional connectivity map of ASD. Integrating features derived from epigenetic, brain, and sensory behavior factors, the current study aims to develop a model that predicts ASD using an eXtreme Gradient Boosting algorithm with ensemble feature selection procedure based on feature occurrence frequency and automatically optimized hyperparameters. We found the full model with all three factors (83.95%) to be superior for ASD classification, compared to models with epigenetic and behavioral factors (76.89%) or with brain and behavioral factors (81.19%).

关键词：

来源：评论

学校读者我要写书评

暂无评论

Robust Frame-level Speaker Localization in Reverberant and Noisy Environments by Exploiting Phase Difference Losses

Robust Frame-level Speaker Localization in Reverberant and N...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Shanmukha Srinivas Battula Hassan Taherian Ashutosh Pandey Daniel Wong Buye Xu DeLiang Wang Department of Computer Science and Engineering The Ohio State University USA Meta Reality Labs USA Center for Cognitive and Brain Sciences The Ohio State University USA

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

This paper investigates robust speaker localization at the frame level on the basis of complex spectral mapping, which is capable of learning both the magnitude and phase of the target signal. Unlike prevailing deep learning methods for speaker localization, we perform MIMO (multi-input multi-output) based multi-channel speech enhancement first and then localize the enhanced speaker using weighted generalized cross correlation. In addition, we propose new multi-channel loss functions that incorporate phase differences in order to preserve inter-channel phase relations, which is key to accurate sound localization. Systematic evaluations using simulated and recorded room impulse responses demonstrate that the proposed model yields excellent frame-level speaker localization results in reverberant and noisy environments and outperforms related methods by a large margin, even surpassing their utterance-level results.

关键词： Location awareness Deep learning Accuracy Systematics Correlation Speech enhancement Signal processing Acoustics Noise measurement

来源：评论

学校读者我要写书评

暂无评论

EEG Spectral Correlates of Rapid and Deep Slow Breathing States and classification using ML

EEG Spectral Correlates of Rapid and Deep Slow Breathing Sta...

引用

2022 IEEE Region 10 Symposium, TENSYMP 2022

作者： Patnaik, Siddesh Pandey, Pankaj Arun, Ishita Yadav, Goldy Miyapuram, Krishna Prasad Lomas, Derek Electrical Engineering IIT Gandhinagar India Computer Science and Engineering IIT Gandhinagar India Centre for Cognitive and Brain Sciences IIT Gandhinagar India Delft University of Technology Vibe Research Lab Netherlands

ISBN: (数字)9781665466585

ISBN: (纸本)9781665466585

One interpretation of breathing exercise is to enforce mind-body harmony, when someone feels well and healthy, different organs of our body function harmoniously. One dysfunctional organ may disturb the resonating mechanism across multiple organs. There are different breathing techniques, and recent scientific evidence encourages understanding the neural correlates of breathing. This research investigates breathing exercises at two paces: Rapid and Deep Slow using neural signals. We collect Electroencephalography (EEG) recordings of 14 participants performing breathing tasks. EEG signals are primarily decomposed in frequency bands that designate different cognitive functions. We extract six primary frequency bands, including delta (1-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), low beta (13-20 Hz), high beta (21-30 Hz), and gamma (30-40 Hz). Two different techniques are utilized to report the findings encompassing power spectral analysis and employing machine learning classifiers to discriminate features among different stages of inhalation and exhalation with the significance of different frequencies bands. Lowered beta power in Slow Deep breathing is observed compared to Rapid Breathing, which may suggest increased relaxation, calmness, and anxiety reduction. Differences between the two conditions observed in the frontoparietal cortex may be attributed to differences in voluntary control between the two tasks. We observed classification accuracy of 72 % using low beta between Rapid and Deep Slow breathing using Decision Tree. Several interesting findings are observed in different scalp regions suggesting future direction for further investigation. This study contributes to the understanding of neural signatures for different breathing practices. The implication of this research in health care is to design personalized therapies and to design better breathing mobile applications for daily use. © 2022 IEEE.

关键词： Electroencephalography

来源：评论

学校读者我要写书评

暂无评论

Explainable artificial intelligence approaches for brain-computer interfaces: a review and design space

arXiv

引用

arXiv 2023年

作者： Rajpura, Param Cecotti, Hubert Meena, Yogesh Kumar Department of Cognitive and Brain Sciences Indian Institute of Technology Gandhinagar India Department of Computer Science California State University FresnoCA United States Department of Computer Science & Engineering Department of Cognitive and Brain Sciences Indian Institute of Technology Gujarat Gandhinagar India

Objective. This review paper provides an integrated perspective of Explainable Artificial Intelligence (XAI) techniques applied to brain-computer Interfaces (BCIs). BCIs use predictive models to interpret brain signals for various high-stake applications. However, achieving explainability in these complex models is challenging as it compromises accuracy. Trust in these models can be established by incorporating reasoning or causal relationships from domain experts. The field of XAI has emerged to address the need for explainability across various stakeholders, but there is a lack of an integrated perspective in XAI for BCI (XAI4BCI) literature. It is necessary to differentiate key concepts like explainability, interpretability, and understanding, often used interchangeably in this context, and formulate a comprehensive framework. Approach. To understand the need of XAI for BCI, we pose six key research questions (RQs) for a systematic review and meta-analysis, encompassing its purposes, applications, usability, and technical feasibility. We employ the PRISMA methodology – preferred reporting items for systematic reviews and meta-analyses to review (n=1246) and analyze (n=84) studies published in 2015 and onwards for key insights. Main results. The results highlight that current research primarily focuses on interpretability for developers and researchers, aiming to justify outcomes and enhance model performance. We discuss the unique approaches, advantages, and limitations of XAI4BCI from the literature. We draw insights from philosophy, psychology, and social sciences. We propose a design space for XAI4BCI, considering the evolving need to visualize and investigate predictive model outcomes customised for various stakeholders in the BCI development and deployment lifecycle. Significance. This paper is the first to focus solely on reviewing XAI4BCI research articles. This systematic review and meta-analysis findings with the proposed design space prompt important di

关键词： brain computer interface

来源：评论

学校读者我要写书评

暂无评论

Audiovisual Speaker Separation with Full- and Sub-Band Modeling in the Time-Frequency Domain

Audiovisual Speaker Separation with Full- and Sub-Band Model...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Vahid Ahmadi Kalkhorani Anurag Kumar Ke Tan Buye Xu DeLiang Wang Department of Computer Science and Engineering Ohio State University USA Meta Reality Labs USA Center for Cognitive and Brain Sciences The Ohio State University USA

We introduce a new deep learning model for talker-independent audiovisual speaker separation in noisy conditions in the time-frequency domain. The inputs to the model include noisy multi-talker mixtures and the corresponding cropped face images. Our approach incorporates cross-attention audiovisual fusion, effectively merging audio and visual features and enabling seamless information interchange between auditory and visual modalities. These fused features drive a separator module, which separates the acoustic features of individual speakers. The separator module is based on the recently proposed TF-Gridnet, which comprises an intra-frame full-band component, a sub-band temporal module that captures frequency-specific temporal dependencies, and a cross-attention module dedicated to extracting long-term fused audiovisual features. To encourage the utilization of visual streams during training, we employ a Signal-to-Noise Ratio (SNR) scheduler. Experimental results demonstrate that the proposed model advances the state-of- the-art speaker separation performance in several audiovisual benchmark datasets.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Evaluating Fast Adaptability of Neural Networks for brain-computer Interface

Evaluating Fast Adaptability of Neural Networks for Brain-Co...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Anupam Sharma Krishna Miyapuram Department of Computer Science & Engineering Indian Institute of Technology Gandhinagar Gujarat India Department of Cognitive & Brain Sciences Indian Institute of Technology Gandhinagar Gujarat India

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Electroencephalography (EEG) classification is a versatile and portable technique for building non-invasive brain-computer Interfaces (BCI). However, the classifiers that decode cognitive states from EEG brain data perform poorly when tested on newer domains, such as tasks or individuals absent during model training. Researchers have recently used complex strategies like Model-agnostic meta-learning (MAML) for domain adaptation. Nevertheless, there is a need for an evaluation strategy to evaluate the fast adaptability of the models, as this characteristic is essential for real-life BCI applications for quick calibration. We used motor movement and imaginary signals as input to Convolutional Neural Networks (CNN) based classifier for the experiments. Datasets with EEG signals typically have fewer examples and higher time resolution. Even though batch-normalization is preferred for Convolutional Neural Networks (CNN), we empirically show that layer-normalization can improve the adaptability of CNN-based EEG classifiers with not more than ten fine-tuning steps. In summary, the present work (i) proposes a simple strategy to evaluate fast adaptability, and (ii) empirically demonstrate fast adaptability across individuals as well as across tasks with simple transfer learning as compared to MAML approach.

关键词： Training Adaptation models Transfer learning brain modeling Motors Electroencephalography brain-computer interfaces

来源：评论

学校读者我要写书评

暂无评论

Evaluating Fast Adaptability of Neural Networks for brain-computer Interface

arXiv

引用

arXiv 2024年

作者： Sharma, Anupam Miyapuram, Krishna Department of Computer Science & Engineering Indian Institute of Technology Gujarat Gandhinagar India Department of Cognitive & Brain Sciences Indian Institute of Technology Gujarat Gandhinagar India

关键词： Electroencephalography

来源：评论

学校读者我要写书评

暂无评论

Emotion Recognition by Learning the Manifold of Fused Multiscale Information of EEG Signals

引用

IEEE Transactions on Affective Computing 2025年

作者： Li, Cunbo Zhang, Shuhan Mu, Yufeng Yang, Lei Peng, Yueheng Li, Fali Zhang, Yangsong Liang, Zhen Cao, Zehong Wan, Feng Yao, Dezhong Li, Peiyang Xu, Peng University of Electronic Science and Technology of China Clinical Hospital of Chengdu Brain Science Institute MOE Key Lab for Neuroinformation School of Life Science and Technology Chengdu610054 China University of Macau Centre for Cognitive and Brain Sciences Institute of Collaborative Innovation China Southwest University of Science and Technology School of Computer Science and Technology Mianyang621010 China Shenzhen University School of Biomedical Engineering Medical School Shenzhen518060 China Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging Shenzhen518060 China University of South Australia STEM Mawson Lakes Campus AdelaideSA5095 Australia School of Computer Science University of Technology Sydney Australia University of Macau Department of Electrical and Computer Engineering Faculty of Science and Technology Centre for Cognitive and Brain Sciences Institute of Collaborative Innovation China Chongqing University of Posts and Telecommunications School of Bioinfomatics Chongqing400065 China

Recent research has consistently indicated that the fusion of electroencephalography (EEG) features from multiple modalities can integrate cognitive state expressions across diverse dimensions, resulting in a substantial increase in emotion recognition accuracy. However, redundant information within the fused multimodal features could lead to the curse of dimensionality and overfitting of the learning model. In this work, we propose a multiscale EEG feature fusion and representation strategy for EEG emotion recognition named manifold of multiscale information fusion (MMIF), in which the optimal manifold of the multiscale fusion of local and global brain activation patterns can be automatically learned to realize an efficient representation of emotional EEG signals. To evaluate the performance, in this work, both off- and online EEG emotion recognition experiments were conducted, and the experimental results consistently verified the effectiveness and feasibility of the MMIF applied in real-time emotion decoding systems. Furthermore, the analytical experiments confirmed the discriminative capabilities and cognitive interpretability of the MMIF. In summary, the proposed MMIF model may provide an efficient avenue for exploring representations and enhancing the discrimination of multimodal fusion features, which may also provide a promising solution for designing online affective braincomputer interaction systems. © 2010-2012 IEEE.

关键词： Information fusion

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：