检索结果-内蒙古大学图书馆

SpikingMiniLM: energy-efficient spiking transformer for natural language understanding

science China(Information sciences) 2024年第10期67卷 115-128页

作者： Jiayu ZHANG Jiangrong SHEN Zeke WANG Qinghai GUO Rui YAN Gang PAN Huajin TANG College of Computer Science and Technology Zhejiang University The State Key Lab of Brain-Machine Intelligence Zhejiang University Collaborative Innovation Center of Artificial Intelligence Zhejiang University Advanced Computing and Storage Laboratory Huawei Technologies Co. Ltd. College of Computer Science and Technology Zhejiang University of Technology MOE Frontier Science Center for Brain Science and Brain-Machine Integration Zhejiang University

In the era of large-scale pretrained models, artificial neural networks（ANNs） have excelled in natural language understanding（NLU） tasks. However, their success often necessitates substantial computational resources and energy consumption. To address this, we explore the potential of spiking neural networks（SNNs） in NLU——a promising avenue with demonstrated advantages, including reduced power consumption and improved efficiency due to their event-driven characteristics. We propose the SpikingMiniLM,a novel spiking Transformer model tailored for natural language understanding. We first introduce a multi-step encoding method to convert text embeddings into spike trains. Subsequently, we redesign the attention mechanism and residual connections to make our model operate on the pure spike-based paradigm without any normalization technique. To facilitate stable and fast convergence, we propose a general parameter initialization method grounded in the stable firing rate principle. Furthermore, we apply an ANN-to-SNN knowledge distillation to overcome the challenges of pretraining SNNs. Our approach achieves a macro-average score of 75.5 on the dev sets of the GLUE benchmark, retaining 98% of the performance exhibited by the teacher model MiniLMv2. Our smaller model also achieves similar performance to BERTMINIwith fewer parameters and much lower energy consumption, underscoring its competitiveness and resource efficiency in NLU tasks.

关键词： spiking neural networks natural language understanding spiking Transformer spike-based attention multi-step encoding ANN-to-SNN distillation

来源：评论

学校读者我要写书评

暂无评论

Darwin3: a large-scale neuromorphic chip with a novel ISA and on-chip learning

引用

National science Review 2024年第5期11卷 29-45页

作者： De Ma Xiaofei Jin Shichun Sun Yitao Li Xundong Wu Youneng Hu Fangchao Yang Huajin Tang Xiaolei Zhu Peng Lin Gang Pan College of Computer Science and Technology Zhejiang University Research Center for Intelligent Computing Hardware Zhejiang Lab The State Key Lab of Brain-Machine Intelligence Zhejiang University MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University College of Micro-Nano College of Micro-Nano Electronics Zhejiang University

Spiking neural networks(SNNs) are gaining increasing attention for their biological plausibility and potential for improved computational efficiency. To match the high spatial-temporal dynamics in SNNs,neuromorphic chips are highly desired to execute SNNs in hardware-based neuron and synapse circuits directly. This paper presents a large-scale neuromorphic chip named Darwin3 with a novel instruction set architecture, which comprises 10 primary instructions and a few extended instructions. It supports flexible neuron model programming and local learning rule designs. The Darwin3 chip architecture is designed in a mesh of computing nodes with an innovative routing algorithm. We used a compression mechanism to represent synaptic connections, significantly reducing memory usage. The Darwin3 chip supports up to2.35 million neurons, making it the largest of its kind on the neuron scale. The experimental results showed that the code density was improved by up to 28.3× in Darwin3, and that the neuron core fan-in and fan-out were improved by up to 4096× and 3072× by connection compression compared to the physical memory depth. Our Darwin3 chip also provided memory saving between 6.8× and 200.8× when mapping convolutional spiking neural networks onto the chip, demonstrating state-of-the-art performance in accuracy and latency compared to other neuromorphic chips.

关键词： neuromorphic computing spiking neural network instruction set architecture connectivity compression

来源：评论

学校读者我要写书评

暂无评论

state-sensitive convolutional sparse coding for potential biomarker identification in brain signals

引用

science China(Information sciences) 2024年第5期67卷 270-286页

作者： Puli WANG Yu QI Gang PAN College of Computer Science and Technology Zhejiang University State Key Laboratory of Brain-Machine Intelligence Zhejiang University MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University Affiliated Mental Health Center & Hangzhou Seventh People's Hospital Zhejiang University School of Medicine

The identification of prototypical waveforms, such as sleep spindles and epileptic spikes, is crucial for the diagnosis of neurological disorders. These prototypical waveforms are usually recurrently presented in certain brain states, serving as potential biomarkers for clinical evaluations. Convolutional sparse coding(CSC) approaches have demonstrated strength in identifying recurrent patterns in time-series. However,existing CSC approaches do not explicitly explore state-specific patterns, making it difficult to identify state-related biomarkers. To address this problem, we propose state-sensitive CSC to learn state-specific prototypical waveforms. Specifically, we model signals of a certain state with specific waveforms that only appear frequently in this state and background waveforms that are independent of states. Based on this,state-sensitive CSC separates state-specific waveforms from background ones explicitly by incorporating incoherence constraints into optimizations. Experiments with epilepsy brain signals demonstrate that our approach can effectively identify prototypical waveforms in pre-ictal states, providing potential biomarkers for seizure prediction. Our approach provides a promising tool for automatic biomarker candidate identification.

关键词： convolutional sparse coding prototypical waveforms state-specific dictionary biomarker identification neural signal processing

来源：评论

学校读者我要写书评

暂无评论

Attention Detection Using EEG Signals and machine Learning: A Review

引用

machine intelligence Research 2025年第2期22卷 219-238页

作者： Qianru Sun Yueying Zhou Peiliang Gong Daoqiang Zhang Key Laboratory of Brain-Machine Intelligence Technology Ministry of EducationNanjing University of Aeronautics and AstronauticsNanjing 211106China College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing 211106China

Attention detection using electroencephalogram(EEG)signals has become a popular ***,there seems to be a notable gap in the literature regarding comprehensive and systematic reviews of machine learning methods for attention detection using EEG ***,this survey outlines recent advances in EEG-based attention detection within the past five years,with a primary focus on auditory attention detection(AAD)and attention level ***,we provide a brief overview of commonly used paradigms,preprocessing techniques,and artifact-handling methods,as well as listing accessible datasets used in these ***,we summarize the machine learning methods for classification in this field and divide them into two categories:traditional machine learning methods and deep learning *** also analyse the most frequently used methods and discuss the factors influencing each technique′s performance and ***,we discuss the existing challenges and future trends in this field.

关键词： Attention detection electroencephalogram(EEG) machine learning deep learning brain-computer interface.

来源：评论

学校读者我要写书评

暂无评论

Inter-Channel Correlation-Based EMI Noise Removal (ICER) for Shielding-Free Low-Field MRI

引用

IEEE Transactions on Biomedical Engineering 2025年第7期72卷 2095-2104页

作者： Huang, Yiman Qu, Shuxian Xie, Yushu Wang, Hanlei Zhang, Xinlin Zhang, Xiaotong Zhejiang University College of Electrical Engineering MOE Frontier Science Center for Brain Science and Brain-machine Integration Hangzhou China Zhejiang University College of Biomedical Engineering & Instrument Science MOE Frontier Science Center for Brain Science and Brain-machine Integration Hangzhou China Fuzhou University College of Physics and Information Engineering Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology Fuzhou China Zhejiang University College of Electrical Engineering the Second Affiliated Hospital of Zhejiang University School of Medicine the State Key Lab of Brain-Machine Intelligence MOE Frontier Science Center for Brain Science and Brain-machine Integration Hangzhou China

Objective: For low-field magnetic resonance imaging (MRI) in unshielded environment, existing methods have been proposed to eliminate electromagnetic interference (EMI) noise in each single radio-frequency (RF) receive coil. In the present study, we propose to use the EMI information from multiple MRI receive coils collectively in EMI denoising. Methods: The proposed method leverages the information of inter-channel correlation, including EMI detectors and RF receive coils to remove EMI noise. Calibration signals from both EMI detectors and RF receivers are concatenated to determine a de-correlation matrix, which is then used to denoise MRI signals. Results: Saline phantom and in vivo experiments demonstrated the efficacy of the proposed method in EMI elimination, showing that the proposed method outperformed advanced EMI elimination methods with up to 16.34% improvement in EMI noise removal percentage (NRP) and 1.58dB improvement in signal-to-noise ratio (SNR), along with reduced computational times. Conclusion: The proposed method effectively removes EMI noise and shows improved performance by using information from all receive coils. Significance: This method allows for the design of multi-receive coils for low-field MRI such as phased-arrays, which have the potential to enhance the performance in noise removal and improve the SNR in MRI signal acquisition. © 2025 IEEE.

关键词： Signal to noise ratio

来源：评论

学校读者我要写书评

暂无评论

FEEL-SNN: Robust Spiking Neural Networks with Frequency Encoding and Evolutionary Leak Factor 38

FEEL-SNN: Robust Spiking Neural Networks with Frequency Enco...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Xu, Mengting Ma, De Tang, HuaJin Zheng, Qian Pan, Gang The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China College of Computer Science and Technology Zhejiang University Hangzhou China

Currently, researchers think that the inherent robustness of spiking neural networks (SNNs) stems from their biologically plausible spiking neurons, and are dedicated to developing more bio-inspired models to defend attacks. However, most work relies solely on experimental analysis and lacks theoretical support, and the direct-encoding method and fixed membrane potential leak factor they used in spiking neurons are simplified simulations of those in the biological nervous system, which makes it difficult to ensure generalizability across all datasets and networks. Contrarily, the biological nervous system can stay reliable even in a highly complex noise environment, one of the reasons is selective visual attention and non-fixed membrane potential leaks in biological neurons. This biological finding has inspired us to design a highly robust SNN model that closely mimics the biological nervous system. In our study, we first present a unified theoretical framework for SNN robustness constraint, which suggests that improving the encoding method and evolution of the membrane potential leak factor in spiking neurons can improve SNN robustness. Subsequently, we propose a robust SNN (FEEL-SNN) with Frequency Encoding (FE) and Evolutionary Leak factor (EL) to defend against different noises, mimicking the selective visual attention mechanism and non-fixed leak observed in biological systems. Experimental results confirm the efficacy of both our FE, EL, and FEEL methods, either in isolation or in conjunction with established robust enhancement algorithms, for enhancing the robustness of SNNs. Our code is available at https://***/zju-bmi-lab/FEEL_SNN. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Vision-Language Navigation with Energy-Based Policy 38

Vision-Language Navigation with Energy-Based Policy

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Liu, Rui Wang, Wenguan Yang, Yi The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China College of Computer Science and Technology Zhejiang University Hangzhou China

Vision-language navigation (VLN) requires an agent to execute actions following human instructions. Existing VLN models are optimized through expert demonstrations by supervised behavioural cloning or incorporating manual reward engineering. While straightforward, these efforts overlook the accumulation of errors in the Markov decision process, and struggle to match the distribution of the expert policy. Going beyond this, we propose an Energy-based Navigation Policy (ENP) to model the joint state-action distribution using an energy-based model. At each step, low energy values correspond to the state-action pairs that the expert is most likely to perform, and vice versa. Theoretically, the optimization objective is equivalent to minimizing the forward divergence between the occupancy measure of the expert and ours. Consequently, ENP learns to globally align with the expert policy by maximizing the likelihood of the actions and modeling the dynamics of the navigation states in a collaborative manner. With a variety of VLN architectures, ENP achieves promising performances on R2R, REVERIE, RxR, and R2R-CE, unleashing the power of existing VLN models. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Blind Driving System Leveraging Multimodal Feedback 10

A Blind Driving System Leveraging Multimodal Feedback

引用

10th IEEE Smart World Congress, SWC 2024

作者： Zhang, Ruizhe Zhao, Sha Yang, Wei Li, Shijian Pan, Gang Zhejiang University the College of Computer Science and Technology Hangzhou China Zhejiang University the State Key Laboratory of Brain-machine Intelligence Hangzhou China

ISBN: (纸本)9798331520861

Driving poses a significant challenge for blind and visually impaired (BVI) individuals, due to the loss of vision. Fortunately, the metaverse presents a promising platform where BVI individuals can experience independent driving just like sighted individuals. In order to assist such individuals in driving in the metaverse, we have devised a blind driving system that leverages multimodal feedback. We utilized stereo headphones and vibration motors to convey necessary information, including current and predicted vehicle states (i.e., speed, position, and direction) during driving, through sound, voice, and vibration feedback. Meanwhile, in order to improve the user-friendliness, we reduced the user's information burden by data combination and compression. With the help of the multimodal feedback, BVI drivers could intuitively perceive the position and direction of the vehicle, thereby improving their driving experience. To evaluate the effectiveness of the system, we designed virtual scenarios to simulate a real driving environment, and we further devised two tasks: lane-keeping and lane-changing. We conducted user studies where sighted individuals wearing blindfolds were involved in the two tasks. The results suggest that our system demonstrates a beneficial effect in assisting BVI individuals in driving. Users could keep the vehicle in the center area of the lane for over ninety percent of driving time, and completed a lane-changing task with a high success rate. This study provides a potential solution for blind driving for BVI people by employing multimodal feedback. © 2024 IEEE.

关键词： Stereo image processing

来源：评论

学校读者我要写书评

暂无评论

Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks 38

Rethinking the Membrane Dynamics and Optimization Objectives...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Shen, Hangchi Zheng, Qian Wang, Huamin Pan, Gang College of Artificial Intelligence Southwest University China Chongqing Key Laboratory of Brain Inspired Computing and Intelligent Chips China The State Key Lab of Brain-Machine Intelligence Zhejiang University China College of Computer Science and Technology Zhejiang University China

Despite spiking neural networks (SNNs) have demonstrated notable energy efficiency across various fields, the limited firing patterns of spiking neurons within fixed time steps restrict the expression of information, which impedes further improvement of SNN performance. In addition, current implementations of SNNs typically consider the firing rate or average membrane potential of the last layer as the output, lacking exploration of other possibilities. In this paper, we identify that the limited spike patterns of spiking neurons stem from the initial membrane potential (IMP), which is set to 0. By adjusting the IMP, the spiking neurons can generate additional firing patterns and pattern mappings. Furthermore, we find that in static tasks, the accuracy of SNNs at each time step increases as the membrane potential evolves from zero. This observation inspires us to propose a learnable IMP, which can accelerate the evolution of membrane potential and enables higher performance within a limited number of time steps. Additionally, we introduce the last time step (LTS) approach to accelerate convergence in static tasks, and we propose a label smooth temporal efficient training (TET) loss to mitigate the conflicts between optimization objective and regularization term in the vanilla TET. Our methods improve the accuracy by 4.05% on ImageNet compared to baseline and achieve state-of-the-art performance of 87.80% on CIFAR10-DVS and 87.86% on N-Caltech101. The code is available at https://***/StephenTaylor1998/IMP-SNN. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Alleviating the Semantic Gap for Generalized fMRI-to-Image Reconstruction 37

Alleviating the Semantic Gap for Generalized fMRI-to-Image R...

引用

37th Conference on Neural Information Processing Systems, NeurIPS 2023

作者： Fang, Tao Zheng, Qian Pan, Gang College of Computer Science and Technology Zhejiang University Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China

ISBN: (纸本)9781713899921

Although existing fMRI-to-image reconstruction methods could predict high-quality images, they do not explicitly consider the semantic gap between training and testing data, resulting in reconstruction with unstable and uncertain semantics. This paper addresses the problem of generalized fMRI-to-image reconstruction by explicitly alleviates the semantic gap. Specifically, we leverage the pre-trained CLIP model to map the training data to a compact feature representation, which essentially extends the sparse semantics of training data to dense ones, thus alleviating the semantic gap of the instances nearby known concepts (i.e., inside the training super-classes). Inspired by the robust low-level representation in fMRI data, which could help alleviate the semantic gap for instances that far from the known concepts (i.e., outside the training super-classes), we leverage structural information as a general cue to guide image reconstruction. Further, we quantify the semantic uncertainty based on probability density estimation and achieve Generalized fMRI-to-image reconstruction by adaptively integrating Expanded Semantics and Structural information (GESS) within a diffusion process. Experimental results demonstrate that the proposed GESS model outperforms state-of-the-art methods, and we propose a generalized scenario split strategy to evaluate the advantage of GESS in closing the semantic gap. Our codes are available at https://***/duolala1/GESS. © 2023 Neural information processing systems foundation. All rights reserved.

关键词： Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：