检索结果-内蒙古大学图书馆

SpikingMiniLM: energy-efficient spiking transformer for natural language understanding

science China(Information sciences) 2024年第10期67卷 115-128页

作者： Jiayu ZHANG Jiangrong SHEN Zeke WANG Qinghai GUO Rui YAN Gang PAN Huajin TANG College of Computer Science and Technology Zhejiang University The State Key Lab of Brain-Machine Intelligence Zhejiang University Collaborative Innovation Center of Artificial Intelligence Zhejiang University Advanced Computing and Storage Laboratory Huawei Technologies Co. Ltd. College of Computer Science and Technology Zhejiang University of Technology MOE Frontier Science Center for Brain Science and Brain-Machine Integration Zhejiang University

In the era of large-scale pretrained models, artificial neural networks（ANNs） have excelled in natural language understanding（NLU） tasks. However, their success often necessitates substantial computational resources and energy consumption. To address this, we explore the potential of spiking neural networks（SNNs） in NLU——a promising avenue with demonstrated advantages, including reduced power consumption and improved efficiency due to their event-driven characteristics. We propose the SpikingMiniLM,a novel spiking Transformer model tailored for natural language understanding. We first introduce a multi-step encoding method to convert text embeddings into spike trains. Subsequently, we redesign the attention mechanism and residual connections to make our model operate on the pure spike-based paradigm without any normalization technique. To facilitate stable and fast convergence, we propose a general parameter initialization method grounded in the stable firing rate principle. Furthermore, we apply an ANN-to-SNN knowledge distillation to overcome the challenges of pretraining SNNs. Our approach achieves a macro-average score of 75.5 on the dev sets of the GLUE benchmark, retaining 98% of the performance exhibited by the teacher model MiniLMv2. Our smaller model also achieves similar performance to BERTMINIwith fewer parameters and much lower energy consumption, underscoring its competitiveness and resource efficiency in NLU tasks.

关键词： spiking neural networks natural language understanding spiking Transformer spike-based attention multi-step encoding ANN-to-SNN distillation

来源：评论

学校读者我要写书评

暂无评论

Attention Detection Using EEG Signals and machine Learning: A Review

引用

machine intelligence Research 2025年第2期22卷 219-238页

作者： Qianru Sun Yueying Zhou Peiliang Gong Daoqiang Zhang Key Laboratory of Brain-Machine Intelligence Technology Ministry of EducationNanjing University of Aeronautics and AstronauticsNanjing 211106China College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing 211106China

Attention detection using electroencephalogram(EEG)signals has become a popular ***,there seems to be a notable gap in the literature regarding comprehensive and systematic reviews of machine learning methods for attention detection using EEG ***,this survey outlines recent advances in EEG-based attention detection within the past five years,with a primary focus on auditory attention detection(AAD)and attention level ***,we provide a brief overview of commonly used paradigms,preprocessing techniques,and artifact-handling methods,as well as listing accessible datasets used in these ***,we summarize the machine learning methods for classification in this field and divide them into two categories:traditional machine learning methods and deep learning *** also analyse the most frequently used methods and discuss the factors influencing each technique′s performance and ***,we discuss the existing challenges and future trends in this field.

关键词： Attention detection electroencephalogram(EEG) machine learning deep learning brain-computer interface.

来源：评论

学校读者我要写书评

暂无评论

FEEL-SNN: Robust Spiking Neural Networks with Frequency Encoding and Evolutionary Leak Factor 38

FEEL-SNN: Robust Spiking Neural Networks with Frequency Enco...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Xu, Mengting Ma, De Tang, HuaJin Zheng, Qian Pan, Gang The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China College of Computer Science and Technology Zhejiang University Hangzhou China

Currently, researchers think that the inherent robustness of spiking neural networks (SNNs) stems from their biologically plausible spiking neurons, and are dedicated to developing more bio-inspired models to defend attacks. However, most work relies solely on experimental analysis and lacks theoretical support, and the direct-encoding method and fixed membrane potential leak factor they used in spiking neurons are simplified simulations of those in the biological nervous system, which makes it difficult to ensure generalizability across all datasets and networks. Contrarily, the biological nervous system can stay reliable even in a highly complex noise environment, one of the reasons is selective visual attention and non-fixed membrane potential leaks in biological neurons. This biological finding has inspired us to design a highly robust SNN model that closely mimics the biological nervous system. In our study, we first present a unified theoretical framework for SNN robustness constraint, which suggests that improving the encoding method and evolution of the membrane potential leak factor in spiking neurons can improve SNN robustness. Subsequently, we propose a robust SNN (FEEL-SNN) with Frequency Encoding (FE) and Evolutionary Leak factor (EL) to defend against different noises, mimicking the selective visual attention mechanism and non-fixed leak observed in biological systems. Experimental results confirm the efficacy of both our FE, EL, and FEEL methods, either in isolation or in conjunction with established robust enhancement algorithms, for enhancing the robustness of SNNs. Our code is available at https://***/zju-bmi-lab/FEEL_SNN. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Vision-Language Navigation with Energy-Based Policy 38

Vision-Language Navigation with Energy-Based Policy

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Liu, Rui Wang, Wenguan Yang, Yi The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China College of Computer Science and Technology Zhejiang University Hangzhou China

Vision-language navigation (VLN) requires an agent to execute actions following human instructions. Existing VLN models are optimized through expert demonstrations by supervised behavioural cloning or incorporating manual reward engineering. While straightforward, these efforts overlook the accumulation of errors in the Markov decision process, and struggle to match the distribution of the expert policy. Going beyond this, we propose an Energy-based Navigation Policy (ENP) to model the joint state-action distribution using an energy-based model. At each step, low energy values correspond to the state-action pairs that the expert is most likely to perform, and vice versa. Theoretically, the optimization objective is equivalent to minimizing the forward divergence between the occupancy measure of the expert and ours. Consequently, ENP learns to globally align with the expert policy by maximizing the likelihood of the actions and modeling the dynamics of the navigation states in a collaborative manner. With a variety of VLN architectures, ENP achieves promising performances on R2R, REVERIE, RxR, and R2R-CE, unleashing the power of existing VLN models. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Blind Driving System Leveraging Multimodal Feedback 10

A Blind Driving System Leveraging Multimodal Feedback

引用

10th IEEE Smart World Congress, SWC 2024

作者： Zhang, Ruizhe Zhao, Sha Yang, Wei Li, Shijian Pan, Gang Zhejiang University the College of Computer Science and Technology Hangzhou China Zhejiang University the State Key Laboratory of Brain-machine Intelligence Hangzhou China

ISBN: (纸本)9798331520861

Driving poses a significant challenge for blind and visually impaired (BVI) individuals, due to the loss of vision. Fortunately, the metaverse presents a promising platform where BVI individuals can experience independent driving just like sighted individuals. In order to assist such individuals in driving in the metaverse, we have devised a blind driving system that leverages multimodal feedback. We utilized stereo headphones and vibration motors to convey necessary information, including current and predicted vehicle states (i.e., speed, position, and direction) during driving, through sound, voice, and vibration feedback. Meanwhile, in order to improve the user-friendliness, we reduced the user's information burden by data combination and compression. With the help of the multimodal feedback, BVI drivers could intuitively perceive the position and direction of the vehicle, thereby improving their driving experience. To evaluate the effectiveness of the system, we designed virtual scenarios to simulate a real driving environment, and we further devised two tasks: lane-keeping and lane-changing. We conducted user studies where sighted individuals wearing blindfolds were involved in the two tasks. The results suggest that our system demonstrates a beneficial effect in assisting BVI individuals in driving. Users could keep the vehicle in the center area of the lane for over ninety percent of driving time, and completed a lane-changing task with a high success rate. This study provides a potential solution for blind driving for BVI people by employing multimodal feedback. © 2024 IEEE.

关键词： Stereo image processing

来源：评论

学校读者我要写书评

暂无评论

Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks 38

Rethinking the Membrane Dynamics and Optimization Objectives...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Shen, Hangchi Zheng, Qian Wang, Huamin Pan, Gang College of Artificial Intelligence Southwest University China Chongqing Key Laboratory of Brain Inspired Computing and Intelligent Chips China The State Key Lab of Brain-Machine Intelligence Zhejiang University China College of Computer Science and Technology Zhejiang University China

Despite spiking neural networks (SNNs) have demonstrated notable energy efficiency across various fields, the limited firing patterns of spiking neurons within fixed time steps restrict the expression of information, which impedes further improvement of SNN performance. In addition, current implementations of SNNs typically consider the firing rate or average membrane potential of the last layer as the output, lacking exploration of other possibilities. In this paper, we identify that the limited spike patterns of spiking neurons stem from the initial membrane potential (IMP), which is set to 0. By adjusting the IMP, the spiking neurons can generate additional firing patterns and pattern mappings. Furthermore, we find that in static tasks, the accuracy of SNNs at each time step increases as the membrane potential evolves from zero. This observation inspires us to propose a learnable IMP, which can accelerate the evolution of membrane potential and enables higher performance within a limited number of time steps. Additionally, we introduce the last time step (LTS) approach to accelerate convergence in static tasks, and we propose a label smooth temporal efficient training (TET) loss to mitigate the conflicts between optimization objective and regularization term in the vanilla TET. Our methods improve the accuracy by 4.05% on ImageNet compared to baseline and achieve state-of-the-art performance of 87.80% on CIFAR10-DVS and 87.86% on N-Caltech101. The code is available at https://***/StephenTaylor1998/IMP-SNN. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Alleviating the Semantic Gap for Generalized fMRI-to-Image Reconstruction 37

Alleviating the Semantic Gap for Generalized fMRI-to-Image R...

引用

37th Conference on Neural Information Processing Systems, NeurIPS 2023

作者： Fang, Tao Zheng, Qian Pan, Gang College of Computer Science and Technology Zhejiang University Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China

ISBN: (纸本)9781713899921

Although existing fMRI-to-image reconstruction methods could predict high-quality images, they do not explicitly consider the semantic gap between training and testing data, resulting in reconstruction with unstable and uncertain semantics. This paper addresses the problem of generalized fMRI-to-image reconstruction by explicitly alleviates the semantic gap. Specifically, we leverage the pre-trained CLIP model to map the training data to a compact feature representation, which essentially extends the sparse semantics of training data to dense ones, thus alleviating the semantic gap of the instances nearby known concepts (i.e., inside the training super-classes). Inspired by the robust low-level representation in fMRI data, which could help alleviate the semantic gap for instances that far from the known concepts (i.e., outside the training super-classes), we leverage structural information as a general cue to guide image reconstruction. Further, we quantify the semantic uncertainty based on probability density estimation and achieve Generalized fMRI-to-image reconstruction by adaptively integrating Expanded Semantics and Structural information (GESS) within a diffusion process. Experimental results demonstrate that the proposed GESS model outperforms state-of-the-art methods, and we propose a generalized scenario split strategy to evaluate the advantage of GESS in closing the semantic gap. Our codes are available at https://***/duolala1/GESS. © 2023 Neural information processing systems foundation. All rights reserved.

关键词： Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

A Graded Speed Control Method for Cyborg Rats Based on Electrical Stimulation of the Cuneiform Nucleus

引用

Journal of Bionic Engineering 2024年第3期21卷 1334-1346页

作者： Hanyi Ling Le Han Nenggan Zheng Qiushi Academy for Advanced Studies(QAAS) Zhejiang UniversityHangzhou310007China The State Key Lab of Brain-Machine Intelligence Zhejiang UniversityHangzhou310007China College of Biomedical Engineering&Instrument Science Zhejiang UniversityHangzhou310007China College of Computer Science and Technology Zhejiang UniversityHangzhou310007China CCAI by MOE and Zhejiang Provincial Government(ZJU) Hangzhou310007China

The precise movement speed regulation is a key factor to improve the control effect and efficiency of the cyborg ***,the current stimulation techniques cannot realize the graded control of the *** this study,we achieved the multi-level speed regulation of cyborg rats in the large open field and treadmill by specifically targeting the Cuneiform Nucleus(CnF)of the Mesencephalic Locomotor Region(MLR).Detailed,we measured the influence of each stimulation parameter on the speed control process which included the real-time speed,accelerated speed,response time,and acceleration *** concluded that the pulse period and the pulse width were the main determinants influencing the accelerated speed of cyborg *** the amplitude of stimulation was found to affect the response time exhibited by the cyborg *** study provides valuable insights into the regulation of rat locomotion speed and highlights the potential for utilizing this approach in various experimental settings.

关键词： Cyborg rats Cuneiform nucleus Graded control Speed regulation

来源：评论

学校读者我要写书评

暂无评论

DRIP: Unleashing Diffusion Priors for Joint Foreground and Alpha Prediction in Image Matting 38

DRIP: Unleashing Diffusion Priors for Joint Foreground and A...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Li, Xiaodi Yang, Zongxin Quan, Ruijie Yang, Yi The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China CCAI College of Computer Science and Technology Zhejiang University Hangzhou China DBMI HMS Harvard University Boston United States

Recovering the foreground color and opacity/alpha matte from a single image (i.e., image matting) is a challenging and ill-posed problem where data priors play a critical role in achieving precise results. Due to the limited matting datasets, traditional methods usually struggle to produce high-quality *** address this, we explore the potential of leveraging vision priors embedded in pre-trained latent diffusion models (LDM) for estimating foreground RGBA values in challenging scenarios and rare objects. We introduce Drip, a novel approach for image matting that harnesses the rich prior knowledge of LDM models. Our method incorporates a switcher and a cross-domain attention mechanism to extend the original LDM for joint prediction of the foreground color and opacity. This setup facilitates mutual information exchange and ensures high consistency across both modalities. To mitigate the inherent reconstruction errors of the LDM's VAE decoder, we propose a latent transparency decoder to align the RGBA prediction with the input image, thereby reducing discrepancies. Comprehensive experimental results demonstrate that our approach achieves state-of-the-art performance in foreground and alpha predictions and shows remarkable generalizability across various benchmarks. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation 41

Safe Reinforcement Learning using Finite-Horizon Gradient-ba...

引用

41st International Conference on machine Learning, ICML 2024

作者： Dai, Juntao Yang, Yaodong Zheng, Qian Pan, Gang College of Computer Science and Technology Zhejiang University Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China Center for AI Safety and Governance Peking University Beijing China

A key aspect of Safe Reinforcement Learning (Safe RL) involves estimating the constraint condition for the next policy, which is crucial for guiding the optimization of safe policy updates. However, the existing Advantage-based Estimation (ABE) method relies on the infinite-horizon discounted advantage function. This dependence leads to catastrophic errors in finite-horizon scenarios with non-discounted constraints, resulting in safety-violation updates. In response, we propose the first estimation method for finite-horizon non-discounted constraints in deep Safe RL, termed Gradient-based Estimation (GBE), which relies on the analytic gradient derived along trajectories. Our theoretical and empirical analyses demonstrate that GBE can effectively estimate constraint changes over a finite horizon. Constructing a surrogate optimization problem with GBE, we developed a novel Safe RL algorithm called Constrained Gradient-based Policy Optimization (CGPO). CGPO identifies feasible optimal policies by iteratively resolving sub-problems within trust regions. Our empirical results reveal that CGPO, unlike baseline algorithms, successfully estimates the constraint functions of subsequent policies, thereby ensuring the efficiency and feasibility of each update. Copyright 2024 by the author(s)

关键词： Deep reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：