检索结果-内蒙古大学图书馆

Hybrid Spiking Vision Transformer for Event-Based Object Detection

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Xu, Qi Deng, Jie Shen, Jiangrong Chen, Biwu Tang, Huajin Pan, Gang School of Computer Science and Technology Dalian University of Technology Dalian China College of Computer Science and Technology Zhejiang University Hangzhou China State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China Shanghai Radio Equipment Research Institute Shanghai China

Event-based object detection has gradually drawn attention for its high time resolution, high dynamic range, and asynchronous address event representation. Spiking Neural Networks (SNNs) offer distinct advantages, including low energy consumption and rich spatiotemporal dynamics. Therefore, in this study, a novel hybrid spike vision Transformer (HsVT) model is proposed, which combines spatial feature extraction and temporal feature extraction components. The spatial feature extraction module is used to extract the local and global features in the input data, and the temporal feature extraction module is used to capture the time dependencies and long-term patterns in the input sequence. With this combination, HsVT models can better capture spatial and temporal features when processing sequence data, thus improving the modeling ability of complex event-based object detection tasks. Besides, we collect the fall detection datasets and make them public as the benchmark of event-based object detection tasks. The Fall detection dataset with event-based camera provides facial privacy protection and saves memory storage due to the event representation manner. We evaluated the performance of HsVT methods on different model sizes and compared their performance on GEN1 and Fall detection datasets. The experimental results show that the HsVT method has achieved remarkable performance improvement in the event detection task with fewer parameters, and provides an effective solution for the event-based object detection task. © 2024, The Authors. All rights reserved.

关键词： Object detection

Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Yangfan Zheng, Qian Li, Guoqi Tang, Huajin Pan, Gang College of Computer Science and Technology Zhejiang University Hangzhou310027 China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou310027 China Institute of Automation Chinese Academy of Sciences China

Deep learning has revolutionized artificial intelligence (AI), achieving remarkable progress in fields such as computer vision, speech recognition, and natural language processing. Moreover, the recent success of large language models (LLMs) has fueled a surge in research on large-scale neural networks. However, the escalating demand for computing resources and energy consumption has prompted the search for energy-efficient alternatives. Inspired by the human brain, spiking neural networks (SNNs) promise energy-efficient computation with event-driven spikes. To provide future directions toward building energy-efficient large SNN models, we present a survey of existing methods for developing deep spiking neural networks, with a focus on emerging Spiking Transformers. Our main contributions are as follows: (1) an overview of learning methods for deep spiking neural networks, categorized by ANN-to-SNN conversion and direct training with surrogate gradients;(2) an overview of network architectures for deep spiking neural networks, categorized by deep convolutional neural networks (DCNNs) and Transformer architecture;and (3) a comprehensive comparison of state-of-the-art deep SNNs with a focus on emerging Spiking Transformers. We then further discuss and outline future directions toward large-scale SNNs. © 2024, CC BY-NC-ND.

关键词： Convolutional neural networks

Deep-Learning-Based Dual Denoising Algorithm for Shielding-Free Magnetic Resonance Imaging

学校读者我要写书评

暂无评论

IEEE Transactions on Artificial Intelligence

IEEE Transactions on Artificial intelligence 2025年

作者： Hu, Yang Huang, Yiman Qu, Shuxian Xie, Yushu Fang, Yanwen Zhang, Xiaotong Zhejiang University College of Electrical Engineering MOE Frontier Science Center for Brain Science and Brainmachine Integration Hangzhou China Zhejiang University College of Biomedical Engineering & Instrument Science MOE Frontier Science Center for Brain Science and Brain-machine Integration Hangzhou China Heye Health Technology Co. Ltd. Anji China Zhejiang University College of Electrical Engineering The Second Affiliated Hospital of Zhejiang University School of Medicine The State Key Lab of Brain-Machine Intelligence MOE Frontier Science Center for Brain Science and Brain-machine Integration Hangzhou China

Electromagnetic shielding is crucial in magnetic resonance imaging (MRI), particularly at low- and ultra-low magnetic fields. However, the cost of employing physical enclosures for shielding can be prohibitively high. In this study, we introduce a comprehensive dual denoising algorithm, leveraging deep-learning techniques, specifically tailored for shielding-free MRI. This algorithm is structured into two distinct steps. In the first step, we employ a residual convolution neural network (residual CNN) to establish a mapping between the electromagnetic interference (EMI) signals detected by two EMI sensing coils and those detected by the MRI receive coils. In the subsequent step, we introduce TWGAN (Triple generators Wasserstein generative adversarial network), a unique composite neural network architecture comprising three generators, to refine the denoised outputs from the first step. Furthermore, we have devised a methodology to construct a denoising dataset that comprises realistic clean-noise pairs, addressing the challenges associated with obtaining such datasets in different field (especially in low-field and ultra-low field) MRI scenarios. We implemented the algorithm on a shielding-free 0.11T lowfield (LF) MRI system and conducted experiments on healthy human subjects. In comparing the convergence speed of the first step with existing algorithms, our results revealed that the utilization of residual CNN significantly accelerated the convergence process. Following the second step, we achieved cleaner images. Additionally, we applied the second step to shielding-free 0.055T ultra-low-field (ULF) MRI images, and the outcomes demonstrated that our method effectively addresses the limited denoising capabilities of existing algorithms in highly noise-contaminated MR images. These results highlight the potential of our algorithm for wider application and generalization in scenarios with complex electromagnetic noise. © 2020 IEEE.

关键词： Electromagnetic shielding

MINDGPT: INTERPRETING WHAT YOU SEE WITH NON-INVASIVE brain RECORDINGS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Jiaxuan Qi, Yu Wang, Yueming Pan, Gang College of Computer Science and Technology Zhejiang University China MOE Frontier Science Center for Brain Science and Brain-Machine Integration Zhejiang University China Qiushi Academy for Advanced Studies Zhejiang University China State Key Lab of Brain-Machine Intelligence Zhejiang Unversity China

Decoding of seen visual contents with non-invasive brain recordings has important scientific and practical values. Efforts have been made to recover the seen images from brain signals. However, most existing approaches cannot faithfully reflect the visual contents due to insufficient image quality or semantic mismatches. Compared with reconstructing pixel-level visual images, speaking is a more efficient and effective way to explain visual information. Here we introduce a non-invasive neural decoder, termed as MindGPT, which interprets perceived visual stimuli into natural languages from fMRI signals. Specifically, our model builds upon a visually guided neural encoder with a cross-attention mechanism, which permits us to guide latent neural representations towards a desired language semantic direction in an end-to-end manner by the collaborative use of the large language model GPT. By doing so, we found that the neural representations of the MindGPT are explainable, which can be used to evaluate the contributions of visual properties to language semantics. Our experiments show that the generated word sequences truthfully represented the visual information (with essential details) conveyed in the seen stimuli. The results also suggested that with respect to language decoding tasks, the higher visual cortex (HVC) is more semantically informative than the lower visual cortex (LVC), and using only the HVC can recover most of the semantic information. The code of the MindGPT model will be publicly available at https://***/JxuanC/MindGPT. Copyright © 2023, The Authors. All rights reserved.

关键词： Decoding

Connecting Consistency Distillation to Score Distillation for Text-to-3D Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Zongrui Hu, Minghui Zheng, Qian Jiang, Xudong Lab. Nanyang Technological University Singapore School of Electrical and Electronic Engineering Nanyang Technological University Singapore College of Computer Science and Technology Zhejiang University China The State Key Lab of Brain-Machine Intelligence Zhejiang University China

Although recent advancements in text-to-3D generation have significantly improved generation quality, issues like limited level of detail and low fidelity still persist, which requires further improvement. To understand the essence of those issues, we thoroughly analyze current score distillation methods by connecting theories of consistency distillation to score distillation. Based on the insights acquired through analysis, we propose an optimization framework, Guided Consistency Sampling (GCS), integrated with 3D Gaussian Splatting (3DGS) to alleviate those issues. Additionally, we have observed the persistent oversaturation in the rendered views of generated 3D assets. From experiments, we find that it is caused by unwanted accumulated brightness in 3DGS during optimization. To mitigate this issue, we introduce a Brightness-Equalized Generation (BEG) scheme in 3DGS rendering. Experimental results demonstrate that our approach generates 3D assets with more details and higher fidelity than state-of-the-art methods. The codes are released at https://***/LMozart/ECCV2024-GCS-BEG. © 2024, CC BY.

关键词： Distillation

Evidentially Calibrated Source-Free Time-Series Domain Adaptation with Temporal Imputation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ragab, Mohamed Gong, Peiliang Eldele, Emadeldeen Zhang, Wenyu Wu, Min Foo, Chuan-Sheng Zhang, Daoqiang Li, Xiaoli Chen, Zhenghua Singapore The Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing211106 China Singapore138632 Singapore

Source-free domain adaptation (SFDA) aims to adapt a model pre-trained on a labeled source domain to an unlabeled target domain without access to source data, preserving the source domain’s privacy. While SFDA is prevalent in computer vision, it remains largely unexplored in time series analysis. Existing SFDA methods, designed for visual data, struggle to capture the inherent temporal dynamics of time series, hindering adaptation performance. This paper proposes MAsk And imPUte (MAPU), a novel and effective approach for time series SFDA. MAPU addresses the critical challenge of temporal consistency by introducing a novel temporal imputation task. This task involves randomly masking time series signals and leveraging a dedicated temporal imputer to recover the original signal within the learned embedding space, bypassing the complexities of noisy raw data. During subsequent adaptation, the imputer network guides the target model to generate target features that exhibit temporal consistency with the source features. Notably, MAPU is the first method to explicitly address temporal consistency in the context of time series SFDA. Additionally, it offers seamless integration with existing SFDA methods, providing greater flexibility. We further introduce E-MAPU, which incorporates evidential uncertainty estimation to address the overconfidence issue inherent in softmax predictions. To achieve that, we leverage evidential deep learning to obtain a better-calibrated pre-trained model and identify out-of-support target samples (those falling outside the source domain’s support) by predicting them with higher entropy than source samples. During adaptation, the target classifier remains fixed while the feature extractor is trained to minimize the evidential entropy of out-of-support target samples. This is achieved by adapting the target encoder to map these samples to a new feature representation closer to the source domain’s support. This fosters better alignment, ultimately

关键词： Entropy

Learning Inverse Dynamics of Multi-segmented Larval Crawling Neuromechanics

学校读者我要写书评

暂无评论

Learning Inverse Dynamics of Multi-segmented Larval Crawling...

Smart World Congress (SWC), IEEE

作者： Weihao Tang Hao Zhang Zhefeng Gong Nenggan Zheng Qiushi Academy for Advanced Studies College of Computer Science and Technology Zhejiang University Hangzhou China Department of neurology of the fourth Hospital School of Brain Science and Brain Medicine Zhejiang University School of Medicine Hangzhou China Qiushi Academy for Advanced Studies the State Key Lab of Brain-Machine Intelligence College of Computer Science and Technology Zhejiang University Hangzhou China Bengbu University Bengbu China

ISBN: (数字)9798331520861

ISBN: (纸本)9798331520878

Neuromechanical modeling that bridges neural activity and locomotion patterns is essential for studying locomotion control. The Drosophila larva, with its diverse softbodied locomotion, serves as an ideal subject for biomimetic research. Constructing a neuromechanical model of Drosophila larvae presents two primary challenges: efficiently simulating the larva’s soft, multi-segmented body and controlling locomotion within a large action space by actuating muscle groups. This work presents a three-dimensional neuromechanical model of the larva and introduces an inverse dynamics learning method for locomotion control. Our approach efficiently simulates the larva’s multi-segmented, deformable body by integrating computational modules for soft tissues, muscles, and friction. We employ a parameterized controller to reproduce peristaltic waves and use a hybrid artificial neural network architecture to learn inverse dynamics from partially observed states. Balancing biological interpretability with modeling complexity, this model advances the understanding of brain-body-environment interactions in Drosophila larvae, contributing to nature-inspired intelligent systems research.

关键词： Training Solid modeling Recurrent neural networks Biological system modeling Computational modeling Friction Muscles Aerospace electronics brain modeling Feature extraction

Make It Easy: Action Quality Assessment of Cyborg Animals Based on Spatial-Temporal Pose Inference

学校读者我要写书评

暂无评论

Make It Easy: Action Quality Assessment of Cyborg Animals Ba...

Smart World Congress (SWC), IEEE

作者： Qiqi Li Le Han Pengfei Wang Nenggan Zheng Qiushi Academy for Advanced Studies College of Computer Science and Technology Zhejiang University Hangzhou China School of Software Technology Zhejiang University Hangzhou China Qiushi Academy for Advanced Studies the State Key Lab of Brain-Machine Intelligence College of Computer Science and Technology Zhejiang University Hangzhou China Bengbu University Bengbu China

ISBN: (数字)9798331520861

ISBN: (纸本)9798331520878

Assessing the action quality of cyborg animals helps to adjust control strategies, guide the development of control algorithm, and enhance the efficiency of navigation and military applications. However, existing research is limited in control algorithm and ignores the assessment of movement smoothness. In this paper, we first propose EASY, an effective assessment framework, which is able to (i) evaluate the action quality of cyborg animals of various sizes and motion patterns; (ii) obtain assessment results directly based on keypoints. Specifically, EASY first captures keypoints sequences through a small object pose estimation module. It employs the convolutional block attention module (CBAM) and depthwise over parameterized convolution (DOConv), allowing more accurate feature recognition with fewer parameters. Then, EASY employs multi-field fusion strategy that merges commonalities and differences, enabling a comprehensive and accurate assessment of fluency. We tested EASY on both CyborgRat-MFA dataset and CyborgBee-MFA dataset. Extensive experiments demonstrate that the assessment results are consistent with human expert scores, which prove that our proposed EASY is reasonable, effective, and interpretable.

关键词： Accuracy Animals Navigation Human-machine systems Pose estimation Electrical stimulation Feature extraction Robustness Quality assessment Motion measurement

Neuromorphic model-based neural decoders for brain-computer interfaces: a comparative study

学校读者我要写书评

暂无评论

Neuromorphic model-based neural decoders for brain-computer ...

IEEE Biomedical Circuits and Systems (BIOCAS)

作者： Yijie Weng Yu Qi Yueming Wang Gang Pan College of Biomedical Engineering & Instrument Science Zhejiang University Hangzhou China MOE Frontier Science Center for Brain Science and Brain-Machine Integration Zhejiang University Hangzhou China College of Computer Science and Technology Zhejiang University Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China Qiushi Academy for Advanced Studies Zhejiang University Hangzhou China

ISBN: (数字)9798350354959

ISBN: (纸本)9798350354966

brain-computer interfaces that convert neural signals into control commands for external devices have demonstrated great potential in neural rehabilitation. Most recently, the growing demand for developing large-scale brain signal recording devices has highlighted the need for power-efficient neural signal processing. Neuromorphic computing models, implemented with chips, can provide ultra-low-cost computation of neural signals, facilitating brain-implantable devices. In our work, we evaluated the neural decoding ability of neuromorphic models in comparison with traditional neural decoding approaches. We found that neuromorphic model-based neural decoders showed high accuracy comparable to existing decoders while with significantly lower computational costs. Our results suggest the potential of using neuromorphic models for highly efficient computing with large-scale brain data.

关键词： Neuromorphic engineering Computational modeling Biological system modeling brain modeling brain-computer interfaces Data models Decoding Recording Computational efficiency Integrated circuit modeling