检索结果-内蒙古大学图书馆

Pathformer3D: A 3D Scanpath Transformer for 360° Images

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Quan, Rong Lai, Yantao Qiu, Mengyu Liang, Dong College of Artificial Intelligence Nanjing University of Aeronautics and Astronautics The Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education Nanjing211106 China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing China Shenzhen Research Institute Nanjing University of Aeronautics and Astronautics Shenzhen China

Scanpath prediction in 360° images can help realize rapid rendering and better user interaction in Virtual/Augmented Reality applications. However, existing scanpath prediction models for 360° images execute scanpath prediction on 2D equirectangular projection plane, which always result in big computation error owing to the 2D plane’s distortion and coordinate discontinuity. In this work, we perform scanpath prediction for 360° images in 3D spherical coordinate system and proposed a novel 3D scanpath Transformer named Pathformer3D. Specifically, a 3D Transformer encoder is first used to extract 3D contextual feature representation for the 360° image. Then, the contextual feature representation and historical fixation information are input into a Transformer decoder to output current time step’s fixation embedding, where the self-attention module is used to imitate the visual working memory mechanism of human visual system and directly model the time dependencies among the fixations. Finally, a 3D Gaussian distribution is learned from each fixation embedding, from which the fixation position can be sampled. Evaluation on four panoramic eye-tracking datasets demonstrates that Pathformer3D outperforms the current state-of-the-art methods. Code is available at https://***/lsztzp/ Pathformer3D. Copyright © 2024, The Authors. All rights reserved.

关键词： Forecasting

Modeling Larval Tumbling Locomotion Based on Differentiable Material Point Method

学校读者我要写书评

暂无评论

Modeling Larval Tumbling Locomotion Based on Differentiable ...

Smart World Congress (SWC), IEEE

作者： Weishi Wang Weihao Tang Yue Tang Nenggan Zheng Qiushi Academy for Advanced Studies School of Software Technology Zhejiang University Ningbo China Qiushi Academy for Advanced Studies College of Computer Science and Technology Zhejiang University Hangzhou China Qiushi Academy for Advanced Studies the State Key Lab of Brain-Machine Intelligence College of Computer Science and Technology Zhejiang University Hangzhou China Bengbu University Bengbu China

ISBN: (数字)9798331520861

ISBN: (纸本)9798331520878

Tumbling locomotion enables Drosophila larvae to rapidly reposition themselves to evade predators. Owing to the complex spatial postural changes involved, studies on this phenomenon are relatively scarce. Neuromechanical models can simulate sequences of biological locomotion postures and synthesize long-term multimodal movement behaviors, facilitating the analysis of the mechanisms linking the nervous system and locomotion. However, constructing neuromechanical models of larval tumbling presents significant challenges due to the extensive deformations resulting from tumbling. Here, we developed a model based on the material point method (MPM) that reflects the biological structure of Drosophila larvae, controlling muscle contractions in response to received signals. By integrating deep learning, we created an artificial neural network-based controller to generate these signals. We built this framework using differentiable physics and leveraged efficient gradient calculations for gradient-based optimization of the neural network controller. Our framework supports both straight and offset tumbling and achieves closed-loop control of a complex 3D soft-bodied neuromechanical model. Experimental results demonstrate that the optimized neural network controller enables the larval model to perform tumbling actions effectively, providing new insights into understanding and simulating larval locomotion behavior.

关键词： Deep learning Training Adaptation models Three-dimensional displays Biological system modeling Computational modeling Optimization methods Artificial neural networks Muscles Biological neural networks

Forget less,count better:a domain-incremental self-distillation learning benchmark for lifelong crowd counting

学校读者我要写书评

暂无评论

Frontiers of Information technology & Electronic Engineering 2023年第2期24卷 187-202页

作者： Jiaqi GAO Jingqi LI Hongming SHAN Yanyun QU James ZWANG Fei-Yue WANG Junping ZHANG Shanghai Key Laboratory of Intelligent Information Processing School of Computer ScienceFudan UniversityShanghai 200433China Institute of Science and Technology for Brain-inspired Intelligence Fudan UniversityShanghai 200433China Shanghai Center for Brain Science and Brain-inspired Technology Shanghai 201210China School of Information Science and Technology Xiamen UniversityXiamen 361005China College of Information Sciences and Technology the Pennsylvania State UniversityUniversity ParkPA 16802USA State Key Laboratory of Management and Control for Complex Systems Institute of AutomationChinese Academy of SciencesBeijing 100190China

Crowd counting has important applications in public safety and pandemic control.A robust and practical crowd counting system has to be capable of continuously learning with the newly incoming domain data in real-world scenarios instead of fitting one domain ***-the-shelf methods have some drawbacks when handling multiple domains:(1)the models will achieve limited performance(even drop dramatically)among old domains after training images from new domains due to the discrepancies in intrinsic data distributions from various domains,which is called catastrophic forgetting;(2)the well-trained model in a specific domain achieves imperfect performance among other unseen domains because of domain shift;(3)it leads to linearly increasing storage overhead,either mixing all the data for training or simply training dozens of separate models for different domains when new ones are *** overcome these issues,we investigate a new crowd counting task in incremental domain training setting called lifelong crowd *** goal is to alleviate catastrophic forgetting and improve the generalization ability using a single model updated by the incremental ***,we propose a self-distillation learning framework as a benchmark(forget less,count better,or FLCB)for lifelong crowd counting,which helps the model leverage previous meaningful knowledge in a sustainable manner for better crowd counting to mitigate the forgetting when new data arrive.A new quantitative metric,normalized Backward Transfer(nBwT),is developed to evaluate the forgetting degree of the model in the lifelong learning *** experimental results demonstrate the superiority of our proposed benchmark in achieving a low catastrophic forgetting degree and strong generalization ability.

关键词： Crowd counting Knowledge distillation Lifelong learning

Online neural sequence detection with hierarchical dirichlet point process 22

学校读者我要写书评

暂无评论

Online neural sequence detection with hierarchical dirichlet...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Weihan Li Yu Qi Gang Pan College of Computer Science and Technology Zhejiang University Hangzhou China and The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China The Affiliated Mental Health Center & Hangzhou Seventh People's Hospital the MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University School of Medicine Hangzhou China and The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China and College of Computer Science and Technology Zhejiang University Hangzhou China

ISBN: (纸本)9781713871088

Neural sequence detection plays a vital role in neuroscience research. Recent impressive works utilize convolutive nonnegative matrix factorization and NeymanScott process to solve this problem. However, they still face two limitations. Firstly, they accommodate the entire dataset into memory and perform iterative updates of multiple passes, which can be inefficient when the dataset is large or grows frequently. Secondly, they rely on the prior knowledge of the number of sequence types, which can be impractical with data when the future situation is unknown. To tackle these limitations, we propose a hierarchical Dirichlet point process model for efficient neural sequence detection. Instead of computing the entire data, our model can sequentially detect sequences in an online unsupervised manner with Particle filters. Besides, the Dirichlet prior enables our model to automatically introduce new sequence types on the fly as needed, thus avoiding specifying the number of types in advance. We manifest these advantages on synthetic data and neural recordings from songbird higher vocal center and rodent hippocampus.

关键词：

A Battery-Free Wireless Temperature Sensing Chip for Food Monitoring

学校读者我要写书评

暂无评论

A Battery-Free Wireless Temperature Sensing Chip for Food Mo...

IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA)

作者： Jiayi Wang Haoyang Li Weixiao Wang Tianying Fang Jiaqing Li Yuxuan Luo Bo Zhao College of Integrated Circuits Zhejiang University Hangzhou China Nanhu Brian-Computer Interface Institute Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China

ISBN: (数字)9798331530723

ISBN: (纸本)9798331530730

This work presents a battery-free wireless temperature sensing chip for long-termly monitoring the food production environment. A calibrated oscillator-based CMOS temperature sensor is proposed instead of the ADC-based power-hungry circuits in conventional works. In addition, the sensor chip can harvest the power transferred by a remote reader to eliminate the use of battery. Meanwhile, the system conduct wireless bidirectional communication between the sensor chip and reader. In this way, the temperature sensing chip is fabricated in 55 nm CMOS process, and the reader chip is implemented in 65 nm CMOS technology. Experimental results show that the temperature measurement error achieves $\pm 1.6^{\circ} \mathrm{C}$ from $25^{\circ} \mathrm{C}$ to $50^{\circ} \mathrm{C}$, with battery-free readout by a remote reader.

关键词： Temperature sensors Temperature measurement Wireless communication Wireless sensor networks Food manufacturing Circuits Bidirectional control CMOS process Sensors Monitoring

Neuromorphic Auditory Perception by Neural Spiketrum

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Tang, Huajin Gu, Pengjie Wijekoon, Jayawan Alsakkal, M.H.D. Anas Wang, Ziming Shen, Jiangrong Yan, Rui The State Key Lab of Brain-Machine Intelligence College of Computer Science Zhejiang University Hangzhou310027 China The College of Computer Science Zhejiang University Hangzhou310027 China The Department of Electrical & Electronic Engineering University of Manchester United Kingdom The College of Computer Science Zhejiang University of Technology Hangzhou310014 China

Neuromorphic computing holds the promise to achieve the energy efficiency and robust learning performance of biological neural systems. To realize the promised brain-like intelligence, it needs to solve the challenges of the neuromorphic hardware architecture design of biological neural substrate and the hardware amicable algorithms with spike-based encoding and learning. Here we introduce a neural spike coding model termed spiketrum, to characterize and transform the time-varying analog signals, typically auditory signals, into computationally efficient spatiotemporal spike patterns. It minimizes the information loss occurring at the analog-to-spike transformation and possesses informational robustness to neural fluctuations and spike losses. The model provides a sparse and efficient coding scheme with precisely controllable spike rate that facilitates training of spiking neural networks in various auditory perception tasks. We further investigate the algorithm-hardware co-designs through a neuromorphic cochlear prototype which demonstrates that our approach can provide a systematic solution for spike-based artificial intelligence by fully exploiting its advantages with spike-based computation. Copyright © 2023, The Authors. All rights reserved.

关键词： Neural networks

Spiking Neural Networks for Temporal Processing: Status Quo and Future Prospects

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ma, Chenxiang Chen, Xinyi Li, Yanchen Yang, Qu Wu, Yujie Li, Guoqi Pan, Gang Tang, Huajin Tan, Kay Chen Wu, Jibin Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University Hong Kong Department of Computing The Hong Kong Polytechnic University Hong Kong Department of Electrical and Computer Engineering National University of Singapore 119077 Singapore Institute of Automation Chinese Academy of Sciences Beijing100045 China State Key Laboratory of Brain-Machine Intelligence College of Computer Science and Technology MOE Frontier Science Center for Brain Science and Brain-Machine Integration Zhejiang University Hangzhou310027 China

Temporal processing is fundamental for both biological and artificial intelligence systems, as it enables the comprehension of dynamic environments and facilitates timely responses. Spiking Neural Networks (SNNs) excel in handling such data with high efficiency, owing to their rich neuronal dynamics and sparse activity patterns. Given the recent surge in the development of SNNs, there is an urgent need for a comprehensive evaluation of their temporal processing capabilities. In this paper, we first conduct an in-depth assessment of commonly used neuromorphic benchmarks, revealing critical limitations in their ability to evaluate the temporal processing capabilities of SNNs. To bridge this gap, we further introduce a benchmark suite consisting of three temporal processing tasks characterized by rich temporal dynamics across multiple timescales. Utilizing this benchmark suite, we perform a thorough evaluation of recently introduced SNN approaches to elucidate the current status of SNNs in temporal processing. Our findings indicate significant advancements in recently developed spiking neuron models and neural architectures regarding their temporal processing capabilities, while also highlighting a performance gap in handling long-range dependencies when compared to state-of-the-art non-spiking models. Finally, we discuss the key challenges and outline potential avenues for future research. Copyright © 2025, The Authors. All rights reserved.

关键词： Spatio-temporal data

SpikingYOLOX: Improved YOLOX Object Detection with Fast Fourier Convolution and Spiking Neural Networks 39

学校读者我要写书评

暂无评论

SpikingYOLOX: Improved YOLOX Object Detection with Fast Four...

39th Annual AAAI Conference on Artificial intelligence, AAAI 2025

作者： Miao, Wei Shen, Jiangrong Xu, Qi Hamalainen, Timo Xu, Yi Cong, Fengyu School of Computer Science and Technology Dalian University of Technology China Faculty of Information Technology University of Jyväskylä Finland School of Computer Science and Technology Xi'an Jiaotong University China State Key Lab of Brain-Machine Intelligence Zhejiang University China School of Control Science and Engineering Dalian University of Technology China School of Biomedical Engineering Dalian University of Technology China

ISBN: (纸本)157735897X

In recent years, with the advancements in brain science, spiking neural networks (SNNs) have garnered significant attention. SNNs can generate spikes that mimic the function of neurons transmission in humans brain, thereby significantly reducing computational costs by the event-driven nature during training. While deep SNNs have shown impressive performance on classification tasks, they still face challenges in more complex tasks such as object detection. In this paper, we propose SpikingYOLOX, extending the structure of the original YOLOX by introducing signed spiking neurons and fast Fourier convolution (FFC). The designed ternary signed spiking neurons could generate three kinds of spikes to obtain more robust features in the deep layer of the backbone. Meanwhile, we integrate FFC with SNN modules to enhance object detection performance, because its global receptive field is beneficial to the object detection task. Extensive experiments demonstrate that the proposed SpikingYOLOX achieves state-of-the-art performance among other SNN-based object detection methods. Copyright © 2025, Association for the Advancement of Artificial intelligence (***). All rights reserved.

关键词： Deep neural networks

CBRAMOD: A CRISS-CROSS brain FOUNDATION MODEL FOR EEG DECODING

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jiquan Zhao, Sha Luo, Zhiling Zhou, Yangxuan Jiang, Haiteng Li, Shijian Li, Tao Pan, Gang State Key Laboratory of Brain-machine Intelligence Zhejiang University China College of Computer Science and Technology Zhejiang University China Alibaba Group China Department of Neurobiology Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital Zhejiang University School of Medicine China MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University China

Electroencephalography (EEG) is a non-invasive technique to measure and record brain electrical activity, widely used in various BCI and healthcare applications. Early EEG decoding methods rely on supervised learning, limited by specific tasks and datasets, hindering model performance and generalizability. With the success of large language models, there is a growing body of studies focusing on EEG foundation models. However, these studies still leave challenges: Firstly, most of existing EEG foundation models employ full EEG modeling strategy. It models the spatial and temporal dependencies between all EEG patches together, but ignores that the spatial and temporal dependencies are heterogeneous due to the unique structural characteristics of EEG signals. Secondly, existing EEG foundation models have limited generalizability on a wide range of downstream BCI tasks due to varying formats of EEG data, making it challenging to adapt to. To address these challenges, we propose a novel foundation model called CBraMod. Specifically, we devise a criss-cross transformer as the backbone to thoroughly leverage the structural characteristics of EEG signals, which can model spatial and temporal dependencies separately through two parallel attention mechanisms. And we utilize an asymmetric conditional positional encoding scheme which can encode positional information of EEG patches and be easily adapted to the EEG with diverse formats. CBraMod is pre-trained on a very large corpus of EEG through patch-based masked EEG reconstruction. We evaluate CBraMod on up to 10 downstream BCI tasks (12 public datasets). CBraMod achieves the state-of-the-art performance across the wide range of tasks, proving its strong capability and generalizability. The source code is publicly available at https://***/wjq-learning/CBraMod. Copyright © 2024, The Authors. All rights reserved.

关键词： Noninvasive medical procedures