检索结果-内蒙古大学图书馆

Context Gating in Spiking Neural Networks: Achieving Lifelong Learning through Integration of Local and Global Plasticity

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Shen, Jiangrong Ni, Wenyao Xu, Qi Pan, Gang Tang, Huajin The State Key Lab of Brain Machine Intelligence College of Computer Science and Technology Zhejiang University Zhejiang310027 China The Faculty of Electronic Information and Electrical Engineering School of Artificial Intelligence Dalian University of Technology Dalian116024 China

Humans learn multiple tasks in succession with minimal mutual interference, through the context gating mechanism in the prefrontal cortex (PFC). The brain-inspired models of spiking neural networks (SNN) have drawn massive attention for their energy efficiency and biological plausibility. To overcome catastrophic forgetting when learning multiple tasks in sequence, current SNN models for lifelong learning focus on memory reserving or regularization-based modification, while lacking SNN to replicate human experimental behavior. Inspired by biological context-dependent gating mechanisms found in PFC, we propose SNN with context gating trained by the local plasticity rule (CG-SNN) for lifelong learning. The iterative training between global and local plasticity for task units is designed to strengthen the connections between task neurons and hidden neurons and preserve the multi-task relevant information. The experiments show that the proposed model is effective in maintaining the past learning experience and has better task-selectivity than other methods during lifelong learning. Our results provide new insights that the CG-SNN model can extend context gating with good scalability on different SNN architectures with different spike-firing mechanisms. Thus, our models have good potential for parallel implementation on neuromorphic hardware and model human's behavior. Copyright © 2024, The Authors. All rights reserved.

关键词： Iterative methods

Temporal Spiking Generative Adversarial Networks for Heading Direction Decoding

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Shen, Jiangrong Wang, Kejun Gao, Wei Liu, Jian K. Xu, Qi Pan, Gang Chen, Xiaodong Tang, Huajin College of Computer Science and Technology Zhejiang University China State Key Lab of Brain-Machine Intelligence Zhejiang University China College of Biomedical Engineering and Instrument Science Zhejiang University China School of Computing China School of Computer Science and Technology Dalian University of Technology China

The spike-based neuronal responses within the ventral intraparietal area (VIP) exhibit intricate spatial and temporal dynamics in the posterior parietal cortex, presenting decoding challenges such as limited data availability at the biological population level. The practical difficulty in collecting VIP neuronal response data hinders the application of sophisticated decoding models. To address this challenge, we propose a unified spike-based decoding framework leveraging spiking neural networks (SNNs) for both generative and decoding purposes, for their energy efficiency and suitability for neural decoding tasks. We propose the Temporal Spiking Generative Adversarial Networks (T-SGAN), a model based on a spiking transformer, to generate synthetic time-series data reflecting the neuronal response of VIP neurons. T-SGAN incorporates temporal segmentation to reduce the temporal dimension length, while spatial self-attention facilitates the extraction of associated information among VIP neurons. This is followed by recurrent SNNs decoder equipped with an attention mechanism, designed to capture the intricate spatial and temporal dynamics for heading direction decoding. Experimental evaluations conducted on biological datasets from monkeys showcase the effectiveness of the proposed framework. Results indicate that T-SGAN successfully generates realistic synthetic data, leading to a significant improvement of up to 1.75% in decoding accuracy for recurrent SNNs. Furthermore, the SNN-based decoding framework capitalizes on the low power consumption advantages, offering substantial benefits for neuronal response decoding applications. © 2024, The Authors. All rights reserved.

关键词： Decoding

Latent Processes Identification From Multi-View Time Series

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Zenan Wang, Haobo Zhao, Junbo Zheng, Nenggan Zhejiang University China The State Key Lab of Brain-Machine Intelligence Zhejiang University China China College of Computer Science and Technology Zhejiang University China

Understanding the dynamics of time series data typically requires identifying the unique latent factors for data generation, a.k.a., latent processes identification. Driven by the independent assumption, existing works have made great progress in handling single-view data. However, it is a nontrivial problem that extends them to multi-view time series data because of two main challenges: (i) the complex data structure, such as temporal dependency, can result in violation of the independent assumption;(ii) the factors from different views are generally overlapped and are hard to be aggregated to a complete set. In this work, we propose a novel framework MuLTI that employs the contrastive learning technique to invert the data generative process for enhanced identifiability. Additionally, MuLTI integrates a permutation mechanism that merges corresponding overlapped variables by the establishment of an optimal transport formula. Extensive experimental results on synthetic and real-world datasets demonstrate the superiority of our method in recovering identifiable latent variables on multi-view time series. Copyright © 2023, The Authors. All rights reserved.

关键词： Time series

Extnco: A Fine-Grained Divide-and-Conquer Approach for Extending Nco to Solve Large-Scale Traveling Salesman Problem

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Chen, Xinwei Li, Yurui Yang, Yifan Zhang, Li Li, Shijian Pan, Gang College of Computer Science and Technology Zhejiang University Hangzhou310027 China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou310027 China

Large-scale Traveling Salesman Problem (TSP) applications are common and important in practice. Unfortunately, the state-of-the-art heuristic solver LKH suffers from exponential time usage. Neural Combinatorial Optimization (NCO) has emerged as an efficient alternative to LKH, but it struggles to handle large-scale instances with zero-shot generalization. Divide-and-conquer is a classical paradigm for handling large-scale problems. However, integrating these two approaches is challenging due to the nearly $O(N^2)$ time complexity of dividing strategy and merging strategy that relies on pre-defined rules. In this paper, we present ExtNCO, a fine-grained divide-and-conquer approach for extending NCO to solve large-scale TSP. In the dividing procedure, we propose LocKMeans algorithm with $O(N)$ time complexity to construct small-scale sub-problems based on the density distribution. The solving procedure imposes minimal constraints on the solver, allowing ExtNCO to serve as a plug-and-play tool for general NCO approaches. We propose MST-baed merging strategy that determines the optimal merging combination with the given the operator, and the solution quality can be further improved by incorporating the K-opt heuristic into the operator. Experimental results demonstrate that our strategy outperforms existing methods, especially in the fine-grained division scenario. Additionally, it is highly scalable over instances ranging from 1K to 1M nodes. When employing POMO (pre-trained on TSP-100) as the solver, ExtNCO surpasses the SOTA baseline H-TSP by a significant margin, yielding the best result for large-scale TSP within the NCO landscape. © 2023, The Authors. All rights reserved.

关键词： Combinatorial optimization

Cauchy Diffusion: A Heavy-tailed Denoising Diffusion Probabilistic Model for Speech Synthesis 39

学校读者我要写书评

暂无评论

Cauchy Diffusion: A Heavy-tailed Denoising Diffusion Probabi...

39th Annual AAAI Conference on Artificial intelligence, AAAI 2025

作者： Lian, Qi Qi, Yu Wang, Yueming The College of Computer Science and Technology Zhejiang University China MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University China Affiliated Mental Health Center Hangzhou Seventh People's Hospital Zhejiang University China State Key Lab of Brain-Machine Intelligence Zhejiang University China

ISBN: (纸本)157735897X

Denoising diffusion probabilistic models (DDPMs) have gained popularity in devising neural vocoders and obtained outstanding performance. However, existing DDPM-based neural vocoders struggle to handle the prosody diversities due to their susceptibility to mode-collapse issues confronted with imbalanced data. We introduced Cauchy Diffusion, a model incorporating the Cauchy noises to address this challenge. The heavy-tailed Cauchy distribution exhibits better resilience to imbalanced speech data, potentially improving prosody modeling. Our experiments on the LJSpeech and VCTK datasets demonstrate that Cauchy Diffusion achieved state-of-the-art speech synthesis performance. Compared to existing neural vocoders, our Cauchy Diffusion notably improved speech diversity while maintaining superior speech quality. Remarkably, Cauchy Diffusion surpassed neural vocoders based on generative adversarial networks (GANs) that are explicitly optimized to improve diversity. Copyright © 2025, Association for the Advancement of Artificial intelligence (***). All rights reserved.

关键词： Speech enhancement

DeCorrNet: Enhancing Neural Decoding Performance by Eliminating Correlations in Noise 39

学校读者我要写书评

暂无评论

DeCorrNet: Enhancing Neural Decoding Performance by Eliminat...

39th Annual AAAI Conference on Artificial intelligence, AAAI 2025

作者： Tan, Xianhan Qi, Yu Wang, Yueming The College of Computer Science and Technology Zhejiang University China MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University China Affiliated Mental Health Center Hangzhou Seventh People’s Hospital Zhejiang University China State Key Lab of Brain-Machine Intelligence Zhejiang University China

ISBN: (纸本)157735897X

Neural decoding, which transforms neural signals into motor commands, plays a key role in brain-computer interfaces (BCIs). Existing neural decoding approaches mainly rely on the assumption of independent noises, which could perform poorly in case the assumption is invalid. However, correlations in noises have been commonly observed in neural signals. Specifically, noise in different neural channels can be similar or highly related, which could degrade the performance of those neural decoders. To tackle this problem, we propose the DeCorrNet, which explicitly removes noise correlation in neural decoding. DeCorrNet could incorporate diverse neural decoders as an ensemble module to enhance the neural decoding performance. Experiments with benchmark BCI datasets demonstrated the superiority of DeCorrNet and achieved state-of-the-art results. Copyright © 2025, Association for the Advancement of Artificial intelligence (***). All rights reserved.

关键词：

Off-OAB: Off-Policy Policy Gradient Method with Optimal Action-Dependent Baseline

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Meng, Wenjia Zheng, Qian Yang, Long Yin, Yilong Pan, Gang The School of Software Shandong University Jinan250000 China The School of Artificial Intelligence Peking University China The State Key Lab of Brain-Machine Intelligence College of Computer Science and Technology Zhejiang University Hangzhou310000 China

Policy-based methods have achieved remarkable success in solving challenging reinforcement learning problems. Among these methods, off-policy policy gradient methods are particularly important due to that they can benefit from off-policy data. However, these methods suffer from the high variance of the off-policy policy gradient (OPPG) estimator, which results in poor sample efficiency during training. In this paper, we propose an off-policy policy gradient method with the optimal action-dependent baseline (Off-OAB) to mitigate this variance issue. Specifically, this baseline maintains the OPPG estimator's unbiasedness while theoretically minimizing its variance. To enhance practical computational efficiency, we design an approximated version of this optimal baseline. Utilizing this approximation, our method (Off-OAB) aims to decrease the OPPG estimator's variance during policy optimization. We evaluate the proposed Off-OAB method on six representative tasks from OpenAI Gym and MuJoCo, where it demonstrably surpasses state-of-the-art methods on the majority of these tasks. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

MITIGATING REWARD OVER-OPTIMIZATION IN RLHF VIA BEHAVIOR-SUPPORTED REGULARIZATION

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Dai, Juntao Chen, Taiye Yang, Yaodong Zheng, Qian Pan, Gang College of Computer Science and Technology Zhejiang University China The State Key Lab of Brain-Machine Intelligence Zhejiang University China LLM Safety Centre Beijing Academy of Artificial Intelligence China Center for AI Safety and Governance Peking University China

Reinforcement learning from human feedback (RLHF) is an effective method for aligning large language models (LLMs) with human values. However, reward over-optimization remains an open challenge leading to discrepancies between the performance of LLMs under the reward model and the true human objectives. A primary contributor to reward over-optimization is the extrapolation error that arises when the reward model evaluates out-of-distribution (OOD) responses. However, current methods still fail to prevent the increasing frequency of OOD response generation during the reinforcement learning (RL) process and are not effective at handling extrapolation errors from OOD responses. In this work, we propose the Behavior-Supported Policy Optimization (BSPO) method to mitigate the reward over-optimization issue. Specifically, we define behavior policy as the next token distribution of the reward training dataset to model the in-distribution (ID) region of the reward model. Building on this, we introduce the behavior-supported Bellman operator to regularize the value function, penalizing all OOD values without impacting the ID ones. Consequently, BSPO reduces the generation of OOD responses during the RL process, thereby avoiding overestimation caused by the reward model’s extrapolation errors. Theoretically, we prove that BSPO guarantees a monotonic improvement of the supported policy until convergence to the optimal behavior-supported policy. Empirical results from extensive experiments show that BSPO outperforms baselines in preventing reward overoptimization due to OOD evaluation and finding the optimal ID policy. Copyright © 2025, The Authors. All rights reserved.

关键词： Extrapolation

Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Dai, Juntao Yang, Yaodong Zheng, Qian Pan, Gang College of Computer Science and Technology Zhejiang University Hangzhou China The State Key Lab of Brain-Machine Intelligence Zhejiang University Hangzhou China Center for AI Safety and Governance Peking University Beijing China

A key aspect of Safe Reinforcement Learning (Safe RL) involves estimating the constraint condition for the next policy, which is crucial for guiding the optimization of safe policy updates. However, the existing Advantage-based Estimation (ABE) method relies on the infinite-horizon discounted advantage function. This dependence leads to catastrophic errors in finite-horizon scenarios with non-discounted constraints, resulting in safety-violation updates. In response, we propose the first estimation method for finite-horizon non-discounted constraints in deep Safe RL, termed Gradient-based Estimation (GBE), which relies on the analytic gradient derived along trajectories. Our theoretical and empirical analyses demonstrate that GBE can effectively estimate constraint changes over a finite horizon. Constructing a surrogate optimization problem with GBE, we developed a novel Safe RL algorithm called Constrained Gradient-based Policy Optimization (CGPO). CGPO identifies feasible optimal policies by iteratively resolving sub-problems within trust regions. Our empirical results reveal that CGPO, unlike baseline algorithms, successfully estimates the constraint functions of subsequent policies, thereby ensuring the efficiency and feasibility of each update. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep reinforcement learning