检索结果-内蒙古大学图书馆

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Küçükoğlu, Burcu Dalm, Sander van Gerven, Marcel Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Netherlands

The effectiveness of credit assignment in reinforcement learning (RL) when dealing with high-dimensional data is influenced by the success of representation learning via deep neural networks, and has implications for the sample efficiency of deep RL algorithms. Input decorrelation has been previously introduced as a method to speed up optimization in neural networks, and has proven impactful in both efficient deep learning and as a method for effective representation learning for deep RL algorithms. We propose a novel approach to online decorrelation in deep RL based on the decorrelated backpropagation algorithm that seamlessly integrates the decorrelation process into the RL training pipeline. Decorrelation matrices are added to each layer, which are updated using a separate decorrelation learning rule that minimizes the total decorrelation loss across all layers, in parallel to minimizing the usual RL loss. We used our approach in combination with the soft actor-critic (SAC) method, which we refer to as decorrelated soft actor-critic (DSAC). Experiments on the Atari 100k benchmark with DSAC shows, compared to the regular SAC baseline, faster training in five out of the seven games tested and improved reward performance in two games with around 50% reduction in wall-clock time, while maintaining performance levels on the other games. These results demonstrate the positive impact of network-wide decorrelation in deep RL for speeding up its sample efficiency through more effective credit assignment. © 2025, CC BY-NC-ND.

关键词： Deep reinforcement learning

PROBABILISTIC FORECASTING VIA AUTOREGRESSIVE FLOW MATCHING

学校读者我要写书评

暂无评论

arXiv 2025年

作者： El-Gazzar, Ahmed van Gerven, Marcel Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

In this work, we propose FLOWTIME, a generative model for probabilistic forecasting of multivariate timeseries data. Given historical measurements and optional future covariates, we formulate forecasting as sampling from a learned conditional distribution over future trajectories. Specifically, we decompose the joint distribution of future observations into a sequence of conditional densities, each modeled via a shared flow that transforms a simple base distribution into the next observation distribution, conditioned on observed covariates. To achieve this, we leverage the flow matching (FM) framework, enabling scalable and simulation-free learning of these transformations. By combining this factorization with the FM objective, FLOWTIME retains the benefits of autoregressive models—including strong extrapolation performance, compact model size, and well-calibrated uncertainty estimates—while also capturing complex multi-modal conditional distributions, as seen in modern transport-based generative models. We demonstrate the effectiveness of FLOWTIME on multiple dynamical systems and real-world forecasting tasks. © 2025, CC BY.

关键词： Generative adversarial networks

Noise-based reward-modulated learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fernández, Jesús García Ahmad, Nasir van Gerven, Marcel Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

Recent advances in reinforcement learning (RL) have led to significant improvements in task performance. However, training neural networks in an RL regime is typically achieved in combination with backpropagation, limiting their applicability in resource-constrained environments or when using non-differentiable neural networks. While noise-based alternatives like reward-modulated Hebbian learning (RMHL) have been proposed, their performance has remained limited, especially in scenarios with delayed rewards, which require retrospective credit assignment over time. Here, we derive a novel noise-based learning rule that addresses these challenges. Our approach combines directional derivative theory with Hebbian-like updates to enable efficient, gradient-free learning in RL. It features stochastic noisy neurons which can approximate gradients, and produces local synaptic updates modulated by a global reward signal. Drawing on concepts from neuroscience, our method uses reward prediction error as its optimization target to generate increasingly advantageous behavior, and incorporates an eligibility trace to facilitate temporal credit assignment in environments with delayed rewards. Its formulation relies on local information alone, making it compatible with implementations in neuromorphic hardware. Experimental validation shows that our approach significantly outperforms RMHL and is competitive with BP-based baselines, highlighting the promise of noise-based, biologically inspired learning for low-power and real-time applications. © 2025, CC BY.

关键词：

Kozax: Flexible and Scalable Genetic Programming in JAX

学校读者我要写书评

暂无评论

arXiv 2025年

作者： de Vries, Sigur Keemink, Sander W. van Gerven, Marcel A.J. Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

Genetic programming is an optimization algorithm inspired by natural selection which automatically evolves the structure of computer programs. The resulting computer programs are interpretable and efficient compared to black-box models with fixed structure. The fitness evaluation in genetic programming suffers from high computational requirements, limiting the performance on difficult problems. To reduce the runtime, many implementations of genetic programming require a specific data format, making the applicability limited to specific problem classes. Consequently, there is no efficient genetic programming framework that is usable for a wide range of tasks. To this end, we developed Kozax, a genetic programming framework that evolves symbolic expressions for arbitrary problems. We implemented Kozax using JAX, a framework for high-performance and scalable machine learning, which allows the fitness evaluation to scale efficiently to large populations or datasets on GPU. Furthermore, Kozax offers constant optimization, custom operator definition and simultaneous evolution of multiple trees. We demonstrate successful applications of Kozax to discover equations of natural laws, recover equations of hidden dynamic variables and evolve a control policy. Overall, Kozax provides a general, fast, and scalable library to optimize white-box solutions in the realm of scientific computing. Copyright © 2025, The Authors. All rights reserved.

关键词： Genetic programming

SUBSPACE NODE PRUNING

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Offergeld, Joshua van Gerven, Marcel Ahmad, Nasir Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

Efficiency of neural network inference is undeniably important in a time where commercial use of AI models increases daily. Node pruning is the art of removing computational units such as neurons, filters, attention heads, or even entire layers to significantly reduce inference time while retaining network performance. In this work, we propose the projection of unit activations to an orthogonal subspace in which there is no redundant activity and within which we may prune nodes while simultaneously recovering the impact of lost units via linear least squares. We identify that, for effective node pruning, this subspace must be constructed using a triangular transformation matrix, a transformation which is equivalent to and unnormalized Gram-Schmidt orthogonalization. We furthermore show that the order in which units are orthogonalized can be optimised to maximally reduce node activations in our subspace and thereby form a more optimal ranking of nodes. Finally, we leverage these orthogonal subspaces to automatically determine layer-wise pruning ratios based upon the relative scale of node activations in our subspace, equivalent to cumulative variance. Our proposed method reaches state of the art when pruning ImageNet trained VGG-16 and rivals more complex state of the art methods when pruning ResNet-50 networks across a range of pruning ratios. Copyright © 2024, The Authors. All rights reserved.

关键词： Complex networks

Gradient-Free Training of Recurrent neural Networks using Random Perturbations

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fernández, Jesús García Keemink, Sander van Gerven, Marcel Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

Recurrent neural networks (RNNs) hold immense potential for computations due to their Turing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle to propagate gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. We subsequently conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scalability when compared to BPTT, strongly outperforming standard node perturbation and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs which can be ideally suited for neuromorphic computing applications. © 2024, CC BY.

关键词： Recurrent neural networks

Modeling and Detection of Critical Slowing Down in Epileptic Dynamics

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qin, Yuzhen van Gerven, Marcel The Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

Epilepsy is a common neurological disorder characterized by abrupt seizures. Although seizures may appear random, they are often preceded by early warning signs in neural signals, notably, critical slowing down, a phenomenon in which the system’s recovery rate from perturbations declines when it approaches a critical point. Detecting these markers could enable preventive therapies. This paper introduces a multi-stable slow-fast system to capture critical slowing down in epileptic dynamics. We construct regions of attraction for stable states, shedding light on how dynamic bifurcations drive pathological oscillations. We derive the recovery rate after perturbations to formalize critical slowing down. A novel algorithm for detecting precursors to ictal transitions is presented, along with a proof-of-concept event-based feedback control strategy to prevent impending pathological oscillations. Numerical studies are conducted to validate our theoretical findings. © 2024, CC BY-NC-ND.

关键词： Feedback control

Discovering Continuous-Time Memory-Based Symbolic Policies using Genetic Programming

学校读者我要写书评

暂无评论

arXiv 2024年

作者： de Vries, Sigur Keemink, Sander van Gerven, Marcel Department of Machine Learning and Neural Computing Donders Institute for Brain Cognition and Behaviour Radboud University Nijmegen Netherlands

Artificial intelligence techniques are increasingly being applied to solve control problems, but often rely on black-box methods without transparent output generation. To improve the interpretability and transparency in control systems, models can be defined as white-box symbolic policies described by mathematical expressions. For better performance in partially observable and volatile environments, the symbolic policies are extended with memory represented by continuous-time latent variables, governed by differential equations. Genetic programming is used for optimisation, resulting in interpretable policies consisting of symbolic expressions. Our results show that symbolic policies with memory compare with black-box policies on a variety of control tasks. Furthermore, the benefit of the memory in symbolic policies is demonstrated on experiments where memory-less policies fall short. Overall, we present a method for evolving high-performing symbolic policies that offer interpretability and transparency, which lacks in black-box models. Copyright © 2024, The Authors. All rights reserved.

关键词： Genetic programming

Ornstein-Uhlenbeck Adaptation as a Mechanism for learning in Brains and machines

学校读者我要写书评

暂无评论

arXiv 2024年

learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex information flow makes its implementation in biological and neuromorphic systems challenging. This has motivated the exploration of alternative learning mechanisms that can operate locally and do not rely on exact gradients. In this work, we introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. Using an Ornstein-Uhlenbeck process with adaptive dynamics, our method balances exploration and exploitation during learning, driven by deviations from error predictions, akin to reward prediction error. Operating in continuous time, Ornstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning in dynamic, time-evolving environments. We validate our approach across a range of different tasks, including supervised learning and reinforcement learning in feedforward and recurrent systems. Additionally, we demonstrate that it can perform meta-learning, adjusting hyper-parameters autonomously. Our results indicate that OUA provides a viable alternative to traditional gradient-based methods, with potential applications in neuromorphic computing. It also hints at a possible mechanism for noise-driven learning in the brain, where stochastic neurotransmitter release may guide synaptic adjustments. © 2024, CC BY.

关键词： Self-supervised learning