Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take t...
ISBN:
(纸本)9781713829546
Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take the first steps in making use of this connection. As Koopman operator theory is a linear theory, a successful implementation of it in evolving network weights and biases offers the promise of accelerated training, especially in the context of deep networks, where optimization is inherently a non-convex problem. We show that Koopman operator theoretic methods allow for accurate predictions of weights and biases of feedforward, fully connected deep networks over a non-trivial range of training time. During this window, we find that our approach is >10x faster than various gradient descent based methods (e.g. Adam, Adadelta, Adagrad), in line with our complexity analysis. We end by highlighting open questions in this exciting intersection between dynamical systems and neural network theory, and additional methods by which our results may be generalized.
Histological evidence suggests that the estrous cycle exerts a powerful influence on CA1 neurons in the mammalian hippocampus. Decades have passed since this landmark observation, yet how the estrous cycle shapes dend...
详细信息
Neural Networks have been identified as potentially powerful tools for the study of complex systems. A noteworthy example is the Neural Network Differential Equation (NN DE) solver, which can provide functional approx...
详细信息
Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take t...
详细信息
The generalized Lotka-Volterra (gLV) equations are a mathematical proxy for ecological dynamics. We focus on a gLV model of the gut microbiome, in which the evolution of the gut microbial state is determined in part b...
详细信息
暂无评论