检索结果-内蒙古大学图书馆

SIMPLICIAL HOPFIELD NETWORKS 11

学校读者我要写书评

暂无评论

SIMPLICIAL HOPFIELD NETWORKS

11th International Conference on Learning Representations, ICLR 2023

作者： Burns, Thomas F. Fukai, Tomoki Neural Coding and Brain Computing Unit OIST Graduate University Okinawa Japan

Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using N neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are higher dimensional analogues of graphs which naturally represent collections of pairwise and setwise relationships. We show that our simplicial Hopfield networks increase memory storage capacity. Surprisingly, even when connections are limited to a small random subset of equivalent size to an all-pairwise network, our networks still outperform their pairwise counterparts. Such scenarios include non-trivial simplicial topology. We also test analogous modern continuous Hopfield networks, offering a potentially promising avenue for improving the attention mechanism in Transformer models. © 2023 11th International Conference on Learning Representations, ICLR 2023. All rights reserved.

关键词： Complex networks

convSeq: Fast and Scalable Method for Detecting Patterns in Spike Data 41

学校读者我要写书评

暂无评论

convSeq: Fast and Scalable Method for Detecting Patterns in ...

41st International Conference on Machine Learning, ICML 2024

作者： Koshkin, Roman Fukai, Tomoki Neural Coding and Brain Computing Unit Okinawa Institute of Science and Technology Okinawa Japan

Spontaneous neural activity, crucial in memory, learning, and spatial navigation, often manifests itself as repetitive spatiotemporal patterns. Despite their importance, analyzing these patterns in large neural recordings remains challenging due to a lack of efficient and scalable detection methods. Addressing this gap, we introduce convSeq, an unsupervised method that employs backpropagation for optimizing spatiotemporal filters that effectively identify these neural patterns. Our method's performance is validated on various synthetic data and real neural recordings, revealing spike sequences with unprecedented scalability and efficiency. Significantly surpassing existing methods in speed, convSeq sets a new standard for analyzing spontaneous neural activity, potentially advancing our understanding of information processing in neural circuits. Copyright 2024 by the author(s)

关键词： Spatio-temporal data

Parallel processing of past and future memories through reactivation and synaptic plasticity mechanisms during sleep

学校读者我要写书评

暂无评论

Nature Communications 2025年第1期16卷 1-19页

作者： Ghandour, Khaled Haga, Tatsuya Ohkawa, Noriaki Fung, Chi Chung Alan Nomoto, Masanori Fayed, Mostafa R. Asai, Hirotaka Sato, Masaaki Fukai, Tomoki Inokuchi, Kaoru Graduate School of Medicine and Pharmaceutical Sciences University of Toyama Toyama Japan Research Center for Idling Brain Science University of Toyama Toyama Japan Center initiative for training international researchers (CITIR) University of Toyama Toyama Japan Department of Biochemistry Faculty of Pharmacy Cairo University Cairo Egypt Neural Coding and Brain Computing unit OIST Okinawa Japan Center for Information and Neural Networks (CiNet) National Institute of Information and Communications Technology Osaka Japan Research Center for Advanced Medical Science Dokkyo Medical University Tochigi Japan Department of Neuroscience City University of Hong Kong Tat Chee Avenue Kowloon Hong Kong Department of Pharmacology and Toxicology Kafrelsheikh University Kafr El Sheikh Egypt Department of Neuropharmacology Hokkaido University Graduate School of Medicine Sapporo Japan Graduate School of Medicine The University of Tokyo Tokyo Japan

Every day, we experience new episodes and store new memories. Although memories are stored in corresponding engram cells, how different sets of engram cells are selected for current and next episodes, and how they create their memories, remains unclear. Here we show that in male mice, hippocampal CA1 neurons show an organized synchronous activity in prelearning home cage sleep that correlates with the learning ensembles only in engram cells, termed preconfigured ensembles. Moreover, after learning, a subset of nonengram cells develops population activity, which is constructed during postlearning offline periods, and then emerges to represent engram cells for new learning. Our model suggests a potential role of synaptic depression and scaling in the reorganization of the activity of nonengram cells. Together, our findings indicate that during offline periods there are two parallel processes occurring: conserving of past memories through reactivation, and preparation for upcoming ones through offline synaptic plasticity mechanisms. © The Author(s) 2025.

关键词：

SIMPLICIAL HOPFIELD NETWORKS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Burns, Thomas F. Fukai, Tomoki Neural Coding and Brain Computing Unit OIST Graduate University Okinawa Japan

Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using N neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are higher dimensional analogues of graphs which naturally represent collections of pairwise and setwise relationships. We show that our simplicial Hopfield networks increase memory storage capacity. Surprisingly, even when connections are limited to a small random subset of equivalent size to an all-pairwise network, our networks still outperform their pairwise counterparts. Such scenarios include non-trivial simplicial topology. We also test analogous modern continuous Hopfield networks, offering a potentially promising avenue for improving the attention mechanism in Transformer *** Codes 68T07, 92B20, 68T01, 00A69 © 2023, CC BY.

关键词： Complex networks

convSeq: fast and scalable method for detecting patterns in spike data 24

学校读者我要写书评

暂无评论

convSeq: fast and scalable method for detecting patterns in ...

Proceedings of the 41st International Conference on Machine Learning

作者： Roman Koshkin Tomoki Fukai Neural Coding and Brain Computing Unit Okinawa Institute of Science and Technology Okinawa Japan

关键词：

convSeq: Fast and Scalable Method for Detecting Patterns in Spike Data

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Koshkin, Roman Fukai, Tomoki Neural Coding and Brain Computing Unit Okinawa Institute of Science and Technology Okinawa Japan

Spontaneous neural activity, crucial in memory, learning, and spatial navigation, often manifests itself as repetitive spatiotemporal patterns. Despite their importance, analyzing these patterns in large neural recordings remains challenging due to a lack of efficient and scalable detection methods. Addressing this gap, we introduce convSeq, an unsupervised method that employs backpropagation for optimizing spatiotemporal filters that effectively identify these neural patterns. Our method’s performance is validated on various synthetic data and real neural recordings, revealing spike sequences with unprecedented scalability and efficiency. Significantly surpassing existing methods in speed, convSeq sets a new standard for analyzing spontaneous neural activity, potentially advancing our understanding of information processing in neural circuits. © 2024, CC BY.

关键词： Scalability

EFFICIENT, PROBABILISTIC ANALYSIS OF COMBINATORIAL neural CODES

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Burns, Thomas F. Irwansyah Neural Coding and Brain Computing Unit OIST Graduate University Okinawa Japan Department of Mathematics University of Mataram West Nusa Tenggara Indonesia

Artificial and biological neural networks (ANNs and BNNs) can encode inputs in the form of combinations of individual neurons' activities. These combinatorial neural codes present a computational challenge for direct and efficient analysis due to their high dimensionality and often large volumes of data. Here we improve the computational complexity - from factorial to quadratic time - of direct algebraic methods previously applied to small examples and apply them to large neural codes generated by experiments. These methods provide a novel and efficient way of probing algebraic, geometric, and topological characteristics of combinatorial neural codes and provide insights into how such characteristics are related to learning and experience in neural networks. We introduce a procedure to perform hypothesis testing on the intrinsic features of neural codes using information geometry. We then apply these methods to neural activities from an ANN for image classification and a BNN for 2D navigation to, without observing any inputs or outputs, estimate the structure and dimensionality of the stimulus or task space. Additionally, we demonstrate how an ANN varies its internal representations across network depth and during *** Codes 92B20 (Primary) 12Y05, 92B05 (Secondary) © 2022, CC BY-NC-SA.

关键词： neural networks

Semantically-correlated memories in a dense associative model 24

学校读者我要写书评

暂无评论

Semantically-correlated memories in a dense associative mode...

Proceedings of the 41st International Conference on Machine Learning

作者： Thomas F Burns Institute for Computational and Experimental Research in Mathematics Brown University and SciAI Center Cornell University and Neural Coding and Brain Computing Unit OIST Graduate University Japan

I introduce a novel associative memory model named Correlated Dense Associative Memory (CDAM), which integrates both auto- and hetero-association in a unified framework for continuous-valued memory patterns. Employing an arbitrary graph structure to semantically link memory patterns, CDAM is theoretically and numerically analysed, revealing four distinct dynamical modes: auto-association, narrow hetero-association, wide hetero-association, and neutral quiescence. Drawing inspiration from inhibitory modulation studies, I employ anti-Hebbian learning rules to control the range of hetero-association, extract multi-scale representations of community structures in graphs, and stabilise the recall of temporal sequences. Experimental demonstrations showcase CDAM's efficacy in handling realworld data, replicating a classical neuroscience experiment, performing image retrieval, and simulating arbitrary finite automata.

关键词：

Goal-oriented inference of environment from redundant observations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Takahashi, Kazuki Fukai, Tomoki Sakai, Yutaka Takekawa, Takashi Informatics Program Graduate School of Engineering Kogakuin University of Technology and Engineering Japan Neural Coding and Brain Computing Unit Okinawa Institute of Science and Technology Japan Brain Science Institute Tamagawa University Japan

The agent learns to organize decision behavior to achieve a behavioral goal, such as reward maximization, and reinforcement learning is often used for this optimization. Learning an optimal behavioral strategy is difficult under the uncertainty that events necessary for learning are only partially observable, called as Partially Observable Markov Decision Process (POMDP). However, the real-world environment also gives many events irrelevant to reward delivery and an optimal behavioral strategy. The conventional methods in POMDP, which attempt to infer transition rules among the entire observations, including irrelevant states, are ineffective in such an environment. Supposing Redundantly Observable Markov Decision Process (ROMDP), here we propose a method for goal-oriented reinforcement learning to efficiently learn state transition rules among reward-related"core states" from redundant observations. Starting with a small number of initial core states, our model gradually adds new core states to the transition diagram until it achieves an optimal behavioral strategy consistent with the Bellman equation. We demonstrate that the resultant inference model outperforms the conventional method for POMDP. We emphasize that our model only containing the core states has high explainability. Furthermore, the proposed method suits online learning as it suppresses memory consumption and improves learning speed. Copyright © 2023, The Authors. All rights reserved.

关键词： Reinforcement learning