检索结果-内蒙古大学图书馆

The role of environmental feedback in a brain state switch from passive to active sensing

BMC Neuroscience 2013年第1期14卷 1-2页

作者： Christopher L Buckley Taro Toyoizumi Lab for Neural Computation and Adaptation Riken Brain Science Institute Tokyo Japan

来源：评论

学校读者我要写书评

暂无评论

Stiff-sloppy analysis of brain networks to reveal individual differences in task performance

arXiv

引用

arXiv 2025年

作者： Chen, Sida Tang, Qianyuan Toyoizumi, Taro Sommer, Werner Yu, Lianchun Zhou, Changsong Department of Physics Hong Kong Baptist University Kowloon Tong Hong Kong Centre for Nonlinear Studies Hong Kong Baptist University Kowloon Tong Hong Kong Lab for Neural Computation and Adaptation RIKEN Center for Brain Science 2-1 Hirosawa Saitama Wako351-0106 Japan Institut für Psychologie Humboldt-Universitat zu Berlin Germany School of Physical Science and Technology Lanzhou Center for Theoretical Physics Lanzhou University Gansu Lanzhou730000 China Key Laboratory of Theoretical Physics of Gansu Province Key Laboratory of Quantum Theory and Applications MoE Lanzhou University Gansu Lanzhou730000 China Life Science Imaging Centre Hong Kong Baptist University Kowloon Tong Hong Kong Faculty of Education National University of Malaysia Kuala Lumpur Malaysia

Understanding how brain networks recruit resources during cognitive tasks is key to explaining individual differences in task performance. Brain network parameters—including activity levels of regions and their connectivity—reflect the integration and segregation of functional subnetworks underlying task processing. However, the complexity and high dimensionality of these parameters pose a significant barrier to identifying functionally relevant individual differences. Here, we introduce stiff-sloppy analysis as a framework for uncovering the stiff parameter combinations that critically influence task-state brain dynamics, exemplified by working memory. Using the pairwise maximum entropy model (PMEM) calibrated to fMRI data and Fisher Information Matrix (FIM) analysis, we reveal that the stiff dimensions of the model parameters capture the most relevant integration and segregation processes of the default mode network and the working memory network. Individual differences along these stiff neural dimensions consistently correlate with working memory performance. Notably, stiff parameters robustly predicted working memory performance, even when the less sensitive ("sloppy") parameters were excluded. This study establishes stiff-sloppy analysis as a powerful approach to identify cognition-related brain networks, bridging neural dynamics and behavior and offering new avenues for personalized neuroscience including therapeutic innovation. Copyright © 2025, The Authors. All rights reserved.

关键词： Fisher information matrix

来源：评论

学校读者我要写书评

暂无评论

ON THE POWER-LAW HESSIAN SPECTRUMS IN DEEP LEARNING

arXiv

引用

arXiv 2022年

作者： Xie, Zeke Tang, Qian-Yuan Cai, Yunfeng Sun, Mingming Li, Ping Cognitive Computing Lab Baidu Research China Lab for Neural Computation and Adaptation RIKEN Center for Brain Science Japan

It is well-known that the Hessian of deep loss landscape matters to optimization, generalization, and even robustness of deep learning. Recent works empirically discovered that the Hessian spectrum in deep learning has a two-component structure that consists of a small number of large eigenvalues and a large number of nearly-zero eigenvalues. However, the theoretical mechanism or the mathematical behind the Hessian spectrum is still largely under-explored. To the best of our knowledge, we are the first to demonstrate that the Hessian spectrums of well-trained deep neural networks exhibit simple power-law structures. Inspired by the statistical physical theories and the spectral analysis of natural proteins, we provide a maximum-entropy theoretical interpretation for explaining why the power-law structure exist and suggest a spectral parallel between protein evolution and training of deep neural networks. By conducing extensive experiments, we further use the power-law spectral framework as a useful tool to explore multiple novel behaviors of deep learning. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Causal Graph in Language Model Rediscovers Cortical Hierarchy in Human Narrative Processing

arXiv

引用

arXiv 2023年

作者： He, Zhengqi Toyoizumi, Taro Lab for Neural Computation and Adaptation RIKEN Center for Brain Science Japan Department of Mathematical Informatics Graduate School of Information Science and Technology The University of Tokyo Japan

Understanding how humans process natural language has long been a vital research direction. The field of natural language processing (NLP) has recently experienced a surge in the development of powerful language models. These models have proven to be invaluable tools for studying another complex system known to process human language: the brain. Previous studies have demonstrated that the features of language models can be mapped to fMRI brain activity. This raises the question: is there a commonality between information processing in language models and the human brain? To estimate information flow patterns in a language model, we examined the causal relationships between different layers. Drawing inspiration from the workspace framework for consciousness, we hypothesized that features integrating more information would more accurately predict higher hierarchical brain activity. To validate this hypothesis, we classified language model features into two categories based on causal network measures:’low in-degree’ and’high in-degree’. We subsequently compared the brain prediction accuracy maps for these two groups. Our results reveal that the difference in prediction accuracy follows a hierarchical pattern, consistent with the cortical hierarchy map revealed by activity time constants. This finding suggests a parallel between how language models and the human brain process linguistic information. © 2023, CC BY.

关键词： Brain

来源：评论

学校读者我要写书评

暂无评论

Spontaneous Emerging Preference in Two-tower Language Model

arXiv

引用

arXiv 2022年

作者： He, Zhengqi Toyoizumi, Taro Lab for Neural Computation and Adaptation Riken Center for Brain Science Japan Department of Mathematical Informatics Graduate School of Information Science and Technology The University of Tokyo Japan

The ever-growing size of the foundation language model has brought significant performance gains in various types of downstream tasks. With the existence of side-effects brought about by the large size of the foundation language model such as deployment cost, availability issues, and environmental cost, there is some interest in exploring other possible directions, such as a divide-and-conquer scheme. In this paper, we are asking a basic question: are language processes naturally dividable? We study this problem with a simple two-tower language model setting, where two language models with identical configurations are trained side-by-side cooperatively. With this setting, we discover the spontaneous emerging preference phenomenon, where some of the tokens are consistently better predicted by one tower while others by another tower. This phenomenon is qualitatively stable, regardless of model configuration and type, suggesting this as an intrinsic property of natural language. This study suggests that interesting properties of natural language are still waiting to be discovered, which may aid the future development of natural language processing techniques. © 2022, CC BY.

关键词： Towers

来源：评论

学校读者我要写书评

暂无评论

Progressive Interpretation Synthesis: Interpreting Task Solving by Quantifying Previously Used and Unused Information

arXiv

引用

arXiv 2021年

作者： He, Zhengqi Toyoizumi, Taro Lab for Neural Computation and Adaptation RIKEN Center for Brain Science Saitama Japan Department of Mathematical Informatics Graduate School of Information Science and Technology The University of Tokyo Tokyo Japan

A deep neural network is a good task solver, but it is difficult to make sense of its operation. People have different ideas about how to form the interpretation about its operation. We look at this problem from a new perspective where the interpretation of task solving is synthesized by quantifying how much and what previously unused information is exploited in addition to the information used to solve previous tasks. First, after learning several tasks, the network acquires several information partitions related to each task. We propose that the network, then, learns the minimal information partition that supplements previously learned information partitions to more accurately represent the input. This extra partition is associated with un-conceptualized information that has not been used in previous tasks. We manage to identify what un-conceptualized information is used and quantify the amount. To interpret how the network solves a new task, we quantify as meta-information how much information from each partition is extracted. We implement this framework with the variational information bottleneck technique. We test the framework with the MNIST and the CLEVR dataset. The framework is shown to be able to compose information partitions and synthesize experience-dependent interpretation in the form of meta-information. This system progressively improves the resolution of interpretation upon new experience by converting a part of the un-conceptualized information partition to a task-related partition. It can also provide a visual interpretation by imaging what is the part of previously un-conceptualized information that is needed to solve a new task. © 2021, CC BY.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Interlaminar processing in auditory cortex before and after auditory trauma: spontaneous and evoked responses of independent sources

引用

BMC Neuroscience 2013年第1期14卷 1-2页

作者： Erin Munro Taro Toyoizumi Shuzo Sakata Lab. for Neural Computation and Adaptation RIKEN Brain Science Institute Wakoshi Saitama 351-0198 Japan Strathclyde Institute of Pharmacy and Biomedical Sciences University of Strathclyde Glasgow G4 0RE UK

来源：评论

学校读者我要写书评

暂无评论

Dynamics-Evolution Correspondence in Protein Structures

引用

Physical Review Letters 2021年第9期127卷 098103-098103页

作者： Qian-Yuan Tang Kunihiko Kaneko Center for Complex Systems Biology Universal Biology Institute University of Tokyo Komaba 3-8-1 Meguro-ku Tokyo 153-8902 Japan Lab for Neural Computation and Adaptation RIKEN Center for Brain Science 2-1 Hirosawa Wako Saitama 351-0198 Japan

The genotype-phenotype mapping of proteins is a fundamental question in structural biology. In this Letter, with the analysis of a large dataset of proteins from hundreds of protein families, we quantitatively demonstrate the correlations between the noise-induced protein dynamics and mutation-induced variations of native structures, indicating the dynamics-evolution correspondence of proteins. Based on the investigations of the linear responses of native proteins, the origin of such a correspondence is elucidated. It is essential that the noise- and mutation-induced deformations of the proteins are restricted on a common low-dimensional subspace, as confirmed from the data. These results suggest an evolutionary mechanism of the proteins gaining both dynamical flexibility and evolutionary structural variability.

关键词： Biological complexity Protein structure Proteins Data analysis

来源：评论

学校读者我要写书评

暂无评论

Learning poly-synaptic paths with traveling waves

arXiv

引用

arXiv 2019年

作者： Ito, Yoshiki Toyoizumi, Taro Graduate School of Information and Technology The Department of Mechano-Informatics The University of Tokyo Tokyo Japan Lab for Neural Computation and Adaptation RIKEN Center for Brain Science Saitama Japan Department of Mathematical Informatics Graduate School of Information Science and Technology The University of Tokyo Tokyo Japan

Traveling waves are commonly observed across the brain. While previous studies have suggested the role of traveling waves in learning, the mechanism remains unclear. We adopted a computational approach to investigate the effect of traveling waves on synaptic plasticity. Our results indicate that traveling waves facilitate the learning of poly-synaptic network paths when combined with a reward-dependent local synaptic plasticity rule. We also demonstrate that traveling waves expedite finding the shortest paths and learning nonlinear input/output mapping, such as exclusive or (XOR) function. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：