Transformers and cortical waves: encoders for pulling in context across time

作     者：Muller, Lyle Churchland, Patricia S. Sejnowski, Terrence J. 

作者机构：Western Univ Dept Math London ON Canada Fields Inst Fields Lab Network Sci Toronto ON Canada Univ Calif San Diego Dept Philosophy San Diego CA 92093 USA Salk Inst Biol Studies Computat Neurobiol Lab San Diego CA 92168 USA Univ Calif San Diego Dept Neurobiol San Diego CA 92103 USA 

出 版 物：《TRENDS IN NEUROSCIENCES》 (Trends Neurosci.)

年 卷 期：2024年第47卷第10期

页      面：788-802页

核心收录：

学科分类：1002[医学-临床医学] 1001[医学-基础医学(可授医学、理学学位)] 10[医学] 

基　　金：ONR [N000014-23-1-2069] NIH [DA055668-01, MH132644-01, R01-EY028723] NSF NeuroNex [DBI-2014862] Swartz Foundation BrainsCAN at Western University through the Canada First Research Excellence Fund (CFREF) NSF/CIHR NeuroNex [DBI-2015276] Natural Sciences and Engineering Research Council of Canada (NSERC) Western Academy for Advanced Research, NIH [U01-NS131914] 

主　　题：large language models artificial intelligence sensory processing visual processing brain computational neuroscience 

摘      要：The capabilities of transformer networks such as ChatGPT and other large language models (LLMs) have captured the world s attention. The crucial computational mechanism underlying their performance relies on transforming a complete input sequence - for example, all the words in a sentence - into a long encoding vector that allows transformers to learn long-range temporal dependencies in naturalistic sequences. Specifically, self-attention applied to this encoding vector enhances temporal context in transformers by computing associations between pairs of words in the input sequence. We suggest that waves of neural activity traveling across single cortical areas, or multiple regions on the whole-brain scale, could implement a similar encoding principle. By encapsulating recent input history into a single spatial pattern at each moment in time, cortical waves may enable a temporal context to be extracted from sequences of sensory inputs, the same computational principle as that used in transformers.