检索结果-内蒙古大学图书馆

Toward optimality in scalable predictive coding

IEEE TRANSACTIONS ON IMAGE PROCESSING 2001年第7期10卷 965-976页

作者： Rose, K Regunathan, SL Univ Calif Santa Barbara Dept Elect & Comp Engn Santa Barbara CA 93106 USA

A method is proposed for efficient scalability in predictive coding, which overcomes known fundamental shortcomings of the prediction loop at enhancement layers. The compression efficiency of an enhancement-layer is substantially improved by casting the design of its prediction module within an estimation-theoretic framework, and thereby exploiting all information available at that layer fur the prediction of the signal, and encoding of the prediction error. While the most immediately important application is in video compression, the method is derived in a general setting and is applicable to any scalable predictive coder. Thus, the estimation-theoretic approach is first developed for basic DPCM compression and demonstrates the power of the technique in a simple setting that only involves straightforward prediction, scalar quantization, and entropy coding, Results for the scalable compression of first-order Gauss-Markov and Laplace-Markov signals illustrate the performance. A specific estimation algorithm is then developed for standard scalable DCT-based video coding. Simulation results show consistent and substantial performance gains due to optimal estimation at the enhancement-layers.

关键词： .DPCM coder prediction error enhancement-layer predictive coding scalability in predictive prediction module Gauss–Markov Laplace–Markov

来源：评论

学校读者我要写书评

暂无评论

Reconciling predictive coding and biased competition models of cortical function

引用

FRONTIERS IN COMPUTATIONAL NEUROSCIENCE 2008年第2008期2卷 1-8页

作者： Spratling, Michael W. Kings Coll London Div Engn London WC2R 2LS England Univ London Ctr Brain & Cognit Dev London England

A simple variation of the standard biased competition model is shown, via some trivial mathematical manipulations, to be identical to predictive coding. Specifically, it is shown that a particular implementation of the biased competition model, in which nodes compete via inhibition that targets the inputs to a cortical region, is mathematically equivalent to the linear predictive coding model. This observation demonstrates that these two important and influential rival theories of cortical function are minor variations on the same underlying mathematical model.

关键词： neural networks cortical circuits cortical feedback biased competition predictive coding

来源：评论

学校读者我要写书评

暂无评论

Brain-inspired predictive coding Improves the Performance of Machine Challenging Tasks

引用

FRONTIERS IN COMPUTATIONAL NEUROSCIENCE 2022年 16卷 1062678页

作者： Lee, Jangho Jo, Jeonghee Lee, Byounghwa Lee, Jung-Hoon Yoon, Sungroh Seoul Natl Univ Dept Elect & Comp Engn Seoul South Korea Seoul Natl Univ Inst New Media & Commun Seoul South Korea Elect & Telecommun Res Inst ETRI CybreBrain Res Sect Daejeon South Korea Seoul Natl Univ Interdisciplinary Program Artificial Intelligence Seoul South Korea

Backpropagation has been regarded as the most favorable algorithm for training artificial neural networks. However, it has been criticized for its biological implausibility because its learning mechanism contradicts the human brain. Although backpropagation has achieved super-human performance in various machine learning applications, it often shows limited performance in specific tasks. We collectively referred to such tasks as machine-challenging tasks (MCTs) and aimed to investigate methods to enhance machine learning for MCTs. Specifically, we start with a natural question: Can a learning mechanism that mimics the human brain lead to the improvement of MCT performances? We hypothesized that a learning mechanism replicating the human brain is effective for tasks where machine intelligence is difficult. Multiple experiments corresponding to specific types of MCTs where machine intelligence has room to improve performance were performed using predictive coding, a more biologically plausible learning algorithm than backpropagation. This study regarded incremental learning, long-tailed, and few-shot recognition as representative MCTs. With extensive experiments, we examined the effectiveness of predictive coding that robustly outperformed backpropagation-trained networks for the MCTs. We demonstrated that predictive coding-based incremental learning alleviates the effect of catastrophic forgetting. Next, predictive coding-based learning mitigates the classification bias in long-tailed recognition. Finally, we verified that the network trained with predictive coding could correctly predict corresponding targets with few samples. We analyzed the experimental result by drawing analogies between the properties of predictive coding networks and those of the human brain and discussing the potential of predictive coding networks in general machine learning.

关键词： brain-inspired learning biologically plausible learning deep learning backpropagation predictive coding

来源：评论

学校读者我要写书评

暂无评论

Spiking neural predictive coding for continually learning from data streams

引用

NEUROCOMPUTING 2023年 544卷

作者： Ororbia, Alexander Rochester Inst Technol Dept Comp Sci Neural Adapt Comp Lab 1 Lomb Mem Dr Rochester NY 14623 USA

For energy-efficient computation in specialized neuromorphic hardware, we present spiking neural coding, an instantiation of a family of artificial neural models grounded in the theory of predictive coding. This model, the first of its kind, works by operating in a never-ending process of "guess-and-check", where neu-rons predict the activity values of one another and then adjust their own activities to make better future pre-dictions. The interactive, iterative nature of our system fits well into the continuous time formulation of sensory stream prediction and, as we show, the model's structure yields a local synaptic update rule, which can be used to complement or as an alternative to online spike-timing dependent plasticity. In this article, we experiment with an instantiation of our model consisting of leaky integrate-and-fire units. However, the framework within which our system is situated can naturally incorporate more complex neurons such as the Hodgkin-Huxley model. Our experimental results in pattern recognition demonstrate the potential of the model when binary spike trains are the primary paradigm for inter-neuron communication. Notably, spiking neural coding is competitive in terms of classification performance and experiences less forgetting when learning from a task sequence, offering a more computationally economical, biologically-motivated alterna-tive to popular artificial neural networks.(c) 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://***/licenses/by/4.0/).

关键词： Spiking neural networks predictive coding Online learning Continual learning

来源：评论

学校读者我要写书评

暂无评论

Onboard Deep Lossless and Near-Lossless predictive coding of Hyperspectral Images With Line-Based Attention

引用

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2024年 62卷 1页

作者： Valsesia, Diego Bianchi, Tiziano Magli, Enrico Politecn Torino Dept Elect & Telecommun I-10129 Turin Italy

Deep learning methods have traditionally been difficult to apply to compression of hyperspectral images onboard spacecrafts due to the large computational complexity needed to achieve adequate representational power, as well as the lack of suitable datasets for training and testing. In this article, we depart from the traditional autoencoder approach, and we design a predictive neural network, called line receptance weighted key value (LineRWKV), which works recursively line by line to limit memory consumption. In order to achieve that, we adopt a novel hybrid attentive-recursive operation that combines the representational advantages of Transformers with the linear complexity and recursive implementation of recurrent neural networks (RNNs). The compression algorithm performs the prediction of each pixel using LineRWKV, followed by entropy coding of the residual. Experiments on multiple datasets show that LineRWKV is highly memory-efficient, significantly outperforms state-of-the-art deep learning methods, and is the first deep learning approach to outperform CCSDS-123.0-B-2 at lossless and near-lossless compression. Promising throughput results are also evaluated on a 7-W embedded system.

关键词： Image coding Hyperspectral imaging Complexity theory Neural networks Deep learning Rate-distortion Transformers hyperspectral image compression predictive coding receptance weighted key value (RWKV) self-attention

来源：评论

学校读者我要写书评

暂无评论

Deep Gated Hebbian predictive coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy

引用

FRONTIERS IN COMPUTATIONAL NEUROSCIENCE 2021年 15卷 666131页

作者： Dora, Shirin Bohte, Sander M. Pennartz, Cyriel M. A. Univ Amsterdam Swammerdam Inst Life Sci Cognit & Syst Neurosci Grp Amsterdam Netherlands Ulster Univ Inteltent Syst Res Ctr Londonderry North Ireland Ctr Math & Comp Sci Machine Learning Grp Amsterdam Netherlands

predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy.

关键词： visual processing predictive coding deep biologically plausible learning selectivity sparseness sensory neocortex inference representation learning

来源：评论

学校读者我要写书评

暂无评论

Examining predictive coding accounts of typical and autistic neurocognitive development

引用

NEUROSCIENCE AND BIOBEHAVIORAL REVIEWS 2024年 167卷 105905页

作者： Rapaport, Hannah Sowman, Paul F. Macquarie Univ Sch Psychol Sci Sydney Australia Univ Cambridge MRC Cognit & Brain Sci Unit 15 Chaucer Rd Cambridge CB2 7EF England Auckland Univ Technol Sch Clin Sci Auckland New Zealand

predictive coding has emerged as a prominent theoretical framework for understanding perception and its neural underpinnings. There has been a recent surge of interest in the predictive coding framework across the mind sciences. However, comparatively little of the research in this field has investigated the neural underpinnings of predictive coding in young neurotypical and autistic children. This paper provides an overview of predictive coding accounts of typical and autistic neurocognitive development and includes a review of the current electrophysiological evidence supporting these accounts. Based on the current evidence, it is clear that more research in pediatrics is needed to evaluate predictive coding accounts of neurocognitive development fully. If supported, these accounts could have wide-ranging practical implications for pedagogy, parenting, artificial intelligence, and clinical approaches to helping autistic children manage the barrage of everyday sensory information.

关键词： predictive coding Child Typical development Autism Auditory oddball Magnetoencephalography Electroencephalography Mismatch negativity Mismatch field

来源：评论

学校读者我要写书评

暂无评论

The N300: An Index for predictive coding of Complex Visual Objects and Scenes

引用

Cerebral cortex communications 2021年第2期2卷 tgab030页

作者： Manoj Kumar Kara D Federmeier Diane M Beck Princeton Neuroscience Institute Princeton University Princeton NJ 08540 USA. Department of Psychology Program in Neuroscience and the Beckman Institute for Advanced Science and Technology University of Illinois at Urbana-Champaign Urbana IL 61801 USA.

predictive coding models can simulate known perceptual or neuronal phenomena, but there have been fewer attempts to identify a reliable neural signature of predictive coding for complex stimuli. In a pair of studies, we test whether the N300 component of the event-related potential, occurring 250-350-ms poststimulus-onset, has the response properties expected for such a signature of perceptual hypothesis testing at the level of whole objects and scenes. We show that N300 amplitudes are smaller to representative ("good exemplars") compared with less representative ("bad exemplars") items from natural scene categories. Integrating these results with patterns observed for objects, we establish that, across a variety of visual stimuli, the N300 is responsive to statistical regularity, or the degree to which the input is "expected" (either explicitly or implicitly) based on prior knowledge, with statistically regular images evoking a reduced response. Moreover, we show that the measure exhibits context-dependency; that is, we find the N300 sensitivity to category representativeness when stimuli are congruent with, but not when they are incongruent with, a category pre-cue. Thus, we argue that the N300 is the best candidate to date for an index of perceptual hypotheses testing for complex visual objects and scenes.

关键词： N300 predictive coding statistical regularities visual perception

来源：评论

学校读者我要写书评

暂无评论

Mathematical Derivation of Wave Propagation Properties in Hierarchical Neural Networks with predictive coding Feedback Dynamics

引用

BULLETIN OF MATHEMATICAL BIOLOGY 2023年第9期85卷 80-80页

作者： Faye, Gregory Fouilhe, Guilhem VanRullen, Rufin Univ Toulouse Inst Math Toulouse UPS IMT UMR5219 F-31062 Toulouse 9 France Univ Toulouse Ctr Rech Cerveau & Cognit CerCo UMR5549 F-31052 Toulouse France Univ Toulouse ANITI F-31062 Toulouse France

Sensory perception (e.g., vision) relies on a hierarchy of cortical areas, in which neural activity propagates in both directions, to convey information not only about sensory inputs but also about cognitive states, expectations and predictions. At the macroscopic scale, neurophysiological experiments have described the corresponding neural signals as both forward and backward-travelling waves, sometimes with characteristic oscillatory signatures. It remains unclear, however, how such activity patterns relate to specific functional properties of the perceptual apparatus. Here, we present a mathematical framework, inspired by neural network models of predictive coding, to systematically investigate neural dynamics in a hierarchical perceptual system. We show that stability of the system can be systematically derived from the values of hyper-parameters controlling the different signals (related to bottom-up inputs, top-down prediction and error correction). Similarly, it is possible to determine in which direction, and at what speed neural activity propagates in the system. Different neural assemblies (reflecting distinct eigenvectors of the connectivity matrices) can simultaneously and independently display different properties in terms of stability, propagation speed or direction. We also derive continuous-limit versions of the system, both in time and in neural space. Finally, we analyze the possible influence of transmission delays between layers, and reveal the emergence of oscillations.

关键词： Hierarchical neural networks predictive coding Wave propagation

来源：评论

学校读者我要写书评

暂无评论

Beta- and gamma-band activity reflect predictive coding in the processing of causal events

引用

SOCIAL COGNITIVE AND AFFECTIVE NEUROSCIENCE 2016年第6期11卷 973-980页

作者： van Pelt, Stan Heil, Lieke Kwisthout, Johan Ondobaka, Sasha van Rooij, Iris Bekkering, Harold Radboud Univ Nijmegen Donders Inst Brain Cognit & Behav Montessorilaan 3 NL-6525 HR Nijmegen Netherlands UCL Wellcome Trust Ctr Neuroimaging London England UCL Sobell Dept London England

In daily life, complex events are perceived in a causal manner, suggesting that the brain relies on predictive processes to model them. Within predictive coding theory, oscillatory beta-band activity has been linked to top-down predictive signals and gamma-band activity to bottom-up prediction errors. However, neurocognitive evidence for predictive coding outside lower-level sensory areas is scarce. We used magnetoencephalography to investigate neural activity during probabilitydependent action perception in three areas pivotal for causal inference, superior temporal sulcus, temporoparietal junction and medial prefrontal cortex, using bowling action animations. Within this network, Granger-causal connectivity in the beta-band was found to be strongest for backward top-down connections and gamma for feed-forward bottom-up connections. Moreover, beta-band power in TPJ increased parametrically with the predictability of the action kinematics-outcome sequences. Conversely, gamma-band power in TPJ and MPFC increased with prediction error. These findings suggest that the brain utilizes predictive-coding-like computations for higher-order cognition such as perception of causal events.

关键词： causal inference action perception predictive coding magnetoencephalography connectivity

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：