Background: Due to the sparse encoding character of the human visual cortex and the scarcity of paired training samples for {images, fMRIs}, voxel selection is an effective means of reconstructing perceived images fro...
详细信息
Background: Due to the sparse encoding character of the human visual cortex and the scarcity of paired training samples for {images, fMRIs}, voxel selection is an effective means of reconstructing perceived images from fMRI. However, the existing data-driven voxel selection methods have not achieved satisfactory results. New method: Here, a novel deep reinforcement learning-guided sparse voxel (DRL-SV) decoding model is proposed to reconstruct perceived images from fMRI. We innovatively describe voxel selection as a Markov decision process (MDP), training agents to select voxels that are highly involved in specific visual encoding. Results: Experimental results on two public datasets verify the effectiveness of the proposed DRL-SV, which can accurately select voxels highly involved in neuralencoding, thereby improving the quality of visual image reconstruction. Comparison with existing methods: We qualitatively and quantitatively compared our results with the state-of-the-art (SOTA) methods, getting better reconstruction results. We compared the proposed DRL-SV with traditional data-driven baseline methods, obtaining sparser voxel selection results, but better reconstruction performance. Conclusions: DRL-SV can accurately select voxels involved in visual encoding on few-shot, compared to data-driven voxel selection methods. The proposed decoding model provides a new avenue to improving the image reconstruction quality of the primary visual cortex.
Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstrac...
详细信息
Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstraction;however, it does not explicitly provide any insights into the internal operations of DNNs. Deep learning's success is appealing to neuroscientists not only as a method for applying DNNs to model biological neural systems but also as a means of adopting concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks, such as PyTorch and TensorFlow, could be used to allow such cross-disciplinary investigations, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed as a mechanism for cognitive neuroscientists to map both DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring the internal representations of DNNs as well as brains. Through the integration of DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios. These include extracting DNN activation, probing and visualizing DNN representations, and mapping DNN representations onto the brain. We expect that our toolbox will accelerate scientific research by both applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.
Recent use of voxel-wise modeling in cognitive neuroscience suggests that semantic maps tile the cortex. Although this impressive research establishes distributed cortical areas active during the conceptual processing...
详细信息
Recent use of voxel-wise modeling in cognitive neuroscience suggests that semantic maps tile the cortex. Although this impressive research establishes distributed cortical areas active during the conceptual processing that underlies semantics, it tells us little about the nature of this processing. While mapping concepts between Man's computational and implementation levels to support neural encoding and decoding, this approach ignores Mares algorithmic level, central for understanding the mechanisms that implement cognition, in general, and conceptual processing, in particular. Following decades of research in cognitive science and neuroscience, what do we know so far about the representation and processing mechanisms that implement conceptual abilities? Most basically, much is known about the mechanisms associated with: (1) feature and frame representations, (2) grounded, abstract, and linguistic representations, (3) knowledge-based inference, (4) concept composition, and (5) conceptual flexibility. Rather than explaining these fundamental representation and processing mechanisms, semantic tiles simply provide a trace of their activity over a relatively short time period within a specific learning context. Establishing the mechanisms that implement conceptual processing in the brain will require more than mapping it to cortical (and sub-cortical) activity, with process models from cognitive science likely to play central roles in specifying the intervening mechanisms. More generally, neuroscience will not achieve its basic goals until it establishes algorithmic-level mechanisms that contribute essential explanations to how the brain works, going beyond simply establishing the brain areas that respond to various task conditions.
In this paper, we investigate neural circuit architectures encoding natural visual scenes with neuron models consisting of dendritic stimulus processors (DSPs) in cascade with biophysical spike generators (BSGs). DSPs...
详细信息
In this paper, we investigate neural circuit architectures encoding natural visual scenes with neuron models consisting of dendritic stimulus processors (DSPs) in cascade with biophysical spike generators (BSGs). DSPs serve as functional models of processing of stimuli up to and including the neuron's active dendritic tree. BSGs model spike generation at the axon hillock level where neurons respond to aggregated synaptic currents. The highly nonlinear behavior of BSGs calls for novel methods of input/output (I/O) analysis of neuralencoding circuits and novel decoding algorithms for signal recovery. On the encoding side we characterize the BSG I/O with a phase response curve (PRC) manifold and interpret neuralencoding as generalized sampling. We provide a decoding algorithm that recovers visual stimuli encoded by a neural circuit with intrinsic noise sources. In the absence of noise, we give conditions on perfect reconstruction of natural visual scenes. We extend the architecture to encompass neuron models with ON-OFF BSGs with self-and cross-feedback. With the help of the PRC manifold, decoding is shown to be tractable even for a wide signal dynamic range. Consequently, bias currents that were essential in the encoding process can largely be reduced or eliminated. Finally, we present examples of massively parallel encoding and decoding of natural visual scenes on a cluster of graphical processing units (GPUs). We evaluate the signal reconstruction under different noise conditions and investigate the performance of signal recovery in the Nyquist region and for different temporal bandwidths.
Motor output mostly depends on sensory input, which also can be affected by action. To further our understanding of how tactile information is processed in the primary somatosensory cortex (S1) in dynamic environments...
详细信息
Motor output mostly depends on sensory input, which also can be affected by action. To further our understanding of how tactile information is processed in the primary somatosensory cortex (S1) in dynamic environments, we recorded neural responses to tactile stimulation of the hand in three awake monkeys under arm/hand passive movement and rest. We found that neurons generally responded to tactile stimulation under both conditions and were modulated by movement: with a higher baseline firing rate, a suppressed peak rate, and a smaller dynamic range during passive movement than during rest, while the area under the response curve was stable across these two states. By using an information theory-based method, the mutual information between tactile stimulation and neural responses was quantified with rate and spatial coding models under the two conditions. The two potential encoding models showed different contributions depending on behavioral contexts. Tactile information encoded with rate coding from individual units was lower than spatial coding of unit pairs, especially during movement;however, spatial coding had redundant information between unit pairs. Passive movement regulated the mutual information, and such regulation might play different roles depending on the encoding strategies used. The underlying mechanisms of our observation most likely come from a bottom-up strategy, where neurons in S1 were regulated through the activation of the peripheral tactile/proprioceptive receptors and the interactions between these different types of information.
Visual context plays an important role in humans' top-down gaze movement control for target searching. Exploring the mental development mechanism in terms of incremental visual context encoding by population cells...
详细信息
Visual context plays an important role in humans' top-down gaze movement control for target searching. Exploring the mental development mechanism in terms of incremental visual context encoding by population cells is an interesting issue. This paper presents a biologically inspired computational model. The visual contextual cues were used in this model for top-down eye-motion control on searching targets in images. We proposed a population cell coding mechanism for visual context encoding and decoding. The model was implemented in a neural network system. A developmental learning mechanism was simulated in this system by dynamically generating new coding neurons to incrementally encode visual context during training. The encoded context was decoded with population neurons in a top-down mode. This allowed the model to control the gaze motion to the centers of the targets. The model was developed with pursuing low encoding quantity and high target locating accuracy. Its performance has been evaluated by a set of experiments to search different facial objects in a human face image set. Theoretical analysis and experimental results show that the proposed visual context encoding algorithm without weight updating is fast, efficient and stable, and the population-cell coding generally performs better than single-cell coding and k-nearest-neighbor (k-NN)-based coding.
暂无评论