The task of visual question answering (VQA) has gained wide popularity in recent times. Effectively solving the VQA task requires the understanding of both the visual content in the image and the language information ...
详细信息
The task of visual question answering (VQA) has gained wide popularity in recent times. Effectively solving the VQA task requires the understanding of both the visual content in the image and the language information associated with the text-based question. In this study, the authors propose a novel method of encoding the visualinformation (categorical and spatial object information) of all the objects present in the image into a sequential format, which is called an object sequence. These object sequences can then be suitably processed by a neural network. They experiment with multiple techniques for obtaining a joint embedding from the visual features (in the form of object sequences) and language-based features obtained from the question. They also provide a detailed analysis on the performance of a neural network architecture using object sequences, on the Oracle task of GuessWhat dataset (a Yes/No VQA task) and benchmark it against the baseline.
Automatic extraction of distinctive features from a visualinformation stream is challenging due to the large amount of information contained in most image data. In recent years deep neural networks (DNNs) have gained...
详细信息
Automatic extraction of distinctive features from a visualinformation stream is challenging due to the large amount of information contained in most image data. In recent years deep neural networks (DNNs) have gained outstanding popularity for solving visualinformation processing tasks. This study reports novel contributions, including a new DNN architecture and training method, which increase the fidelity of DNN-based representations to encodings extracted by visual processing neurons. Our local receptive field constrained DNNs (LRF-DNNs) are pre-trained with a modified restricted Boltzmann machine, the LRF-RBM, which utilizes biologically inspired Gaussian receptive field constraints to encourage the emergence of local features. Moreover, we propose a method for concurrently finding advantageous receptive field centers, while training the LRF-RBM. By utilizing LRF-RBMs with gradually increasing receptive field sizes on each layer, our LRF-DNN learns features of increasing complexity and demonstrates hierarchical part-based compositionality. We show superior face completion and reconstruction results on the challenging LFW face dataset. (C) 2016 Elsevier Inc. All rights reserved.
Any kind of visualinformation is encoded in terms of patterns of neural activity occurring inside the brain. Decoding neural patterns or its classification is a challenging task. Functional magnetic resonance imaging...
详细信息
ISBN:
(纸本)9781424492701
Any kind of visualinformation is encoded in terms of patterns of neural activity occurring inside the brain. Decoding neural patterns or its classification is a challenging task. Functional magnetic resonance imaging (fMRI) and Electroencephalography (EEG) are non-invasive neuroimaging modalities to capture the brain activity pattern in term of images and electric potential respectively. To get higher spatiotemporal resolution of human brain from these two complementary neuroimaging modalities, simultaneous EEG-fMRI can be helpful. In this paper, we proposed a framework for classifying the brain activity patterns with simultaneous EEG-fMRI. We have acquired five human participants' data with simultaneous EEG-fMRI by showing different object categories. Further, combined analysis of EEG and fMRI data was carried out. Extracted information through combine analysis is passed to support vector machine (SVM) classifier for classification purpose. We have achieved better classification accuracy using simultaneous EEG-fMRI i.e., 81.8% as compared to fMRI data standalone. This shows that multimodal neuroimaging can improve the classification accuracy of brain activity patterns as compared to individual modalities reported in literature.
暂无评论