CSRNCVA: A MODEL OF CROSS-MEDIA SEMANTIC RETRIEVAL BASED ON NEURAL COMPUTING OF VISUAL AND AUDITORY SENSATIONS

作     者：Liu, Y. Cai, K. Liu, C. Zheng, F. 

作者机构：Henan Univ Intelligent Technol & Applicat Engn Res Ctr Henan Key Lab Big Data Anal & Proc Henan Prov Kaifeng 475004 Peoples R China Henan Univ Coll Comp Sci & Informat Engn Kaifeng 475004 Peoples R China Henan Univ Intelligent Technol & Applicat Engn Res Ctr Henan Kaifeng 475004 Peoples R China Henan Univ Key Lab Big Data Anal & Proc Henan Prov Kaifeng 475004 Peoples R China Henan Univ Engn Lab Spatial Informat Henan Prov Kaifeng 475004 Peoples R China 

出 版 物：《NEURAL NETWORK WORLD》 (Neural Network World)

年 卷 期：2018年第28卷第4期

页      面：305-323页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China Projects of Center for Remote Sensing Mission Study of China National Space Administration [2012A03A0939] Key Research and Promotion Projects of Henan Province Science and Technological Research of Key Projects of Henan Province 

主　　题：cross-media cognitive neural computing cross-media semantic retrieval deep belief network hierarchical temporal memory probabilistic graphical model hierarchical reinforcement learning 

摘      要：Cross-media semantic retrieval (CSR) and cross-modal semantic mapping are key problems of the multimedia search engine. The cognitive function and neural structure for visual and auditory information process are an important reference for the study of brain-inspired CSR. In this paper, we analyze the hierarchy, the functionality and the structure of visual and auditory in the brain. Considering an idea from deep belief network and hierarchical temporal memory, we presented a brain-inspired intelligent model, called cross-media semantic retrieval based on neural computing of visual and auditory sensation (CSRNCVA). Algorithms based on CSRNCVA were developed. It employs belief propagation algorithms of probabilistic graphical model and hierarchical learning. The experiments show that our model and algorithms can be effectively applied to the CSR. This work provides an important significance for brain-inspired cross-media intelligence framework.