MQA: Answering the Question via Robotic Manipulation

作     者：Deng, Yuhong Guo, Di Guo, Xiaofeng Zhang, Naifu Liu, Huaping Sun, Fuchun 

作者机构：Department of Computer Science and Technology Tsinghua University China Beijing National Research Center for Information Science and Technology Center of Intelligent Control and Telescience Tsinghua Shenzhen International Graduate School Tsinghua University China 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2020年

核心收录：

摘      要：In this paper, we propose a novel task, Manipulation Question Answering (MQA), where the robot performs manipulation actions to change the environment in order to answer a given question. To solve this problem, a framework consisting of a QA module and a manipulation module is proposed. For the QA module, we adopt the method for the Visual Question Answering (VQA) task. For the manipulation module, a Deep Q Network (DQN) model is designed to generate manipulation actions for the robot to interact with the environment. We consider the situation where the robot continuously manipulating objects inside a bin until the answer to the question is found. Besides, a novel dataset that contains a variety of object models, scenarios and corresponding question-answer pairs is established in a simulation environment. Extensive experiments have been conducted to validate the effectiveness of the proposed framework. Copyright © 2020, The Authors. All rights reserved.