Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing

作     者：Si, Zilin Zhu, Zirui Agarwal, Arpit Anderson, Stuart Yuan, Wenzhen 

作者机构：The Robotics Institute Carnegie Mellon University United States The Department of Electrical Engineering Tsinghua University China Meta Reality Labs Research 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2022年

核心收录：

主　　题：End effectors 

摘      要：Robot simulation has been an essential tool for data-driven manipulation tasks. However, most existing simulation frameworks lack either efficient and accurate models of physical interactions with tactile sensors or realistic tactile simulation. This makes the sim-to-real transfer for tactile-based manipulation tasks still challenging. In this work, we integrate simulation of robot dynamics and vision-based tactile sensors by modeling the physics of contact. This contact model uses simulated contact forces at the robot’s end-effector to inform the generation of realistic tactile outputs. To eliminate the simto-real transfer gap, we calibrate our physics simulator of robot dynamics, contact model, and tactile optical simulator with real-world data, and then we demonstrate the effectiveness of our system on a zero-shot sim-to-real grasp stability prediction task where we achieve an average accuracy of 90.7% on various objects. Experiments reveal the potential of applying our simulation framework to more complicated manipulation tasks. We open-source our simulation framework at https://***/CMURoboTouch/Taxim/tree/taxim-robot. © 2022, CC BY-NC-SA.