MemX: An Atention-Aware Smart Eyewear System for Personalized Moment Auto-capture

作     者：Chang, Yuhu Zhao, Yingying Dong, Mingzhi Wang, Yujiang Lu, Yutian Lv, Qin Dick, Robert P. Lu, Tun Gu, Ning Shang, Li 

作者机构：School of Computer Science Fudan University Shanghai200438 China Shanghai Key Laboratory of Data Science Fudan University Shanghai200438 China Department of Computing Imperial College London London United Kingdom University of Colorado Boulder BoulderCO United States University of Michigan Ann ArborMI United States 

出 版 物：《Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies》 (Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.)

年 卷 期：2021年第5卷第2期

核心收录：

基　　金：This work was supported in part by the National Natural Science Foundation of China under Grant No. 62090025 and 61932007 and in part by the National Science Foundation of the United States under grant CNS-2008151 

主　　题：Energy efficiency 

摘      要：This work presents MemX: a biologically-inspired attention-aware eyewear system developed with the goal of pursuing the long-awaited vision of a personalized visual Memex. MemX captures human visual attention on the fly, analyzes the salient visual content, and records moments of personal interest in the form of compact video snippets. Accurate attentive scene detection and analysis on resource-constrained platforms is challenging because these tasks are computation and energy intensive. We propose a new temporal visual attention network that unifies human visual attention tracking and salient visual content analysis. Attention tracking focuses computation-intensive video analysis on salient regions, while video analysis makes human attention detection and tracking more accurate. Using the YouTube-VIS dataset and 30 participants, we experimentally show that MemX significantly improves the attention tracking accuracy over the eye-tracking-alone method, while maintaining high system energy efficiency. We have also conducted 11 in-field pilot studies across a range of daily usage scenarios, which demonstrate the feasibility and potential benefits of MemX. © 2021 ACM.