FVV Live: A Real-Time Free-Viewpoint Video System With Consumer Electronics Hardware

作     者：Carballeira, Pablo Carmona, Carlos Diaz, Cesar Berjon, Daniel Corregidor, Daniel Cabrera, Julian Moran, Francisco Doblado, Carmen Arnaldo, Sergio Martin, Maria del Mar Garcia, Narciso 

作者机构：Univ Politecn Madrid Grp Tratamiento Imagenes Madrid Spain Univ Autonoma Madrid Escuela Politecn Super Video Proc & Understanding Lab Madrid 28049 Spain Univ Politecn Madrid Grp Tratamiento Imagenes Informat Proc & Telecommun Ctr Madrid 28040 Spain Univ Politecn Madrid ETSI Telecomunicac Madrid 28040 Spain 

出 版 物：《IEEE TRANSACTIONS ON MULTIMEDIA》 (IEEE Trans Multimedia)

年 卷 期：2022年第24卷

页      面：2378-2391页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Ministerio de Ciencia, Innovacion y Universidades (AEI/FEDER) of the Spanish Government [TEC2016-75981] Huawei Technologies Company, Ltd European Union [H2020-957102] 

主　　题：Cameras Streaming media Real-time systems Rendering (computer graphics) Bit rate Hardware Servers Visual communications free-viewpoint video consumer electronics multiview video depth coding depth image-based rendering subjective assessment 

摘      要：FVV Live is a novel end-to-end free-viewpoint video system, designed for real-time operation, using consumer-grade cameras and hardware, which enables low deployment costs and easy installation for immersive event-broadcasting or videoconferencing. All the blocks of the system have been designed to maximize perceptual video quality, overcoming the limitations imposed by hardware and network, which impact directly the accuracy of depth data and thus the quality of virtual view synthesis. Therefore, it does not sacrifice perceptual video quality with respect to high-end counterparts. The results presented in this paper correspond to an implementation with nine stereo-based depth cameras. However, the design of the acquisition block of FVV Live allows scalability for an arbitrary number of cameras. In addition, FVV Live presents low motion-to-photon and end-to-end delays, which enables a responsive free-viewpoint navigation and bilateral immersive communications. Moreover, the visual quality of FVV Live has been assessed through subjective assessment with satisfactory results, and additional comparative tests show that it is preferred over state-of-the-art DIBR alternatives.