generative face video coding (GFVC) can achieve high-quality visual face communication at ultra-low bit-rate ranges via strong facial prior learning and realistic generation. However, different kinds of feature repres...
详细信息
ISBN:
(纸本)9798350385885;9798350385878
generative face video coding (GFVC) can achieve high-quality visual face communication at ultra-low bit-rate ranges via strong facial prior learning and realistic generation. However, different kinds of feature representations hinder the interoperability of GFVC, as the bitstream generated from one type of feature representation can only be correctly understood by the corresponding decoder. In this paper, we make the first attempt to propose a face feature transcoding framework that enables translatability in GFVC. By integrating a face feature transcoder at the decoder side, received face features can be translated to decoder-specific ones for subsequent face reconstruction. Furthermore, the translation between different types of face features can be achieved using a unified transcoding framework, facilitating seamless interoperability between different facial representations and their associated decoders. Experimental results demonstrate that three main-stream GFVC codecs, each utilizing different face features, can be effectively adapted to one another while retaining promising coding performance, largely extending the generality of the GFVC system. The project page can be found at https://***/xyzysz/GFVC_Software-Decoder_Interoperability.
Recently, generative Face Video Compression (GFVC) has advanced the concept of Model-based coding (MBC) with promising rate-distortion performance relying on the strong inference capabilities of deep generative models...
详细信息
ISBN:
(纸本)9798350387261;9798350387254
Recently, generative Face Video Compression (GFVC) has advanced the concept of Model-based coding (MBC) with promising rate-distortion performance relying on the strong inference capabilities of deep generative models. In particular, GFVC can capture temporal evolution of face video using compact representations (i.e., 2D/3D key-points, facial semantics, compact feature), thus achieving the quality and bandwidth trade-offs for ultra-low bit-rate communication. However, there remains an unaddressed challenge, i.e., the existing GFVC models are not light-weighted and low-latency enough for practical applications. To address these obstacles, this paper proposes a practical lightweight scheme based on the Compact Feature Temporal Evolution (CFTE) model, which aims to provide insights into practical deployments and efficient inference. Specifically, the lightweight network architecture is built with depth-wise convolutions and Inverted Residual Blocks to lower the computational complexity. Moreover, a feature-level knowledge distillation is further introduced to improve the performance of lightweight student CFTE model. Experimental results demonstrate that our proposed lightweight GFVC model can achieve an obvious complexity reduction, whilst maintaining competitive rate-distortion performance.
Intelligent video coding (IVC), which dates back to the late 1980s with the concept of encoding videos with knowledge and semantics, includes visual content compact representation models and methods enabling structura...
详细信息
Much qualitative research produces little new knowledge. We argue that this is largely due to deficits of analysis. Researchers too seldom venture beyond cataloguing data into pre-existing concepts and scouting for &q...
详细信息
Much qualitative research produces little new knowledge. We argue that this is largely due to deficits of analysis. Researchers too seldom venture beyond cataloguing data into pre-existing concepts and scouting for "themes," and fail to exploit the distinctive powers of insight of qualitative methodology. The paper introduces a "value-adding" approach to qualitative analysis that aims to extend and enrich researchers' analytic interpretive practices and enhance the worth of the knowledge generated. We outline key features of this form of analysis, including how it is constituted by principles of interpretation, contextualization, criticality, and the "creative presence" of the researcher. Using concrete examples from our own research, we describe some analytic "devices" that can free up and stretch a researcher's analytic capacities, including putting reflexivity to work, treating everything as data, reading data for what is invisible, anomalous and "gestalt," engaging in "generative" coding, deploying heuristics for theorizing, and recognizing writing as a key analytic activity. We argue that at its core, value-adding analysis is a scientific craft rather than a scientific formula, a creative assemblage of reality rather than a procedural determination of it. The researcher is the primary generative and synthesizing mechanism for transforming empirically observed data into the key products of qualitative research-concepts, accounts and explanations. The ultimate value of value-adding analysis resides in its ability to generate new knowledge, including not just the "discovery" of things heretofore unknown but also the re-conceptualization of what is already known, and, importantly, the reframing and reconstitution of the research problem.
This thesis aims to study the usage of computer generated imagery in VJ Performance. The main workflow of VJ Practices has been divided into content, process, and output. The project includes the creation of computer ...
详细信息
This thesis aims to study the usage of computer generated imagery in VJ Performance. The main workflow of VJ Practices has been divided into content, process, and output. The project includes the creation of computer generated visual material via generative/creative coding defined as content, VJ performance with these materials defined as process, and presentation of visuals via Video Projection Mapping Technology defined as output.
暂无评论