Task-Switchable Pre-Processor for Image Compression for Multiple Machine Vision Tasks

作     者：Yang, Mingyi Yang, Fei Murn, Luka Blanch, Marc Gorriz Sock, Juil Wan, Shuai Yang, Fuzheng Herranz, Luis 

作者机构：Xidian Univ Sch Telecommun Engn Xian Peoples R China Nankai Univ Coll Comp Sci Tianjin 300350 Peoples R China BBC Res & Dev London EC4Y 0DS England Northwestern Polytech Univ Sch Elect & Informat Xian 710072 Peoples R China RMIT Univ Sch Engn Melbourne Vic 3001 Australia Univ Autonoma Barcelona Comp Vis Ctr Barcelona 08193 Spain 

出 版 物：《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 (IEEE Trans Circuits Syst Video Technol)

年 卷 期：2024年第34卷第7期

页      面：6416-6429页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 

基　　金：National Natural Science Foundation of China [62171353, 62101409] Fundamental Research Funds for the Central Universities [JB190116] Ministry of Science, Innovation and Universities (MICINN), Spain [PID2021-128178OB-I00] Ramon y Cajal [RYC2019-027020-I] 

主　　题：Task analysis Codecs Machine vision Image coding Semantics Bit rate Feature extraction Image compression for machine vision Pre-processor Multiple tasks 

摘      要：Visual content is increasingly being processed by machines for various automated content analysis tasks instead of being consumed by humans. Despite the existence of several compression methods tailored for machine tasks, few consider real-world scenarios with multiple tasks. In this paper, we aim to address this gap by proposing a task-switchable pre-processor that optimizes input images specifically for machine consumption prior to encoding by an off-the-shelf codec designed for human consumption. The proposed task-switchable pre-processor adeptly maintains relevant semantic information based on the specific characteristics of different downstream tasks, while effectively suppressing irrelevant information to reduce bitrate. To enhance the processing of semantic information for diverse tasks, we leverage pre-extracted semantic features to modulate the pixel-to-pixel mapping within the pre-processor. By switching between different modulations, multiple tasks can be seamlessly incorporated into the system. Extensive experiments demonstrate the practicality and simplicity of our approach. It significantly reduces the number of parameters required for handling multiple tasks while still delivering impressive performance. Our method showcases the potential to achieve efficient and effective compression for machine vision tasks, supporting the evolving demands of real-world applications.