检索结果-内蒙古大学图书馆

International Workshop on Advanced Imaging Technology (IWAIT)

作者： Lee, Jin Young Lee, HeeKyung Choo, Hyon-Gon Cheong, Won-Sik Seo, Jeongil Elect & Telecommun Res Inst Daejeon 34129 South Korea

ISBN: (数字)9781510653320

ISBN: (纸本)9781510653320;9781510653313

The coding objective of image and video that are targeted for machine consumption may differ from that for human consumption. For example, machine may only use a part of image or video requested or required by an application whereas human consumption requires whole captured area of image and video. In addition, machine may require grayscale or certain light spectrum, whereas human consumption requires full visible light spectrum. To identify an object of interest, a neural network based image or video analysis task may be performed and the output of a task is an identified feature (latent) and an associated descriptor (inference). Depending on the usage, multiple tasks can be performed in parallel or in series, and as a number of identified feature increases, the chance of feature area overlap increases as well. We propose a pipeline of descriptor based video coding for machine for multi-task. The proposed method is expected to increase coding efficiency when multiple tasks are performed, by minimizing redundant encoding of overlapped area of objects of interest and to increase utilization and re-utilization of features by transmitting inference separately.

关键词： video coding for machine machine vision task

来源：评论

学校读者我要写书评

暂无评论

A Super-Resolution-Based Feature Map Compression for machine-Oriented video coding

引用

IEEE ACCESS 2023年 11卷 34198-34209页

作者： Kang, Jung-Heum Ali, Muhammad Salman Jeong, Hye-Won Choi, Chang-Kyun Kim, Younhee Jeong, Se Yoon Bae, Sung-Ho Kim, Hui Yong Kyung Hee Univ Dept Comp Sci & Engn Yongin 17104 South Korea Elect & Telecommun Res Inst ETRI Daejeon 34129 South Korea

Recently, video and image compression methods using neural networks have received much attention. In MPEG standardization, video coding for machine (VCM) is a newly arising topic which attempts to compress features/images for the purpose of machine vision tasks. Especially, compressing features has advantages in terms of privacy protection and computation off-loading. In this paper, we propose an effective feature compression method equipped with a super-resolution (SR) module for features. Our main motivation comes from the observation that features are somewhat robust to spatial distortions (e.g., AWGN, blur, quantization distortions, coding artifacts), which leads us to integrating an SR module into the compression framework. We also further explore the best training strategy of the proposed method, i.e., finding the best combination of various losses and proper input feature shapes. Our comprehensive experiments show that the proposed method outperforms the baseline in the original VCM anchor scenario on various QP values with Versatile video coding (VVC). Specifically, the proposed framework achieved up to 50% BD-rate reduction compared to the conventional P-layer feature map compression method for the object detection task on the OpenImage dataset.

关键词： Versatile video codec video coding for machine feature compression deep neural network super resolution

来源：评论

学校读者我要写书评

暂无评论

Facial Image Compression via Neural Image Manifold Compression

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR video TECHNOLOGY 2025年第4期35卷 3812-3824页

作者： Yang, Wenhan Huang, Haofeng Liu, Jiaying Kot, Alex C. Peng Cheng Lab Shenzhen 518066 Peoples R China Nanyang Technol Univ Sch Elect & Elect Engn Singapore 639798 Singapore Peking Univ Wangxuan Inst Comp Technol Beijing 100080 Peoples R China

Although the recent learning-based image and video coding techniques achieve rapid development, the signal fidelity-driven target in these methods leads to the divergence to a highly effective and efficient coding framework for both human and machine. In this paper, we aim to address the issue by making use of the power of generative models to bridge the gap between full fidelity (for human vision) and high discrimination (for machine vision). Therefore, relying on existing pretrained generative adversarial networks (GAN), we build a GAN inversion framework that projects the image into a low-dimensional natural image manifold. In this manifold, the feature is highly discriminative and also encodes the appearance information of the image, named as latent code. Taking a variational bit-rate constraint with a hyperprior model to model/suppress the entropy of image manifold code, our method is capable of fulfilling the needs of both machine and human visions at very low bit-rates. To improve the visual quality of image reconstruction, we further propose multiple latent codes and scalable inversion. The former gets several latent codes in the inversion, while the latter additionally compresses and transmits a shallow compact feature to support visual reconstruction. Experimental results demonstrate the superiority of our method in both human vision tasks, i.e. image reconstruction, and machine vision tasks, including semantic parsing and attribute prediction.

关键词： Image coding Codes Image reconstruction Optimization Task analysis Manifolds video coding for machine generative adversarial networks GAN inversion multiple latent codes scalable GAN inversion scalable GAN inversion

来源：评论

学校读者我要写书评

暂无评论

Joint super-resolution-based fast face image coding for human and machine vision

引用

VISUAL COMPUTER 2025年第2期41卷 1395-1408页

作者： Shi, Wuzhen Tao, Fei Wen, Yang Shenzhen Univ Coll Elect & Informat Engn Guangdong Prov Engn Lab Digital Creat Technol Guangdong Prov Key Lab Intelligent Informat Proc 3688 Nanhai Ave Shenzhen Peoples R China

As the Internet of Things continues to grow and thrive, more and more data are consumed by machines for intelligent analysis. How to simultaneously support fast machine vision analysis and obtain high-quality reconstructed images to serve human vision has become a problem that needs to be solved. We propose a fast face image compression scheme based on a Joint Super-Resolution Network (JSRNet), where images are encoded hierarchically to support machine vision and human vision. To support fast machine vision tasks, we compress the downsampled version of the input image at the encoding side and implement base layer decoding at the decoding side with a lightweight super-resolution network, thus effectively reducing the codec time. To obtain high-quality reconstructed images, we design a content transfer module to transfer the visual information generated by the base layer to the enhancement layer. The joint optimization of the base layer and the enhancement layer also allows them to better cooperate to achieve better performance. We verify the performance of this scheme on the facial landmark detection task, and the experimental results show that our method saves more bit-rate and achieves more accurate facial landmark detection results with faster codec time than baselines. The reconstructed images are more in line with human visual characteristics, and the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are also improved significantly.

关键词： Scalable coding Image compression video coding for machine Super-resolution

来源：评论

学校读者我要写书评

暂无评论

HMFVC: A Human-machine Friendly video Compression Scheme

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR video TECHNOLOGY 2025年第4期35卷 3757-3769页

作者： Huang, Zhimeng Jia, Chuanmin Wang, Shanshe Ma, Siwei Peking Univ Natl Engn Res Ctr Visual Technol Sch Comp Sci Beijing 100871 Peoples R China Peking Univ Wangxuan Inst Comp Technol Beijing 100871 Peoples R China Peking Univ Informat Technol Res & Dev Innovat Ctr Shaoxing 312000 Peoples R China Peng Cheng Lab Shenzhen 518066 Peoples R China

d Recent studies on video coding for machine (VCM) have achieved remarkable results. However, in practical visual-analytic applications, except for transmitting the visual feature for machine analysis, video textures are also mandatory for human monitoring and decision-making. To this end, this paper proposes a human-machine friendly video compression scheme (HMFVC) which can satisfy both human viewing and machine analysis well. First, we propose a learned semantic representation (LSR) method to extract semantic information between temporal neighboring frames. LSR could be utilized in signal reconstruction for human viewing and visual analysis for machine understanding. Second, given the proposed LSR, we design an end-to-end optimized video compression framework to jointly optimize the visual quality for human perception, analysis accuracy for machines, and compression efficiency as well. Finally, an HMFVC codec is developed, which can achieve higher action recognition accuracy and better reconstruction quality than the traditional codecs and learned video compression approaches. Specifically, HMFVC saves 77% bitrate to achieve the same analysis performance with the original videos compared to x265. To our knowledge, HMFVC is the first end-to-end optimized video compression scheme to serve both humans and machines. It is a promising framework for human-machine friendly video compression approaches.

关键词： video compression human-machine friendly coding video coding for machine deep learning

来源：评论

学校读者我要写书评

暂无评论

video coding for machines: A Paradigm of Collaborative Compression and Intelligent Analytics

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2020年 29卷 8680-8695页

作者： Duan, Lingyu Liu, Jiaying Yang, Wenhan Huang, Tiejun Gao, Wen Peking Univ Beijing 100871 Peoples R China Peking Univ Cooperat Medianet Innovat Ctr Sch Elect Engn & Comp Sci Natl Engn Lab Video Technol Beijing 100871 Peoples R China Peking Univ Dept Comp Sci Beijing 100871 Peoples R China Peking Univ Comp Sci Beijing 100871 Peoples R China

video coding, which targets to compress and reconstruct the whole frame, and feature compression, which only preserves and transmits the most critical information, stand at two ends of the scale. That is, one is with compactness and efficiency to serve for machine vision, and the other is with full fidelity, bowing to human perception. The recent endeavors in imminent trends of video compression, e.g. deep learning based coding tools and end-to-end image/video coding, and MPEG-7 compact feature descriptor standards, i.e. Compact Descriptors for Visual Search and Compact Descriptors for video Analysis, promote the sustainable and fast development in their own directions, respectively. In this article, thanks to booming AI technology, e.g. prediction and generation models, we carry out exploration in the new area, video coding for machines (VCM), arising from the emerging MPEG standardization efforts. (1) Towards collaborative compression and intelligent analytics, VCM attempts to bridge the gap between feature coding for machine vision and video coding for human vision. Aligning with the rising Analyze then Compress instance Digital Retina, the definition, formulation, and paradigm of VCM are given first. Meanwhile, we systematically review state-of-the-art techniques in video compression and feature compression from the unique perspective of MPEG standardization, which provides the academic and industrial evidence to realize the collaborative compression of video and feature streams in a broad range of AI applications. Finally, we come up with potential VCM solutions, and the preliminary results have demonstrated the performance and efficiency gains. Further direction is discussed as well. (1) https://***/mailman/listinfo/mpeg-vcm

关键词： Image coding video coding machine learning Visualization Encoding Collaboration Transform coding video coding for machine video compression feature compression generative model prediction model

来源：评论

学校读者我要写书评

暂无评论

Towards coding for Human and machine Vision: Scalable Face Image coding

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2021年 23卷 2957-2971页

作者： Yang, Shuai Hu, Yueyu Yang, Wenhan Duan, Ling-Yu Liu, Jiaying Peking Univ Beijing 100080 Peoples R China

The past decades have witnessed the rapid development of image and video coding techniques in the era of big data. However, the signal fidelity-driven coding pipeline design limits the capability of the existing image/video coding frameworks to fulfill the needs of both machine and human vision. In this paper, we come up with a novel face image coding framework by leveraging both the compressive and the generative models, to support machine vision and human perception tasks jointly. Given an input image, the feature analysis is first applied, and then the generative model is employed to reconstruct image with compact structure and color features, where sparse edges are extracted to connect both kinds of vision and a key reference pixel selection method is proposed to determine the priorities of the reference color pixels for scalable coding. The compact edge map serves as the basic layer for machine vision tasks, and the reference pixels act as an enhanced layer to guarantee signal fidelity for human vision. By introducing advanced generative models, we train a decoding network to reconstruct images from compact structure and color representations, which is flexible to accept inputs in a scalable way and to control the imagery effect of the outputs between signal fidelity and visual realism. Experimental results and comprehensive performance analysis over the face image dataset demonstrate the superiority of our framework in both human vision tasks and machine vision tasks, which provide useful evidence on the emerging standardization efforts on MPEG VCM (video coding for machine).

关键词： Image coding machine vision Task analysis Image reconstruction Visualization Feature extraction Image color analysis Generative compression image coding scalable coding video coding for machine

来源：评论

学校读者我要写书评

暂无评论

Task-Driven video Compression for Humans and machines: Framework Design and Optimization

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2023年 25卷 8091-8102页

作者： Yi, Xiaokai Wang, Hanli Kwong, Sam Jay Kuo, C. -C. Tongji Univ Dept Comp Sci & Technol Minist Educ Key Lab Embedded Syst & Serv Comp Shanghai 200092 Peoples R China City Univ Hong Kong Dept Comp Sci Hong Kong Peoples R China Univ Southern Calif Signal & Image Proc Inst Ming Hsieh Dept Elect Engn Los Angeles CA 90089 USA

Learned video compression has developed rapidly and achieved impressive progress in recent years. Despite efficient compression performance, existing signal fidelity oriented or semantic fidelity oriented video compression methods limit the capability to meet the requirements of both machine and human vision. To address this problem, a task-driven video compression framework is proposed to flexibly support vision tasks for both human vision and machine vision. Specifically, to improve the compression performance, the backbone of the video compression framework is optimized by using three novel modules, including multi-scale motion estimation, multi-frame feature fusion, and reference based in-loop filters. Then, based on the proposed efficient compression backbone, a task-driven optimization approach is designed to achieve the trade-off between signal fidelity oriented compression and semantic fidelity oriented compression. Moreover, a post-filter module is employed for the framework to further improve the performance of the human vision branch. Finally, rate-distortion performance, rate-accuracy performance, and subjective quality are employed as the evaluation metrics, and experimental results show the superiority of the proposed framework for both human vision and machine vision.

关键词： video compression Task analysis Image coding machine vision Feature extraction Semantics Neural networks neural network video coding for machine multi-task optimization action recognition

来源：评论

学校读者我要写书评

暂无评论

Learning to Predict Object-Wise Just Recognizable Distortion for Image and video Compression

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2024年 26卷 5925-5938页

作者： Zhang, Yun Lin, Haoqin Sun, Jing Zhu, Linwei Kwong, Sam Sun Yat Sen Univ Sch Elect & Commun Engn Shenzhen 518107 Peoples R China Chinese Acad Sci Shenzhen Inst Adv Technol Shenzhen 518055 Peoples R China Lingnan Univ Dept Comp & Decis Sci Hong Kong 999077 Peoples R China

Just Recognizable Distortion (JRD) refers to the minimum distortion that notably affects the recognition performance of a machine vision model. If a distortion added to images or videos falls within this JRD threshold, the degradation of the recognition performance will be unnoticeable. Based on this JRD property, it will be useful to video coding for machine (VCM) to minimize the bit rate while maintaining the recognition performance of compressed images. In this study, we propose a deep learning-based JRD prediction model for image and video compression. We first construct a large image dataset of Object-Wise JRD (OW-JRD) containing 29 218 original images with 80 object categories, and each image was compressed into 64 distorted versions using Versatile video coding (VVC). Secondly, we analyze the distribution of the OW-JRD, formulate JRD prediction as binary classification problems and propose a deep learning-based OW-JRD prediction framework. Thirdly, we propose a deep learning based binary OW-JRD predictor to predict whether an image object is still detectable or not under different compression levels. Also, we propose an error-tolerance strategy that corrects misclassifications from the binary classifier. Finally, extensive experiments on large JRD image datasets demonstrate that the Mean Absolute Errors (MAEs) of the predicted OW-JRD are 4.90 and 5.92 on different numbers of the classes, which is significantly better than the state-of-the-art JRD prediction model. Moreover, ablation studies on deep network structures, object sizes, features, data padding strategies and image/video coding schemes are presented to validate the effectiveness of the proposed JRD model.

关键词： Deep learning just recognizable distortion object detection video coding for machine

来源：评论

学校读者我要写书评

暂无评论

Sketch Assisted Face Image coding for Human and machine Vision: A Joint Training Approach

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR video TECHNOLOGY 2023年第10期33卷 6086-6100页

作者： Fang, Xin Duan, Yiping Du, Qiyuan Tao, Xiaoming Li, Fan Jiaotong Univ Sch Informat & Commun Engn Shaanxi Key Lab Deep Space Explorat Intelligent In Xian 710049 Peoples R China Tsinghua Univ Dept Elect Engn Beijing 100084 Peoples R China Beijing Natl Res Ctr Informat Sci & Technol BNRist Beijing 100084 Peoples R China

Image coding is one of the most fundamental techniques and is widely used in image/video processing and multimedia communications. Current image coding methods are mainly human-oriented, and the visual quality is always unsatisfactory, especially at low bitrates. Moreover, the recent emergence of machine vision goes beyond the scope of current coding. With these considerations, we proposed a sketch assisted face image coding for human and machine vision by a joint training approach. In the proposed approach, we design a new feature representation: a color sketch, which aims to satisfy both low-frequency features of human vision and high-frequency features of machine analysis. Then, we present a novel end-to-end image codec framework with joint training that consists of three models: an image-to-image translation module, a coding module, and a two-stage reconstruction module. Specifically, the input image is first translated into the edge map with the Canny edge as the auxiliary label to merely preserve the structure information. Afterward, the backpropagation from reconstruction module guides the edge map to increase or decrease the information through joint training, which results in the generation of color sketch. Then, the generated sketch is compressed into the bitstream and decompressed back to a sketch in the coding module. Finally, the decompressed sketch is reconstructed to support the machine and human tasks, respectively. In this way, the color sketch is designed to bridge the gap between human and machine vision, and the joint training strategy helps to adjust the low-frequency information in the sketch. The experimental results on challenge datasets demonstrate that our proposed algorithm offers 40.9%-86.6% bitrate savings on machine vision and is comparable to state-of-the-art image coding methods on human vision.

关键词： Image coding Image reconstruction Task analysis Image edge detection Training Feature extraction machine vision video coding for machine image compression deep neural network

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：