检索结果-内蒙古大学图书馆

Blur CLIP Context Model-Based image coding for machines Framework With Down and Up Sample Pairs for Multitasks

IEEE ACCESS 2025年 13卷 45951-45963页

作者： Zhang, Zhongpeng Liu, Ying Santa Clara Univ Dept Comp Sci & Engn Santa Clara CA 95053 USA

image coding for machines (ICM) aims to develop systems and frameworks for image compression and transmission tailored for computer vision tasks such as object detection and instance segmentation. This paper focuses on multi-task ICM not only for object detection and instance segmentation but also for image compression. In recent years, Contrastive Language-image Pre-training (CLIP) has demonstrated its powerful capability in extracting text and image features and has been applied to various vision tasks. Inspired by CLIP, channel-wise context model and mask convolutional neural network (PixelCNN), we propose a Blur CLIP context model (BCcm) for reducing bitrate usage and a two-hyperprior multitask framework augmented by BCcm (TMFBC). Through experiments, we have observed that down and up sample resize pairs can significantly reduce the bitrate usage. We integrated a plug-and-play down and up sample resize pair at both ends of TMFBC. Additionally, we propose a Resnet Pyramid Hierarchical Feature Extractor (RPHFE) based on down and up sample resize pairs to endow decoded images with richer multi-scale features. We term the aforementioned pipeline as TMFBC with down and up Sample Pairs (TMFBC-SP). We compare the proposed TMFBC-SP with state-of-the-art (SOTA) methods and demonstrate that we can achieve higher object detection and instance segmentation accuracy, measured by mean average precision (mAP), using fewer bitrates than existing approaches. We also compare ours with coarse-to-fine and context models on image compression, ours achieves higher PSNR and uses less bitrate.

关键词： image coding Feature extraction Codecs Object detection Context modeling Bit rate Instance segmentation Visualization Semantics Accuracy CLIP down sample image coding for machines instance segmentation object detection resize up sample

来源：评论

学校读者我要写书评

暂无评论

image coding for machines: AN END-TO-END LEARNED APPROACH

IMAGE CODING FOR MACHINES: AN END-TO-END LEARNED APPROACH

引用

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Le, Nam Zhang, Honglei Cricri, Francesco Ghaznavi-Youvalari, Ramin Rahtu, Esa Nokia Technol Tampere Finland Tampere Univ Tampere Finland

ISBN: (纸本)9781728176055

Over recent years, deep learning-based computer vision systems have been applied to images at an ever-increasing pace, oftentimes representing the only type of consumption for those images. Given the dramatic explosion in the number of images generated per day, a question arises: how much better would an image codec targeting machine-consumption perform against state-of-the-art codecs targeting human-consumption? In this paper, we propose an image codec for machines which is neural network (NN) based and end-to-end learned. In particular, we propose a set of training strategies that address the delicate problem of balancing competing loss functions, such as computer vision task losses, image distortion losses, and rate loss. Our experimental results show that our NN-based codec outperforms the state-of-the-art Versatile Video coding (VVC) standard on the object detection and instance segmentation tasks, achieving -37.87% and -32.90% of BD-rate gain, respectively, while being fast thanks to its compact size. To the best of our knowledge, this is the first end-to-end learned machine-targeted image codec.

关键词： image coding for machines image compression loss weighting multitask learning video coding for machines

来源：评论

学校读者我要写书评

暂无评论

image coding for machines WITH EDGE INFORMATION LEARNING USING SEGMENT ANYTHING 31

IMAGE CODING FOR MACHINES WITH EDGE INFORMATION LEARNING USI...

引用

2024 International Conference on image Processing

作者： Shindo, Takahiro Yamada, Kein Watanabe, Taiju Watanabe, Hiroshi Waseda Univ Grad Sch Fundamental Sci & Engn Tokyo Japan

ISBN: (纸本)9798350349405;9798350349399

image coding for machines (ICM) is an image compression technique for image recognition. This technique is essential due to the growing demand for image recognition AI. In this paper, we propose a method for ICM that focuses on encoding and decoding only the edge information of object parts in an image, which we call SA-ICM. This is an Learned image Compression (LIC) model trained using edge information created by Segment Anything. Our method can be used for image recognition models with various tasks. SA-ICM is also robust to changes in input data, making it effective for a variety of use cases. Additionally, our method provides benefits from a privacy point of view, as it removes human facial information on the encoder's side, thus protecting one's privacy. Furthermore, this LIC model training method can be used to train Neural Representations for Videos (NeRV), which is a video compression model. By training NeRV using edge information created by Segment Anything, it is possible to create a NeRV that is effective for image recognition (SA-NeRV). Experimental results confirm the advantages of SA-ICM, presenting the best performance in image compression for image recognition. We also show that SA-NeRV is superior to ordinary NeRV in video compression for machines. Code is available at https://***/final-0/SA-ICM.

关键词： image coding for machines ICM Segment Anything image Recognition NeRV

来源：评论

学校读者我要写书评

暂无评论

image coding for machines with Object Region Learning 21

Image Coding for Machines with Object Region Learning

引用

IEEE 21st Consumer Communications and Networking Conference (CCNC)

作者： Shindo, Takahiro Watanabe, Taiju Yamada, Kein Watanabe, Hiroshi Waseda Univ Grad Sch Fundamental Sci & Engn Tokyo Japan

ISBN: (纸本)9798350304572

Compression technology is essential for efficient image transmission and storage. With the rapid advances in deep learning, images are beginning to be used for image recognition as well as for human vision. For this reason, research has been conducted on image coding for image recognition, and this field is called image coding for machines (ICM). We propose an image compression model that learns object regions. This compression model can cleanly decode only the object regions in the image. In the experiments, we verify the effectiveness of our model by comparing it with previous methods.

关键词： image coding for machines ICM image compression object detection segmentation

来源：评论

学校读者我要写书评

暂无评论

image coding for machines with Omnipotent Feature Learning 17th

Image Coding for Machines with Omnipotent Feature Learning

引用

17th European Conference on Computer Vision (ECCV)

作者： Feng, Ruoyu Jin, Xin Guo, Zongyu Feng, Runsen Gao, Yixin He, Tianyu Zhang, Zhizheng Sun, Simeng Chen, Zhibo Univ Sci & Technol China Hefei Peoples R China Eastern Inst Adv Study Hefei Peoples R China Microsoft Res Asia Beijing Peoples R China

ISBN: (纸本)9783031198359;9783031198366

image coding for machines (ICM) aims to compress images for AI tasks analysis rather than meeting human perception. Learning a kind of feature that is both general (for AI tasks) and compact (for compression) is pivotal for its success. In this paper, we attempt to develop an ICM framework by learning universal features while also considering compression. We name such features as omnipotent features and the corresponding framework as Omni-ICM. Considering self-supervised learning (SSL) improves feature generalization, we integrate it with the compression task into the Omni-ICM framework to learn omnipotent features. However, it is non-trivial to coordinate semantics modeling in SSL and redundancy removing in compression, so we design a novel information filtering (IF) module between them by co-optimization of instance distinguishment and entropy minimization to adaptively drop information that is weakly related to AI tasks (e.g., some texture redundancy). Different from previous task-specific solutions, Omni-ICM could directly support AI tasks analysis based on the learned omnipotent features without joint training or extra transformation. Albeit simple and intuitive, Omni-ICM significantly outperforms existing traditional and learned-based codecs on multiple fundamental vision tasks.

关键词： image coding for machines Self-supervised learning Information filtering

来源：评论

学校读者我要写书评

暂无评论

Saliency Map-Guided End-to-End image coding for machines

引用

IEEE SIGNAL PROCESSING LETTERS 2024年 31卷 1755-1759页

作者： Peng, Bo Lin, Tianxiang Jin, Dengchao Pan, Zhaoqing Lei, Jianjun Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China

Existing end-to-end image coding for machines (ICM) methods generally use joint training strategies to promote the compression efficiency for machine vision without considering the influence of different regions in the image. To encourage the image compression network to focus on the regions that are critical to the subsequent visual task, this paper proposes a saliency map-guided image compression network (SMIC-Net) for ICM. Specifically, a saliency map-guided transform module (SMTM) is proposed to improve the representation ability of image features for object detection task by exploring the semantic and structural information of the detected object. Besides, a saliency map-guided mean square error (SM-MSE) loss is designed to place more emphasis on the detected object regions. Experimental results demonstrate that the proposed SMIC-Net effectively promotes the compression efficiency for machine vision.

关键词： image coding Task analysis Object detection Visualization Training Semantics Machine vision image coding for machines end-to-end image compression saliency map mean square error loss

来源：评论

学校读者我要写书评

暂无评论

Side Information Driven image coding for machines

Side Information Driven Image Coding for Machines

引用

Picture coding Symposium (PCS)

作者： Zhang, Zhongpeng Liu, Ying Santa Clara Univ Dept Comp Sci & Engn Santa Clara CA 95053 USA

ISBN: (纸本)9781665492577

With the continuous improvement of computer vision technology, more and more image information is consumed by machines rather than humans. image coding for machines (ICM) is to compress image data such that they can be more efficiently sent to the receiver side for machines to conduct visual analysis. A typical deep learning-based ICM structure contains one codec network which compresses and transmits images through the Internet and one semantic analysis task network such as image classification and object recognition. In the codec part, the side information is the hyper-prior or hierarchical layers of hyper-priors for the compression of image latent representations. In this paper, we propose a Side Information Driven image coding (SIIC) framework based on deep learning. It only compresses and transmits the side information to the receiver for image classification tasks. We obtain a top-1 accuracy of 70.38% on the imageNet1K dataset with 0.046 bits per pixel.

关键词： side information hyper-prior image classification image coding for machines transformer

来源：评论

学校读者我要写书评

暂无评论

Redundancy Removal Module for Reducing the Bitrates of image coding for machines

Redundancy Removal Module for Reducing the Bitrates of Image...

引用

IEEE International Symposium on Circuits and Systems (ISCAS)

作者： Zhang, Zhongpeng Liu, Ying Santa Clara Univ Dept Comp Sci & Engn Santa Clara CA 95053 USA

ISBN: (纸本)9798350330991;9798350331004

With the popularity of the Internet of Things (IoT) and surveillance, the amount of image and video information collected by internet has exploded. On the one hand, transmitting the massive information requires plenty of bandwidth. On the other hand, processing these data requires tremendous manpower. In order to solve these two problems at the same time, image coding for machines (ICM) came into being. At present, most ICM technologies combine traditional codecs such as BPG or machine learning codecs such as the hyperprior codec and the coarse-to-fine codec with task networks such as image classification and semantic segmentation. This process requires the complete restoration of the image, which greatly increases the bitrates. Moreover, the restored images are for human eyes and contain a large amount of redundant information, which is not required by the task network. In order to solve this problem, side information driven image coding for machines (SIIC) was proposed, which only needs to input hyperprior information to the image classification network, which greatly reduces the scale of data transmission. Now, we propose a redundancy removal module that can further reduce the usage of bitrates for SIIC. Through this module, the new codec uses 3% to 4% less bitrates than the original SIIC when the bitrates are under 0.06 bpp, and can save up to 10% of bitrates when the bitrates are over 0.1 bpp.

关键词： bitrate image classification image coding for machines redundancy removal transpose convolution

来源：评论

学校读者我要写书评

暂无评论

BRIDGING THE GAP BETWEEN image coding for machines AND HUMANS 29

BRIDGING THE GAP BETWEEN IMAGE CODING FOR MACHINES AND HUMAN...

引用

IEEE International Conference on image Processing (ICIP)

作者： Le, Nam Zhang, Honglei Cricri, Francesco Youvalari, Ramin G. Tavakoli, Hamed Rezazadegan Aksu, Emre Hannuksela, Miska M. Rahtu, Esa Nokia Technol Tampere Finland Tampere Univ Tampere Finland

ISBN: (数字)9781665496209

ISBN: (纸本)9781665496209

image coding for machines (ICM) aims at reducing the bitrate required to represent an image while minimizing the drop in machine vision analysis accuracy. In many use cases, such as surveillance, it is also important that the visual quality is not drastically deteriorated by the compression process. Recent works on using neural network (NN) based ICM codecs have shown significant coding gains against traditional methods;however, the decompressed images, especially at low bitrates, often contain checkerboard artifacts. We propose an effective decoder finetuning scheme based on adversarial training to significantly enhance the visual quality of ICM codecs, while preserving the machine analysis accuracy, without adding extra bitcost or parameters at the inference phase. The results show complete removal of the checkerboard artifacts at the negligible cost of -1.6% relative change in task performance score. In the cases where some amount of artifacts is tolerable, such as when machine consumption is the primary target, this technique can enhance both pixel-fidelity and feature-fidelity scores without losing task performance.

关键词： image coding for machines GANs finetuning VCM

来源：评论

学校读者我要写书评

暂无评论

Enhancing image coding for machines with Compressed Feature Residuals 23

Enhancing Image Coding for Machines with Compressed Feature ...

引用

23rd IEEE International Symposium on Multimedia (ISM)

作者： Seppala, Joni Zhang, Honglei Le, Nam Youvalari, Ramin G. Cricri, Francesco Tavakoli, Hamed Rezazadegan Aksu, Emre Hannuksela, Miska M. Rahtu, Esa Tampere Univ Tampere Finland Nokia Technol Tampere Finland

ISBN: (纸本)9781665437349

As computer vision technologies have tremendously improved over the last decade, videos and images are often consumed by machines instead of humans which are the main target for traditional video codecs. In many use cases, although machines are the main consumers, human involvement is also required, or even mandatory. In this paper, we propose a novel image coding technique targeted for machines, while maintaining the capability for human consumption. Our proposed codec generates two bitstreams: one bitstream from a traditional codec, referred to as human bitstream, optimized for human consumption;the other bitstream, referred to as machine bitstream, generated from an end-to-end learned neural network-based codec and optimized for machine tasks. Instead of working on the image domain, the proposed machine bitstream is derived from feature residuals - the difference between the features extracted from the input image and the features extracted from the reconstructed image generated by the traditional codec. With the help of the machine bitstream, we can significantly improve machine task performance in the low bitrate range. Our system beats the state-of-the-art traditional codec, the Versatile Video coding (VVC/H.266), achieving -40.5% in Bjontegaard delta bitrate reduction on average for bitrates up to 0.07 BPP.

关键词： image coding for machines video coding for machines feature residual image compression

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：