检索结果-内蒙古大学图书馆

learned image coding for Human-Machine Collaborative Optimization

IEEE TRANSACTIONS ON BROADCASTING 2025年第1期71卷 203-216页

作者： He, Jingbo He, Xiaohai Xiong, Shuhua Chen, Honggang Sichuan Univ Coll Elect & Informat Engn Chengdu 610065 Peoples R China

The exponential growth in the volume of image data has imposed immense pressure on transmission and storage systems, while simultaneously presenting opportunities for intelligent image analysis towards machine vision. Recent years, learned image coding approach have made remarkable advancements with impressive performance. The application of the learned image coding method in machine vision holds promising prospects for achieving human-machine collaboration. In this paper, we propose a learned image coding approach based on Transformer-CNN interaction structure for human-machine vision collaborative optimization, which can generate a single and compact bitstream for efficient representation in image compression. The bitstream can be directly decoded to generate a reconstructed image for human visual perception. In parallel, without the need for decoding and reconstructing the image, the bitstream can serve as input for machine vision tasks. This not only reduces computational costs on the decoding end but also enhances machine analysis efficiency. Experimental results demonstrate that our proposed learned image coding method achieves a single bitstream that concurrently considers image reconstruction and machine task analysis, ensuring high accuracy in machine tasks and superior quality in reconstructed images compared to state-of-the-art (SOTA) methods.

关键词： learned image coding compressed domain machine task collaborative optimization transformer learned image coding compressed domain machine task collaborative optimization transformer

来源：评论

学校读者我要写书评

暂无评论

Region of interest enabled learned image coding for machines 25

Region of interest enabled learned image coding for machines

引用

25th IEEE International Workshop on Multimedia Signal Processing (MMSP)

作者： Ahonen, Jukka, I Le, Nam Zhang, Honglei Cricri, Francesco Rahtu, Esa Nokia Technol Tampere Finland Tampere Univ Tampere Finland

ISBN: (纸本)9798350338935

image and video coding for machines has been recently gaining more and more interest from both the industry and the research community. One successful approach is based on end-to-end (E2E) learned compression and has shown significant gains over the state-of-the-art conventional image coding methods. However, one of the remaining challenges for such E2E-learned image codecs for machines is to adaptively allocate the bits over different regions of the image, while retaining the machine vision performance. In this paper, we propose a method that leverages Regions-Of-Interest (ROIs) for bitrate allocation within a learned image Codec (LIC) for machines. In particular, the proposed method reduces the bits allocated for the background regions of the image by reducing the variance of the elements corresponding to the background regions in the latent representation. This results in more heavily quantized background areas, while keeping the quality of the ROI areas suitable for machine tasks. The proposed method achieves significant gains, -15.80% and -22.43% Pareto BD-rate reduction, over the baseline LIC on object detection and instance segmentation tasks, respectively. To the best of our knowledge, this is the first research paper proposing an ROI-based inference-time technology for learned image coding for machines.

关键词： region of interest learned image coding video coding for machines machine vision neural networks

来源：评论

学校读者我要写书评

暂无评论

Boosting Neural image Compression for Machines Using Latent Space Masking

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2025年第4期35卷 3719-3731页

作者： Fischer, Kristian Brand, Fabian Kaup, Andre Friedrich Alexander Univ Erlangen Nurnberg Chair Multimedia Commun & Signal Proc D-91058 Erlangen Germany

Today, many image coding scenarios do not have a human as final intended user, but rather a machine fulfilling computer vision tasks on the decoded image. Thereby, the primary goal is not to keep visual quality but maintain the task accuracy of the machine for a given bitrate. Due to the tremendous progress of deep neural networks setting benchmarking results, mostly neural networks are employed to solve the analysis tasks at the decoder side. Moreover, neural networks have also found their way into the field of image compression recently. These two developments allow for an end-to-end training of the neural compression network for an analysis network as information sink. Therefore, we first roll out such a training with a task-specific loss to enhance the coding performance of neural compression networks. Compared to the standard VVC, 41.4% of bitrate are saved by this method for Mask R-CNN as analysis network on the uncompressed Cityscapes dataset. As a main contribution, we propose LSMnet, a network that runs in parallel to the encoder network and masks out elements of the latent space that are presumably not required for the analysis network. By this approach, additional 27.3% of bitrate are saved compared to the basic neural compression network optimized with the task loss. In addition, we are the first to utilize a feature-based distortion in the training loss within the context of machine-to-machine communication, which allows for a training without annotated data. We provide extensive analyses on the Cityscapes dataset including cross-evaluation with different analysis networks and present exemplary visual results.

关键词： Encoding image coding Task analysis Training Neural networks Machine-to-machine communications Bit rate Video coding for machines neural network compression computer vision learned image coding machine-to-machine communication

来源：评论

学校读者我要写书评

暂无评论

Effectiveness of learning-based image codecs on fingerprint storage 16

Effectiveness of learning-based image codecs on fingerprint ...

引用

2024 International Workshop on Information Forensics and Security

作者： Mari, Daniele Cavasin, Saverio Milani, Simone Conti, Mauro Univ Padua Padua Italy

ISBN: (纸本)9798350364439;9798350364422

The success of learning-based coding techniques and the development of learning-based image coding standards, such as JPEG-AI, point towards the adoption of such solutions in different fields, including the storage of biometric data, like fingerprints. However, the peculiar nature of learning-based compression artifacts poses several issues concerning their impact and effectiveness on extracting biometric features and landmarks, e.g., minutiae. This problem is utterly stressed by the fact that most models are trained on natural color images, whose characteristics are very different from usual biometric images, e.g, fingerprint or iris pictures. As a matter of fact, these issues are deemed to be accurately questioned and investigated, being such analysis still largely unexplored. This study represents the first investigation about the adaptability of learning-based image codecs in the storage of fingerprint images by measuring its impact on the extraction and characterization of minutiae. Experimental results show that at a fixed rate point, learned solutions considerably outperform previous fingerprint coding standards, like JPEG2000, both in terms of distortion and minutiae preservation. Indeed, experimental results prove that the peculiarities of learned compression artifacts do not prevent automatic fingerprint identification (since minutiae types and locations are not significantly altered), nor do compromise image quality for human visual inspection (as they gain in terms of BD rate and PSNR of 47.8% and +3.97dB respectively).

关键词： Fingerprints learned image coding Compression Deep Learning

来源：评论

学校读者我要写书评

暂无评论

QUANTIZED DECODER IN learned image COMPRESSION FOR DETERMINISTIC RECONSTRUCTION 49

QUANTIZED DECODER IN LEARNED IMAGE COMPRESSION FOR DETERMINI...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Koyuncu, Esin Solovyev, Timofey Sauer, Johannes Alshina, Elena Kaup, Andre Friedrich Alexander Univ Erlangen Nurnberg Erlangen Germany Huawei Technol Munich Germany

ISBN: (纸本)9798350344868;9798350344851

learned image compression has a problem of non-bit-exact reconstruction due to different calculations of floating point arithmetic on different devices. This paper shows a method to achieve a deterministic reconstructed image by quantizing only the decoder of the learned image compression model. From the implementation perspective of an image codec, it is beneficial to have the results reproducible when decoded on different devices. In this paper, we study quantization of weights and activations without overflow of accumulator in all decoder subnetworks. We show that the results are bit-exact at the output, and the resulting BD-rate loss of quantization of decoder is 0.5% in the case of 16-bit weights and 16-bit activations, and 7.9% in the case of 8-bit weights and 16-bit activations.

关键词： quantization learned image coding reproducibility

来源：评论

学校读者我要写书评

暂无评论

ON ANNOTATION-FREE OPTIMIZATION OF VIDEO coding FOR MACHINES 31

ON ANNOTATION-FREE OPTIMIZATION OF VIDEO CODING FOR MACHINES

引用

2024 International Conference on image Processing

作者： Windsheimer, Marc Brand, Fabian Kaup, Andre Friedrich Alexander Univ Erlangen Nurnberg Multimedia Commun & Signal Proc Cauerstr 7 D-91058 Erlangen Germany

ISBN: (纸本)9798350349405;9798350349399

Today, image and video data is not only viewed by humans, but also automatically analyzed by computer vision algorithms. However, current coding standards are optimized for human perception. Emerging from this, research on video coding for machines tries to develop coding methods designed for machines as information sink. Since many of these algorithms are based on neural networks, most proposals for video coding for machines build upon neural compression. So far, optimizing the compression by applying the task loss of the analysis network, for which ground truth data is needed, is achieving the best coding performance. But ground truth data is difficult to obtain and thus an optimization without ground truth is preferred. In this paper, we present an annotation-free optimization strategy for video coding for machines. We measure the distortion by calculating the task loss of the analysis network. Therefore, the predictions on the compressed image are compared with the predictions on the original image, instead of the ground truth data. Our results show that this strategy can even outperform training with ground truth data with rate savings of up to 7.5 %. By using the non-annotated training data, the rate gains can be further increased up to 8.2 %.

关键词： Video coding for Machines Neural Network Compression Computer Vision learned image coding Machine-to-Machine Communication

来源：评论

学校读者我要写书评

暂无评论

LEVERAGING REDUNDANCY IN FEATURE FOR EFFICIENT learned image COMPRESSION 49

LEVERAGING REDUNDANCY IN FEATURE FOR EFFICIENT LEARNED IMAGE...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Qin, Peng Bao, Youneng Meng, Fanyang Tan, Wen Li, Chao Wang, Genhong Liang, Yongsheng Harbin Inst Technol Shenzhen Peoples R China Peng Cheng Lab Shenzhen Peoples R China

ISBN: (纸本)9798350344868;9798350344851

In recent years, with the development of the field of learned image compression, numerous models with excellent ratedistortion performance have emerged. However, the considerable computational complexity inherent in these models poses challenges for their practical deployment. In this paper, we investigate feature redundancy in learned image compression (LIC) algorithms for efficient feature extraction and introduce an efficient and lightweight LIC framework. Specifically, we explore the existence of a large number of similar features in the network. Subsequently, we design effective feature extraction modules across various levels, such as layer and block. In addition, based on the fact that the role of the codec's encoder is to remove redundancy and the decoder is to reconstruct, we propose an asynchronous feature fusion block. This fusion block incorporates an "edge smoothing" operator in the encoder and an "edge enhancement" operator in the decoder. Our methodology strikes an ideal balance between rate-distortion performance and efficiency. The experimental results indicate that our approach necessitates only 310KMac/pixel computation and 9.5M parameters, while in terms of performance, our method achieves a 20.7% BD-rate advantage over BPG on Kodak data, mirroring VVC's performance. Compared to other learned image compression algorithms with SOTA performance, our method has a great advantage in terms of computation/parameter count.

关键词： learned image coding neural network lightweight convolution

来源：评论

学校读者我要写书评

暂无评论

A Differentiable Entropy Model for learned image Compression 22nd

A Differentiable Entropy Model for Learned Image Compression

引用

22nd International Conference on image Analysis and Processing (ICIAP)

作者： Presta, Alberto Fiandrotti, Attilio Tartaglione, Enzo Grangetto, Marco Univ Turin Comp Sci Dept Turin Italy Telecom Paris Inst Polytech Paris LTCI Palaiseau France

ISBN: (纸本)9783031431470;9783031431487

In an end-to-end learned image compression framework, an encoder projects the image on a low-dimensional, quantized, latent space while a decoder recovers the original image. The encoder and decoder are jointly trained with standard gradient backpropagation to minimize a rate-distortion (RD) cost function accounting for both distortions between the original and reconstructed image and the quantized latent space rate. State-of-the-art methods rely on an auxiliary neural network to estimate the rate R of the latent space. We propose a non-parametric entropy model that estimates the statistical frequencies of the quantized latent space during training. The proposed model is differentiable, so it can be plugged into the cost function to be minimized as a rate proxy and can be adapted to a given context without retraining. Our experiments show comparable performance with a learned rate estimator and better performance when is adapted over a temporal context.

关键词： learned image coding entropy estimation differentiable entropy autoencoder image compression

来源：评论

学校读者我要写书评

暂无评论

Opening the Black Box of learned image Coders

Opening the Black Box of Learned Image Coders

引用

Picture coding Symposium (PCS)

作者： Duan, Zhihao Lu, Ming Ma, Zhan Zhu, Fengqing Purdue Univ W Lafayette IN 47907 USA Nanjing Univ Nanjing Peoples R China

ISBN: (纸本)9781665492577

End-to-end learned lossy image coders (LICs), as opposed to hand-crafted image codecs, have shown increasing superiority in terms of the rate-distortion performance. However, they are mainly treated as black-box systems and their interpretability is not well studied. In this paper, we show that LICs learn a set of basis functions to transform input image for its compact representation in the latent space, as analogous to the orthogonal transforms used in image coding standards. Our analysis provides insights to help understand how learned image coders work and could benefit future design and development.

关键词： learned image coding transform basis linear superimposition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：