检索结果-内蒙古大学图书馆

26th International Workshop on Multimedia Signal Processing

作者： Fernandez-Menduina, Samuel Pavez, Eduardo Ortega, Antonio Univ Southern Calif Dept Elect & Comp Engn Los Angeles CA 90007 USA

ISBN: (纸本)9798350387261;9798350387254

With the increasing number of images and videos consumed by computer vision algorithms, compression methods are evolving to consider both perceptual quality and performance in downstream tasks. Traditional codecs can tackle this problem by performing rate-distortion optimization (RDO) to minimize the distance at the output of a feature extractor. However, neural network non-linearities can make the rate-distortion landscape irregular, leading to reconstructions with poor visual quality even for high bit rates. Moreover, RDO decisions are made block-wise, while the feature extractor requires the whole image to exploit global information. In this paper, we address these limitations in three steps. First, we apply Taylor's expansion to the feature extractor, recasting the metric as an input-dependent squared error involving the Jacobian matrix of the neural network. Second, we make a localization assumption to compute the metric block-wise. Finally, we use randomized dimensionality reduction techniques to approximate the Jacobian. The resulting expression is monotonic with the rate and can be evaluated in the transform domain. Simulations with AVC show that our approach provides bit-rate savings while preserving accuracy in downstream tasks with less complexity than using the feature distance directly.

关键词： RDO coding for machines feature distance Jacobian rate-distortion image compression

来源：评论

学校读者我要写书评

暂无评论

Privacy-Preserving Feature coding for machines

Privacy-Preserving Feature Coding for Machines

引用

Picture coding Symposium (PCS)

作者： Azizian, Bardia Bajic, Ivan, V Simon Fraser Univ Sch Engn Sci Burnaby BC Canada

ISBN: (纸本)9781665492577

Automated machine vision pipelines do not need the exact visual content to perform their tasks. Therefore, there is a potential to remove private information from the data without significantly affecting the machine vision accuracy. We present a novel method to create a privacy-preserving latent representation of an image that could be used by a downstream machine vision model. This latent representation is constructed using adversarial training to prevent accurate reconstruction of the input while preserving the task accuracy. Specifically, we split a Deep Neural Network (DNN) model and insert an autoencoder whose purpose is to both reduce the dimensionality as well as remove information relevant to input reconstruction while minimizing the impact on task accuracy. Our results show that input reconstruction ability can be reduced by about 0.8 dB at the equivalent task accuracy, with degradation concentrated near the edges, which is important for privacy. At the same time, 30% bit savings are achieved compared to coding the features directly.

关键词： Deep neural network coding for machines privacy adversarial training feature coding

来源：评论

学校读者我要写书评

暂无评论

Learned scalable video coding for humans and machines

引用

EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING 2024年第1期2024卷 41页

作者： Hadizadeh, Hadi Bajic, Ivan V. Simon Fraser Univ Sch Engn Sci 8888 Univ Dr Burnaby BC V5A 1S6 Canada

Video coding has traditionally been developed to support services such as video streaming, videoconferencing, digital TV, and so on. The main intent was to enable human viewing of the encoded content. However, with the advances in deep neural networks (DNNs), encoded video is increasingly being used for automatic video analytics performed by machines. In applications such as automatic traffic monitoring, analytics such as vehicle detection, tracking and counting, would run continuously, while human viewing could be required occasionally to review potential incidents. To support such applications, a new paradigm for video coding is needed that will facilitate efficient representation and compression of video for both machine and human use in a scalable manner. In this manuscript, we introduce an end-to-end learnable video codec that supports a machine vision task in its base layer, while its enhancement layer, together with the base layer, supports input reconstruction for human viewing. The proposed system is constructed based on the concept of conditional coding to achieve better compression gains. Comprehensive experimental evaluations conducted on four standard video datasets demonstrate that our framework outperforms both state-of-the-art learned and conventional video codecs in its base layer, while maintaining comparable performance on the human vision task in its enhancement layer.

关键词： Video compression Video analytics Scalable coding Deep learning coding for machines

来源：评论

学校读者我要写书评

暂无评论

Privacy-Preserving Autoencoder for Collaborative Object Detection

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2024年 33卷 4937-4951页

作者： Azizian, Bardia Bajic, Ivan V. Simon Fraser Univ Sch Engn Sci Burnaby BC V5A 1S6 Canada

Privacy is a crucial concern in collaborative machine vision where a part of a Deep Neural Network (DNN) model runs on the edge, and the rest is executed on the cloud. In such applications, the machine vision model does not need the exact visual content to perform its task. Taking advantage of this potential, private information could be removed from the data insofar as it does not significantly impair the accuracy of the machine vision system. In this paper, we present an autoencoder-style network integrated within an object detection pipeline, which generates a latent representation of the input image that preserves task-relevant information while removing private information. Our approach employs an adversarial training strategy that not only removes private information from the bottleneck of the autoencoder but also promotes improved compression efficiency for feature channels coded by conventional codecs like VVC-Intra. We assess the proposed system using a realistic evaluation framework for privacy, directly measuring face and license plate recognition accuracy. Experimental results show that our proposed method is able to reduce the bitrate significantly at the same object detection accuracy compared to coding the input images directly, while keeping the face and license plate recognition accuracy on the images recovered from the bottleneck features low, implying strong privacy protection. Our code is available at https://***/bardia-az/ppa-code.

关键词： Image coding Data privacy Training Privacy Codecs Visualization Machine vision Deep neural network coding for machines privacy model inversion attack collaborative intelligence adversarial training feature compression

来源：评论

学校读者我要写书评

暂无评论

Balancing the encoder and decoder complexity in image compression for classification

引用

EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING 2024年第1期2024卷 38页

作者： Duan, Zhihao Hossain, Md Adnan Faisal He, Jiangpeng Zhu, Fengqing Purdue Univ Elmore Family Sch Elect & Comp Engn W Lafayette IN 47907 USA

This paper presents a study on the computational complexity of coding for machines, with a focus on image coding for classification. We first conduct a comprehensive set of experiments to analyze the size of the encoder (which encodes images to bitstreams), the size of the decoder (which decodes bitstreams and predicts class labels), and their impact on the rate-accuracy trade-off in compression for classification. Through empirical investigation, we demonstrate a complementary relationship between the encoder size and the decoder size, i.e., it is better to employ a large encoder with a small decoder and vice versa. Motivated by this relationship, we introduce a feature compression-based method for efficient image compression for classification. By compressing features at various layers of a neural network-based image classification model, our method achieves adjustable rate, accuracy, and encoder (or decoder) size using a single model. Experimental results on ImageNet classification show that our method achieves competitive results with existing methods while being much more flexible. The code will be made publicly available.

关键词： coding for machines Rate-accuracy-complexity Learned image compression

来源：评论

学校读者我要写书评

暂无评论

Learned Point Cloud Compression for Classification 25

Learned Point Cloud Compression for Classification

引用

25th IEEE International Workshop on Multimedia Signal Processing (MMSP)

作者： Ulhaq, Mateen Bajic, Ivan, V Simon Fraser Univ Sch Engn Sci Burnaby BC Canada

ISBN: (纸本)9798350338935

Deep learning is increasingly being used to perform machine vision tasks such as classification, object detection, and segmentation on 3D point cloud data. However, deep learning inference is computationally expensive. The limited computational capabilities of end devices thus necessitate a codec for transmitting point cloud data over the network for server-side processing. Such a codec must be lightweight and capable of achieving high compression ratios without sacrificing accuracy. Motivated by this, we present a novel point cloud codec that is highly specialized for the machine task of classification. Our codec, based on PointNet, achieves a significantly better rate-accuracy trade-off in comparison to alternative methods. In particular, it achieves a 94% reduction in BD-bitrate over non-specialized codecs on the ModelNet40 dataset. For low-resource end devices, we also propose two lightweight configurations of our encoder that achieve similar BD-bitrate reductions of 93% and 92% with 3% and 5% drops in top-1 accuracy, while consuming only 0.470 and 0.048 encoder-side kMACs/point, respectively. Our codec demonstrates the potential of specialized codecs for machine analysis of point clouds, and provides a basis for extension to more complex tasks and datasets in the future.

关键词： Point cloud compression coding for machines

来源：评论

学校读者我要写书评

暂无评论

Visual Analysis Motivated Super-Resolution Model for Image Reconstruction

Visual Analysis Motivated Super-Resolution Model for Image R...

引用

IEEE International Conference on Visual Communications and Image Processing (VCIP)

作者： Wang, Huifen Xue, Junda Yang, Mingchuan Zhang, Yuan China Telecom Res Inst Coll Big Data & Artificial Intelligence Shanghai Peoples R China China Telecom Res Inst Coll Big Data & Artificial Intelligence Beijing Peoples R China Zhejiang Univ Coll Informat Sci & Elect Engn Hangzhou Peoples R China China Telecom Res Inst Coll Big Data & Artificial Intelligence Hangzhou Peoples R China

ISBN: (纸本)9781665475921

This paper presents a concise end-to-end visual analysis motivated super-resolution model VASR for image reconstruction. Compatible with the existing machine vision feature coding framework, the features extracted from the machine vision task model are super-resolution amplified to reconstruct the original image for human vision. The experimental results show that without additional bit-streams, VASR can well complete the task of image reconstruction based on the extracted machine features, and has achieved good results on COCO, OpenImages, TVD, and DIV2K datasets.

关键词： coding for machines image reconstruction super-resolution

来源：评论

学校读者我要写书评

暂无评论

Scalable Human-Machine Point Cloud Compression

Scalable Human-Machine Point Cloud Compression

引用

Picture coding Symposium (PCS)

作者： Ulhaq, Mateen Bajic, Ivan V. Simon Fraser Univ Sch Engn Sci Burnaby BC Canada

ISBN: (纸本)9798350358483;9798350358490

Due to the limited computational capabilities of edge devices, deep learning inference can be quite expensive. One remedy is to compress and transmit point cloud data over the network for server-side processing. Unfortunately, this approach can be sensitive to network factors, including available bitrate. Luckily, the bitrate requirements can be reduced without sacrificing inference accuracy by using a machine task-specialized codec. In this paper, we present a scalable codec for point-cloud data that is specialized for the machine task of classification, while also providing a mechanism for human viewing. In the proposed scalable codec, the "base" bitstream supports the machine task, and an "enhancement" bitstream may be used for better input reconstruction performance for human viewing. We base our architecture on PointNet++, and test its efficacy on the ModelNet40 dataset. We show significant improvements over prior non-specialized codecs.

关键词： deep learning point cloud compression coding for machines scalable coding classification

来源：评论

学校读者我要写书评

暂无评论

Deep Video Codec Control for Vision Models

Deep Video Codec Control for Vision Models

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Reich, Christoph Debnath, Biplob Patel, Deep Prangemeier, Tim Cremers, Daniel Chakradhar, Srimat NEC Labs Amer Inc San Jose CA 95110 USA Tech Univ Munich Munich Germany Tech Univ Darmstadt Darmstadt Germany Munich Ctr Machine Learning MCML Munich Germany

ISBN: (纸本)9798350365474

Standardized lossy video coding is at the core of almost all real-world video processing pipelines. Rate control is used to enable standard codecs to adapt to different network bandwidth conditions or storage constraints. However, standard video codecs (e.g., H.264) and their rate control modules aim to minimize video distortion w.r.t. human quality assessment. We demonstrate empirically that standard-coded videos vastly deteriorate the performance of deep vision models. To overcome the deterioration of vision performance, this paper presents the first end-to-end learnable deep video codec control that considers both bandwidth constraints and downstream deep vision performance, while adhering to existing standardization. We demonstrate that our approach better preserves downstream deep vision performance than traditional standard video coding.

关键词： Codec coding for machines Deep Video Codec Control Deep Vision Models End-to-end Learning Rate Control Self-supervised Learning Standardization Video coding

来源：评论

学校读者我要写书评

暂无评论

Learned Multimodal Compression for Autonomous Driving 26

Learned Multimodal Compression for Autonomous Driving

引用

26th International Workshop on Multimedia Signal Processing

作者： Hadizadeh, Hadi Bajic, Ivan, V Simon Fraser Univ Burnaby BC Canada

ISBN: (纸本)9798350387261;9798350387254

Autonomous driving sensors generate an enormous amount of data. In this paper, we explore learned multimodal compression for autonomous driving, specifically targeted at 3D object detection. We focus on camera and LiDAR modalities and explore several coding approaches. One approach involves joint coding of fused modalities, while others involve coding one modality first, followed by conditional coding of the other modality. We evaluate the performance of these coding schemes on the nuScenes dataset. Our experimental results indicate that joint coding of fused modalities yields better results compared to the alternatives.

关键词： Multimodal data compression coding for machines autonomous driving object detection camera LiDAR

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：