检索结果-内蒙古大学图书馆

A Non-Local Enhanced Network for Image Restoration

IEEE ACCESS 2022年 10卷 29528-29542页

作者： Huang, Yuan Hou, Xingsong Dun, Yujie Chen, Zan Qian, Xueming Xi An Jiao Tong Univ Sch Informat & Commun Engn Xian 710049 Peoples R China Zhejiang Univ Technol Coll Informat Engn Hangzhou 310014 Peoples R China

Non-local modules have been widely studied in image restoration (IR) tasks since they can learn long-range dependencies to enhance local features. However, most existing non-local modules still focus on extracting long-range dependencies within a single image or feature map. On the other hand, most IR methods simply employ a single type of non-local module in the network. A combination of various types of non-local modules to enhance local features can be more effective. In this paper, we propose a batch-wise non-local module to explore richer non-local dependencies within images. Furthermore, we combine various non-local extractors (different attention modules) with the proposed batch-wise non-local module as the Enhanced Batch-wise Non-local Attentive module (EBNA). Besides exploring richer non-local information, we build the Non-local and Local Information extracting Block (NLIB), in which we combine the EBNA with DEformable-Convolution Block (DECB) to utilize richer non-local and adaptive local information. Finally, We embed the NLIB within a U-net-like structure and build the Non-local Enhanced Network (NLENet). Extensive experiments on synthetic image denoising, real image denoising, JPEG artifacts removal, and real image super resolution tasks demonstrate that our proposed network achieves state-of-the-art performance on several IR benchmark datasets.

关键词： Feature extraction Task analysis Image restoration Image denoising Data mining Image resolution transform coding Image restoration non-local information synthetic image denoising real image denoising JPEG artifacts removal real image super resolution

来源：评论

学校读者我要写书评

暂无评论

Discrete Cosin transformer: Image Modeling From Frequency Domain 23

Discrete Cosin TransFormer: Image Modeling From Frequency Do...

引用

23rd IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

作者： Li, Xinyu Zhang, Yanyi Yuan, Jianbo Lu, Hanlin Zhu, Yibo ByteDance Beijing Peoples R China Rutgers State Univ New Brunswick NJ USA

ISBN: (纸本)9781665493468

In this paper, we propose Discrete Cosin transformer (DCFormer) that directly learn semantics from DCT-based frequency domain representation. We first show that transformer-based networks are able to learn semantics directly from frequency domain representation based on discrete cosine transform (DCT) without compromising the performance. To achieve the desired efficiency-effectiveness trade-off, we then leverage an input information compression on its frequency domain representation, which highlights the visually significant signals inspired by JPEG compression. We explore different frequency domain downsampling strategies and show that it is possible to preserve the semantic meaningful information by strategically dropping the high-frequency components. The proposed DCFormer is tested on various downstream tasks including image classification, object detection and instance segmentation, and achieves state-of-the-art comparable performance with less FLOPs, and outperforms the commonly used backbone (e.g. SWIN) at similar FLOPs. Our ablation results also show that the proposed method generalizes well on different transformer backbones.

关键词： Wavelet domain Frequency-domain analysis Computational modeling Semantics Refining transform coding Object detection

来源：评论

学校读者我要写书评

暂无评论

MW-GAN plus for Perceptual Quality Enhancement on Compressed Video

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2022年第7期32卷 4224-4237页

作者： Wang, Jianyi Xu, Mai Deng, Xin Shen, Liquan Song, Yuhang Beihang Univ Sch Elect & Informat Engn Beijing 100191 Peoples R China Beihang Univ Sch Cyber Sci & Technol Beijing 100191 Peoples R China Shanghai Univ Sch Commun & Informat Engn Shanghai 200444 Peoples R China Univ Oxford Somerville Coll Dept Comp Sci Oxford OX2 6HD England

The great success of deep learning has boosted the fast development of video quality enhancement. However, existing methods mainly focus on enhancing the objective quality of compressed video, and ignore their perceptual quality that plays a key role in determining quality of experience (QoE) of videos. In this paper, we aim at enhancing the perceptual quality of compressed video. Our main observation is that perceptual quality enhancement mostly relies on recovering the high-frequency details with fine textures. Accordingly, we propose a novel generative adversarial network (GAN) based on multi-level wavelet packet transform (WPT), which is called multi-level wavelet-based GAN+ (MW-GAN+), to exploit high-frequency details for enhancing the perceptual quality of compressed video. In MW-GAN+, we first propose a multi-level wavelet pixel-adaptive (MWP) module to extract temporal information across video frames, such that frame similarity can be utilized in recovering high-frequency details. Then, a wavelet reconstruction network, consisting of wavelet-dense residual blocks (WDRB), is developed to recover high-frequency details in a multi-level manner for enhanced frame reconstruction. Finally, we develop a 3D discriminator to encourage temporal coherence with a 3D-CNN based architecture. Experimental results demonstrate the superiority of our method over state-of-the-art methods in enhancing the perceptual quality of compressed video. Our code is available at https://***/IceClear/MW-GAN.

关键词： Image coding Measurement transform coding Image restoration Distortion Video recording Quality assessment Video perceptual quality enhancement wavelet packet transform generative adversarial network

来源：评论

学校读者我要写书评

暂无评论

transform Quantization for CNN Compression

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2022年第9期44卷 5700-5714页

作者： Young, Sean, I Zhe, Wang Taubman, David Girod, Bernd Stanford Univ Dept Elect Engn Informat Syst Lab ISL Stanford CA 94305 USA ASTAR Inst Infocomm Res Singapore 138632 Singapore Univ New South Wales Sch Elect Engn & Telecommun Sydney NSW 2052 Australia

In this paper, we compress convolutional neural network (CNN) weights post-training via transform quantization. Previous CNN quantization techniques tend to ignore the joint statistics of weights and activations, producing sub-optimal CNN performance at a given quantization bit-rate, or consider their joint statistics during training only and do not facilitate efficient compression of already trained CNN models. We optimally transform (decorrelate) and quantize the weights post-training using a rate-distortion framework to improve compression at any given quantization bit-rate. transform quantization unifies quantization and dimensionality reduction (decorrelation) techniques in a single framework to facilitate low bit-rate compression of CNNs and efficient inference in the transform domain. We first introduce a theory of rate and distortion for CNN quantization and pose optimum quantization as a rate-distortion optimization problem. We then show that this problem can be solved using optimal bit-depth allocation following decorrelation by the optimal End-to-end Learned transform (ELT) we derive in this paper. Experiments demonstrate that transform quantization advances the state of the art in CNN compression in both retrained and non-retrained quantization scenarios. In particular, we find that transform quantization with retraining is able to compress CNN models such as AlexNet, ResNet and DenseNet to very low bit-rates (1-2 bits).

关键词： Quantization (signal) transforms Kernel Decorrelation Convolution Training Image coding Convolutional neural networks transform coding compression quantization learned transforms

来源：评论

学校读者我要写书评

暂无评论

Implicitly Selected transform for AVS3

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2022年 31卷 1298-1310页

作者： Zhang, Yuhuai Zhang, Kai Zhang, Li Wang, Shanshe Gao, Wen Peking Univ Inst Digital Media Dept Elect Engn & Comp Sci Beijing 100871 Peoples R China Bytedance Inc San Diego CA 92121 USA Peking Univ Informat Technol Res & Dev Innovat Ctr Shaoxing 312000 Peoples R China Peng Cheng Lab Shenzhen 518066 Peoples R China

transform coding removes redundancy by de-correlating the residual data, which plays a crucial role in video coding. Since different transform cores can adapt to different video content and residual data, multiple cores transform has been proposed to improve the coding performance. However, the overhead for representing the transform type is inevitable. This paper intensively studies the statistical characteristics of transform coefficients. Theoretical analysis shows that the implicit method in the Implicitly Selected transform (IST) is superior to the explicit signalling method. As such, we propose the parity adjustment scheme which seamlessly cooperates with the rate-distortion optimization quantization for IST. Furthermore, we propose two combination methods, size-based and number-based, to optimize IST. Moreover, a restriction region of IST is applied to reduce the complexity. Experimental results on HPM-6.0, which is the reference software of the AVS3 video coding standard, show that our proposed method can achieve 1.76% and 0.76% BD-Rate savings on average under AI and RA configurations, respectively, along with negligible decoding time variations. The comparison results between the proposed method and explicit signalling method illustrate that the proposed method can achieve better coding gain than the latter. Our method has been adopted in AVS3.

关键词： transforms Image coding Encoding Video coding Quantization (signal) Optimization Standards AVS3 transform coding implicitly selected transform quantization

来源：评论

学校读者我要写书评

暂无评论

JPEG Robust Invertible Grayscale

引用

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2022年第12期28卷 4403-4417页

作者： Liu, Kunlin Chen, Dongdong Liao, Jing Zhang, Weiming Zhou, Hang Zhang, Jie Zhou, Wenbo Yu, Nenghai Univ Sci & Technol China Dept Elect Engn & Informat Sci Hefei 230026 Anhui Peoples R China Microsoft Res Redmond WA 98052 USA City Univ Hong Kong Dept Comp Sci Hong Kong Peoples R China

Invertible grayscale is a special kind of grayscale from which the original color can be recovered. Given an input color image, this seminal work tries to hide the color information into its grayscale counterpart while making it hard to recognize any anomalies. This powerful functionality is enabled by training a hiding sub-network and restoring sub-network in an end-to-end way. Despite its expressive results, two key limitations exist: 1) The restored color image often suffers from some noticeable visual artifacts in the smooth regions. 2) It is very sensitive to JPEG compression, i.e., the original color information cannot be well recovered once the intermediate grayscale image is compressed by JPEG. To overcome these two limitations, this article introduces adversarial training and JPEG simulator respectively. Specifically, two auxiliary adversarial networks are incorporated to make the intermediate grayscale images and final restored color images indistinguishable from normal grayscale and color images. And the JPEG simulator is utilized to simulate real JPEG compression during the online training so that the hiding and restoring sub-networks can automatically learn to be JPEG robust. Extensive experiments demonstrate that the proposed method is superior to the original invertible grayscale work both qualitatively and quantitatively while ensuring the JPEG robustness. We further show that the proposed framework can be applied under different types of grayscale constraints and achieve excellent results.

关键词： Gray-scale transform coding Image color analysis Image restoration Color Training Image coding Invertible grayscale adversarial training JPEG robust

来源：评论

学校读者我要写书评

暂无评论

A Feature-Enriched Deep Convolutional Neural Network for JPEG Image Compression Artifacts Reduction and Its Applications

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022年第1期33卷 430-444页

作者： Chen, Honggang He, Xiaohai Yang, Hong Qing, Linbo Teng, Qizhi Sichuan Univ Coll Elect & Informat Engn Chengdu 610065 Peoples R China

The amount of multimedia data, such as images and videos, has been increasing rapidly with the development of various imaging devices and the Internet, bringing more stress and challenges to information storage and transmission. The redundancy in images can be reduced to decrease data size via lossy compression, such as the most widely used standard Joint Photographic Experts Group (JPEG). However, the decompressed images generally suffer from various artifacts (e.g., blocking, banding, ringing, and blurring) due to the loss of information, especially at high compression ratios. This article presents a feature-enriched deep convolutional neural network for compression artifacts reduction (FeCarNet, for short). Taking the dense network as the backbone, FeCarNet enriches features to gain valuable information via introducing multi-scale dilated convolutions, along with the efficient 1 x 1 convolution for lowering both parameter complexity and computation cost. Meanwhile, to make full use of different levels of features in FeCarNet, a fusion block that consists of attention-based channel recalibration and dimension reduction is developed for local and global feature fusion. Furthermore, short and long residual connections both in the feature and pixel domains are combined to build a multi-level residual structure, thereby benefiting the network training and performance. In addition, aiming at reducing computation complexity further, pixel-shuffle-based image downsampling and upsampling layers are, respectively, arranged at the head and tail of the FeCarNet, which also enlarges the receptive field of the whole network. Experimental results show the superiority of FeCarNet over state-of-the-art compression artifacts reduction approaches in terms of both restoration capacity and model complexity. The applications of FeCarNet on several computer vision tasks, including image deblurring, edge detection, image segmentation, and object detection, demonstrate the effectiveness

关键词： Image coding transform coding Image restoration Feature extraction Discrete cosine transforms Costs Videos 1 x 1 convolution attention-based recalibration compression artifacts reduction feature-enriched deep network multi-scale dilated convolution pixel-shuffle-based downsampling and upsampling

来源：评论

学校读者我要写书评

暂无评论

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2022年第9期32卷 6360-6374页

作者： Mieloch, Dawid Garus, Patrick Milovanovic, Marta Jung, Joel Jeong, Jun Young Ravi, Smitha Lingadahalli Salahieh, Basel Poznan Univ Tech Inst Multimedia Telecommun PL-60965 Poznan Poland Orange Labs F-35512 Cesson Sevigne France Inst Polytech Paris Telecom Paris LTCI F-91120 Palaiseau France Tencent Media Lab Palo Alto CA 94306 USA Elect & Telecommun Res Inst Daejeon 34129 South Korea Vimmerse Inc San Jose CA 95110 USA

This paper presents the overview and rationale behind the Decoder-Side Depth Estimation (DSDE) mode of the MPEG Immersive Video (MIV) standard, using the Geometry Absent profile, for efficient compression of immersive multiview video. A MIV bitstream generated by an encoder operating in the DSDE mode does not include depth maps. It only contains the information required to reconstruct them in the client or in the cloud: decoded views and metadata. The paper explains the technical details and techniques supported by this novel MIV DSDE mode. The description additionally includes the specification on Geometry Assistance Supplemental Enhancement Information which helps to reduce the complexity of depth estimation, when performed in the cloud or at the decoder side. The depth estimation in MIV is a non-normative part of the decoding process, therefore, any method can be used to compute the depth maps. This paper lists a set of requirements for depth estimation, induced by the specific characteristics of the DSDE. The depth estimation reference software, continuously and collaboratively developed with MIV to meet these requirements, is presented in this paper. Several original experimental results are presented. The efficiency of the DSDE is compared to two MIV profiles. The combined non-transmission of depth maps and efficient coding of textures enabled by the DSDE leads to efficient compression and rendering quality improvement compared to the usual encoder-side depth estimation. Moreover, results of the first evaluation of state-of-the-art multiview depth estimators in the DSDE context, including machine learning techniques, are presented.

关键词： Estimation Decoding Encoding transform coding Standards Geometry Software Depth map immersive video video codecs video processing cloud computing

来源：评论

学校读者我要写书评

暂无评论

Video Source Characterization Using Encoding and Encapsulation Characteristics

引用

IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 2022年 17卷 3211-3224页

作者： Altinisik, Enes Sencar, Husrev Taha Tabaa, Diram Hamad Bin Khalifa Univ Qatar Comp Res Inst Doha Qatar Carnegie Mellon Univ Dept Comp Sci Doha Qatar

We introduce the use of video coding settings for source identification and propose a new approach that incorporates encoding and encapsulation aspects of a video. To this end, a joint representation of the overall file metadata is developed and used in conjunction with a two-level hierarchical classification method. At the first level, our method groups videos into metaclasses considering several abstractions that represent high-level structural properties of file metadata. This is followed by a more nuanced classification of classes that comprise each metaclass. The method is evaluated on more than 20K videos obtained by combining four public video datasets. Tests show that a balanced accuracy of 91% is achieved in correctly identifying the class of a video among 119 video classes. This corresponds to an improvement of 6.5% over the conventional approach based on video file encapsulation characteristics. Analysis performed on a large, unlabeled video set also confirmed the aptness of our approach. To further demonstrate the versatility of encoding parameters, we consider attribution of partial video files where file metadata is not available. Our results show that, even in this limited setting that is intrinsic to forensic file recovery, an identification accuracy of 57% can be achieved through the use of a subset of encoding parameters estimated from coded video data.

关键词： Encoding Metadata Cameras Streaming media transform coding Standards Media Video coding video file containers file metadata model identification

来源：评论

学校读者我要写书评

暂无评论

On the Importance of Denoising when Learning to Compress Images 23

On the Importance of Denoising when Learning to Compress Ima...

引用

23rd IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

作者： Brummer, Benoit De Vleeschouwer, Christophe Univ Louvain intoPIX Mont St Guibert Belgium Univ Louvain Louvain La Neuve Belgium

ISBN: (纸本)9781665493468

Image noise is ubiquitous in photography. However, image noise is not compressible nor desirable, thus attempting to convey the noise in compressed image bitstreams yields sub-par results in both rate and distortion. We propose to explicitly learn the image denoising task when training a codec. Therefore, we leverage the Natural Image Noise Dataset, which offers a wide variety of scenes captured with various ISO numbers, leading to different noise levels, including insignificant ones. Given this training set, we supervise the codec with noisy-clean image pairs, and show that a single model trained based on a mixture of images with variable noise levels appears to yield best-in-class results with both noisy and clean images, achieving better rate-distortion than a compression-only model or even than a pair of denoising-then-compression models with almost one order of magnitude fewer GMac operations.

关键词： Training Adaptation models Image coding Noise reduction Rate-distortion transform coding Noise measurement

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：