检索结果-内蒙古大学图书馆

DQT-CALF: Content adaptive neural network based in-loop filter in VVC using dual query transformer

NEUROCOMPUTING 2025年 637卷

作者： Liu, Yunfeng Jung, Cheolkon Xidian Univ Sch Elect Engn Xian 710071 Peoples R China

As auxiliary inputs for the neural network-based in-loop filter (NNLF) in the versatile video coding (VVC), the prediction frame, partition map and quantization parameter (QP) map are effectively used for enhancing the reconstruction frame, i.e., main input. The prediction frame and partition map play a crucial role in reducing compression artifacts, while the QP map distinguishes between inputs with different QPs. However, direct concatenation of the auxiliary inputs with the reconstruction frame may not fully leverage their advantages potentially causing conflicting effects. In this paper, we propose a content adaptive NNLF in VVC using dual query transformer (DQT), named DQT-CALF. We adopt DQT based on a dual query mechanism to effectively fuse low-frequency global features and high-frequency local features, thereby learning rich feature representations. The dual query mechanism enhances comprehensiveness and representation ability of the model, reduces the risk of information loss, and is suitable for fusion of data multiple types. DQT-CALF mainly includes three parts of feature extraction (head), feature enhancement (body), and reconstruction (tail). In feature extraction, we use a parallel network structure for the main and auxiliary inputs and assign feature maps to different weights according to the richness of the input information. For the auxiliary inputs, we do not directly concatenate them with the reconstruction frame, but generate a spatial attention map based on them to enhance the reconstruction frame. In feature enhancement, we design a multi-type feature fusion module that divides the input tensor into the low-frequency and high-frequency features from the frequency viewpoint and the local and global features from the spatial viewpoint. The two groups of features are processed separately and are mutually transformed by the high-frequency local feature generation and low-frequency global feature generation, respectively. In reconstruction, we recon

关键词： Versatile video coding Content adaptive Convolutional neural network Dual query transformer in-loop filter

来源：评论

学校读者我要写书评

暂无评论

Reference-based in-loop filter with Robust Neural Feature Transfer for Video Coding

引用

ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 2025年第1期21卷 1-24页

作者： Kim, Nayoung Webtoon, Naver Lee, Jung-kyung Kang, Je-won NAVER WEBTOON AI Seoul South Korea Ewha W Univ Dept Elect & Elect Engn Seoul South Korea Ewha W Univ Dept Elect & Elect Engn Seoul South Korea Ewha W Univ Grad Program Smart Factory Seoul South Korea

In this article, we propose an efficient reference-based deep in-loop filtering method for video coding. Existing reference-based in-loop filters often face challenges in improving coding efficiency due to the difficulty in capturing relevant textures from the reference frames. Our method accurately predicts the texture of a reference block and uses this information to restore the current block. To achieve this, we develop a reference-to-current feature estimation module that conveys high-quality information from previously coded frames in the feature domain, thereby preventing loss of detail due to inaccurate prediction. Although a neural network is trained to restore a coded video frame to be similar to the current frame, their performance can significantly degrade when operating with various quantization parameters (QPs) and managing different levels of distortion. This problem becomes further severe in the reference-to-current feature estimation, in which QP values are applied differently to video frames. We address this problem by developing a QP-aware convolution layer with a small number of learnable parameters to generate reliable features and adapt to fine-grained adaptive QPs among consecutive frames. The proposed method is implemented into the versatile video coding (VVC) reference software, VTM version 10.0. Experimental results demonstrate that the proposed method improves coding performance significantly in VVC.

关键词： Deep learning in-loop filter Versatile Video Coding reference-based in-loop filter quantization artifact removal

来源：评论

学校读者我要写书评

暂无评论

in-loop filter FOR OBJECT MASK CODING IN VERSATILE VIDEO CODING 31

IN-LOOP FILTER FOR OBJECT MASK CODING IN VERSATILE VIDEO COD...

引用

2024 International Conference on Image Processing

作者： Schwarz, Sebastian Hannuksela, Miska M. Sansli, Done Bugdayci Nokia Technol Munich Germany

ISBN: (纸本)9798350349405;9798350349399

This paper explores the challenges and solutions in compressing object mask video, which can provide additional scene context for machine learning applications. Object masks, identifying specific objects or regions in images, are crucial for precise visual analysis. However, compressing them with video codecs tuned for human consumption may lead to detrimental artefacts for machine tasks. To alleviate such artefacts, a modification to the luma mapping and chroma scaling (LMCS) in-loop filter in the Versatile Video Coding standard is proposed, targeting the reconstruction of object mask video. By performing object mask reconstruction within the encoding loop, the encoder has access to correctly reference pictures for prediction. Consequently, coding performance is significantly enhanced. Objective evaluation against post-state-of-the-art processing mask reconstruction confirms a 42.0dB and 66.1dB improvement in Y-PSNR for random access and all intra encoding, respectively, while maintaining coding complexity similar to unmodified encoding without post-processing.

关键词： video coding object masks in-loop filter VVC VCM

来源：评论

学校读者我要写书评

暂无评论

Texture and motion aware perception in-loop filter for AV1

引用

JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION 2024年 98卷

作者： Liu, Tianqi Huang, Hong Lei, Zhijun Fang, Ruogu Wu, Dapeng Univ Florida Dept Elect & Comp Engn Gainesville FL 32611 USA Meta Menlo Pk CA 94025 USA Univ Florida J Crayton Pruitt Family Dept Biomed Engn Gainesville FL 32611 USA City Univ Hong Kong Dept Comp Sci Hong Kong Peoples R China

Lossy compression introduces artifacts, and many conventional in-loop filters have been adopted in the AV1 standard to reduce these artifacts. Researchers have explored deep learning-based filters to remove artifacts in the compression loop. However, the high computational complexity of CNN-based filters remains a challenge. In this paper, a Texture-and Motion-Aware Perception (TMAP) in-loop filter is proposed to addresses this issue by selectively applying CNNs to texture-rich and high-motion regions, while utilizing non-learning methods to detect these regions. The proposed method introduces a new CNN structure, the Dense-Dual-Field Network (DDFN), which leverages a larger receptive field to enhance the quality of reconstructed frames by incorporating more contextual information. Furthermore, to improve perceptual quality, a novel loss function integrating wavelet-based perceptual information is presented. Experimental results demonstrate the superiority of our proposed models over other lightweight CNN models, and the effectiveness of the perceptual loss function is validated using the VMAF metric.

关键词： AV1 in-loop filter Lightweight model Texture and motion aware Receptive field Perceptual loss

来源：评论

学校读者我要写书评

暂无评论

Low Complexity in-loop filter for VVC Based on Convolution and Transformer

引用

IEEE ACCESS 2024年 12卷 120316-120325页

作者： Feng, Zhen Jung, Cheolkon Zhang, Hao Liu, Yang Li, Ming Xidian Univ Sch Elect Engn Xian 710071 Peoples R China Guangdong OPPO Mobile Telecommun Corp Ltd Dongguan 523860 Peoples R China

The Joint Video Experts Team (JVET) has explored neural network-based video coding (NNVC) and is trying to introduce NNVC into the versatile video coding (VVC). In NNVC, the NN-based in-loop filter is the most active area, which is very close to deployment of software. Recent NN-based in-loop filters start adopting Transformer to capture context information, but it causes a remarkable increase of complexity to about 1000 kMAC/Pixel. In this paper, we propose a low complexity NN-based in-loop filter for VVC based on convolution and Transformer, named ConvTransNet. ConvTransNet adopts a pyramid structure in feature extraction to capture both global contextual information and local details at multiple scales. Moreover, ConvTransNet combines convolutional neural network (CNN) and Transformer into the in-loop filter. CNN captures local features and reduces compression artifacts in an image, while Transformer captures long-range spatial dependency and enhances global structures in an image. Thus, ConvTransNet enables the NN-based in-loop filter to reduce compression artifacts and enhance visual quality in an image. In ConvTransNet, we use grouped convolutions in CNN and depthwise convolutions in Transformer to reduce the network complexity. Therefore, ConvTransNet successfully captures both local spatial structure and global contextual information in an image and achieves outstanding performance in terms of BD-rate and complexity. Experimental results show that the proposed NN-based in-loop filter based on ConvTransNet achieves average {6.58%, 23.02%, 23.04%} and {8.18%, 22.67%, 22.00%} BD-rate reductions for {Y, U, V} channels over VTM_11.0-NNVC_2.0 anchor under AI and RA configurations, respectively.

关键词： Transformers Convolutional neural networks Artificial neural networks Feature extraction Training Image coding Video coding Versatile video coding compression artifacts in-loop filter convolutional neural network transformer

来源：评论

学校读者我要写书评

暂无评论

RTNN: A Neural Network-Based in-loop filter in VVC Using Resblock and Transformer

引用

IEEE ACCESS 2024年 12卷 104599-104610页

作者： Zhang, Hao Liu, Yunfeng Jung, Cheolkon Liu, Yang Li, Ming Xidian Univ Sch Elect Engn Xian 710071 Peoples R China Guangdong OPPO Mobile Telecommun Corp Ltd Dongguan 523860 Peoples R China

Recently, the neural network (NN)-based in-loop filters are actively explored and being attempted to integrate in the versatile video coding (VVC). Transformer shows better global feature processing capability than convolutional neural network (CNN), which is complimentary to CNN that has advantage of local feature extraction. Since the in-loop filter operates in the coding tree unit (CTU) with fixed and relatively small size, it is very suitable for introducing Transformer into the in-loop filter in VVC. In this paper, we propose an NN-based in-loop filter in VVC using Resblock and Transformer, named RTNN, to suppress complicated compression artifacts in VVC. The proposed RTNN is an efficient filter, which combines Resblock and Transformer to extract deep features and capture local and non-local correlation between features. Moreover, a novel attention module is designed in RTNN to better refine features by introducing auxiliary information. Besides, a multi-stage training strategy is used to consider QP distance in training and maximize the learning ability of RTNN. The proposed RTNN filter is embedded into the VVC Test Model (VTM)-11.0_Neural Network-based Video Coding (NNVC)-3.0. Experimental results show that the proposed RTNN achieves average {8.49%, 22.98%, 24.07%} and {9.23%, 22.47%, 22.31%} BD-rate reductions over VTM-11.0_NNVC-3.0 for {Y, Cb, Cr} channels in AI and RA configurations, respectively, as well as yields significant performance with acceptable complexity.

关键词： Feature extraction Image restoration Training Correlation Image reconstruction Convolutional neural networks Video coding Neural networks Transformers in-loop filter compression artifact removal attention mechanism versatile video coding neural network transformer

来源：评论

学校读者我要写书评

暂无评论

Swin Transformer-based in-loop filter for VVC Intra Coding

Swin Transformer-based In-Loop Filter for VVC Intra Coding

引用

Picture Coding Symposium (PCS)

作者： Tong, Ouyang Chen, Xin Wang, Huairui Zhu, Han Chen, Zhenzhong Wuhan Univ Sch Remote Sensing & Informat Engn Wuhan Peoples R China Tencent Shanghai Peoples R China

ISBN: (纸本)9798350358483;9798350358490

As an emerging video coding standard, H.266/VVC is widely recognized for efficiently reducing bit rate and improving compression ratio. However, adopting a block-based hybrid coding framework, VVC still encounters the challenge of compression artifacts, which cause a serious impact on both subjective and objective quality. Recently, the emergence of deep learning techniques has boosted the development of neural network-based in-loop filter methods, most of which adopt convolutional neural networks (CNNs) as their backbone. However, the local receptive field of CNN limits its ability to extract global features, which leads to a performance bottleneck in the CNN-base in-loop filter. In contrast, this paper offers a swin transformer-based in-loop filter for VVC, which has a more flexible and global receptive field than CNN. Specifically, we first introduce and optimize the swin transformer-based network for the video compression task. In addition, to ensure the rate-distortion performance, a coding tree unit-level flag is designed to involve the network in the RDO decision-making process at the block level. The experimental results show that compared with VTM-11.0, the proposed swin transformer-based in-loop filter method can achieve an average of 6.77%, 14.62%, 14.97% and 6.88%, 17.24%, 17.50% Bjontegaard Delta (BD)-Bitrate savings under All-intra (AI) configurations for Y, U and V components when using PSNR and MS-SSIM as the quality metric, respectively.

关键词： swin transformer VVC in-loop filter rate-distortion performance

来源：评论

学校读者我要写书评

暂无评论

PFT-ILF: in-loop filter with Partition Feature Transform for Versatile Video Coding

PFT-ILF: In-loop Filter with Partition Feature Transform for...

引用

2024 Conference on Visual Communications and Image Processing

作者： Cui, Xin-Yi Liu, Zhikai Zhou, Zhidao Chen, Li Liang, Fan Sun Yat Sen Univ Sch Elect & Informat Technol Guangzhou Peoples R China

ISBN: (纸本)9798331529543;9798331529550

The new generation of video coding standards, Versatile Video Coding (VVC), integrates a range of loop filter mechanisms, notably the De-Blocking filter (DBF), Sample Adaptive Offset (SAO), and Adaptive loop filter (ALF). However, these traditional tools are handcrafted empirically and have limitations. Thus many CNN-based loop filters have been proposed to achieve better image quality. In this paper, we propose a novel network based on Coding Unit (CU) partition feature transformation, called PFT-ILF. Considering that the CU partition map contains image distortion information, our approach innovatively utilizes the CU partition map as prior information to guide filtering. And we design a Partition Feature Transform (PFT) layer, which uses partition features to generate modulation parameters pair for adjusting the features of several intermediate layers in the network. We integrate our proposed filter into the NNVC standard software VTM11.0_NNVC-4.0 and conduct ablation experiments. Under the all intra configuration, our method achieves Bjontegaard-Delta Bit-Rate (BD-BR) reductions of 7.52%, 17.36%, and 18.65% for Y, U, and V components, respectively.

关键词： Video coding in-loop filter convolutional neural network versatile video coding

来源：评论

学校读者我要写书评

暂无评论

A nonlocal HEVC in-loop filter using CNN-based compression noise estimation

引用

APPLIED INTELLIGENCE 2022年第15期52卷 17810-17828页

作者： Sun, Weiheng He, Xiaohai Chen, Honggang Xiong, Shuhua Xu, Yifei Sichuan Univ Coll Elect & Informat Engn Chengdu Sichuan Peoples R China

High-efficiency video coding (HEVC) effectively reduces the amount of video data while unavoidably introducing compression noise. The in-loop filter can enhance the reconstructed frames at the encoder to prevent compression noise from transmitting to the subsequent frames. The existing incorporated in-loop filters for HEVC do not fully combine the advantages of CNN and video coding priors. Moreover, obtaining the accuracy noise level is essential for adaptively in-loop filtering. To further enhance the compressed video quality in the encoder, we propose a nonlocal in-loop filter for HEVC using a CNN-based compression noise estimation network(CNEN). In the noise estimation part, we adopt a classification network to estimate the noise of compressed HEVC videos according to video content characteristics. In the denoising part, we propose a spatial-temporal nonlocal low-rank(STNLLR) prior by simultaneously exploiting the nonlocal self-similarity of video in spatial and temporal domains. We also propose an adaptive narrow quantization constraint (ANQC) prior by limiting the reconstructed pixel values adaptively according to the quantization parameters(QPs). The experimental results show that CNEN outperforms the existing compression noise estimation methods. Furthermore, our in-loop filter can improve the quality of the HEVC reconstructed frames under AI, LDP, and RA configurations, achieving an average reduction of 4.17%, 10.46%, and 6.10% in the Bjontegaard Delta Bit Rate (BDBR), respectively.

关键词： Noise estimation HEVC in-loop filter CNN

来源：评论

学校读者我要写书评

暂无评论

Group-normalized deep CNN-based in-loop filter for HEVC scalable extension

引用

SIGNAL IMAGE AND VIDEO PROCESSING 2022年第2期16卷 437-445页

作者： Dhanalakshmi, A. Nagarajan, G. Sathyabama Inst Sci & Technol Sch Elect & Elect Chennai Tamil Nadu India Panimalar Engn Coll Dept Comp & Commun Engn Chennai Tamil Nadu India Sathyabama Inst Sci & Technol Sch Comp Dept Comp Sci & Engn Chennai Tamil Nadu India

High Efficiency Video Coding (HEVC) is the recent video coding standard that can compress raw video at a higher compression state. The extension of HEVC, Scalable High Efficiency Video Coding (SHVC), also has the similar compression phenomenon of HEVC in addition to the implementation of multiple single-layer HEVC streams along with the interlayer reference modules, although the layer-based SHVC incurs more artifacts after compression compared to HEVC resulting with severe degradation in the video quality. To ease this, in-loop filter is used to remove artifacts in H.265 video coding standard. Although the artifacts will be more severe for multiple-layered codec SHVC compared to single-layer HEVC. With the development in deep learning, a group-normalized deep convolutional neural network (gDCNN) is proposed for SHVC in-loop filter to enhance the performance. Initially, the troubles that are met while modeling the traditional CNN that includes normalization, learning capability and the loss functions are examined. Following, on the basis of statistical analysis, the proposed gDCNN is introduced to remove the artifacts efficiently. It is achieved by a group-wise normalization approach, a feature extraction and fusion and a precise loss function. The simulation setting shows 4.2% BD-BR decrement with 0.46 dB increment in BD-PSNR.

关键词： SHVC in-loop filter Convolutional neural network Group-normalization PSNR Bitrate

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：