检索结果-内蒙古大学图书馆

M-LVC: Multiple frames prediction for learned video compression

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Lin, Jianping Liu, Dong Li, Houqiang Wu, Feng Cas Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei230027

We propose an end-to-end learned video compression scheme for low-latency scenarios. Previous methods are limited in using the previous one frame as reference. Our method introduces the usage of the previous multiple frames as references. In our scheme, the motion vector (MV) field is calculated between the current frame and the previous one. With multiple reference frames and associated multiple MV fields, our designed network can generate more accurate prediction of the current frame, yielding less residual. Multiple reference frames also help generate MV prediction, which reduces the coding cost of MV field. We use two deep auto-encoders to compress the residual and the MV, respec-tively. To compensate for the compression error of the autoencoders, we further design a MV refinement network and a residual refinement network, taking use of the multiple ref-erence frames as well. All the modules in our scheme are jointly optimized through a single rate-distortion loss func-tion. We use a step-by-step training strategy to optimize the entire scheme. Experimental results show that the proposed method outperforms the existing learned video compression methods for low-latency mode. Our method also performs better than H.265 in both PSNR and MS-SSIM. Our code and models are publicly available. Copyright © 2020, The Authors. All rights reserved.

关键词： Image compression

Deep Local and Global Spatiotemporal Feature Aggregation for Blind Video Quality Assessment

学校读者我要写书评

暂无评论

Deep Local and Global Spatiotemporal Feature Aggregation for...

IEEE Visual Communications and Image processing (VCIP)

作者： Wei Zhou Zhibo Chen CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei China

ISBN: (数字)9781728180687

ISBN: (纸本)9781728180694

In recent years, deep learning has achieved promising success for multimedia quality assessment, especially for image quality assessment (IQA). However, since there exist more complex temporal characteristics in videos, very little work has been done on video quality assessment (VQA) by exploiting powerful deep convolutional neural networks (DCNNs). In this paper, we propose an efficient VQA method named Deep SpatioTemporal video Quality assessor (DeepSTQ) to predict the perceptual quality of various distorted videos in a no-reference manner. In the proposed DeepSTQ, we first extract local and global spatiotemporal features by pre-trained deep learning models without fine-tuning or training from scratch. The composited features consider distorted video frames as well as frame difference maps from both global and local views. Then, the feature aggregation is conducted by the regression model to predict the perceptual video quality. Finally, experimental results demonstrate that our proposed DeepSTQ outperforms state-of-the-art quality assessment algorithms.

关键词： Quality assessment Video recording Feature extraction Spatiotemporal phenomena Databases Indexes Deep learning

Improving Compression Artifact Reduction via End-to-End Learning of Side information

学校读者我要写书评

暂无评论

Improving Compression Artifact Reduction via End-to-End Lear...

IEEE Visual Communications and Image processing (VCIP)

作者： Haichuan Ma Dong Liu Feng Wu CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei China

ISBN: (数字)9781728180687

ISBN: (纸本)9781728180694

We propose to improve neural network-based compression artifact reduction by transmitting side information for the neural network. The side information consists of artifact descriptors that are obtained by analyzing the original and compressed images in the encoder. In the decoder, the received descriptors are used as additional input to a well-designed conditional post-processing neural network. To reduce the transmission overhead, the entire model is optimized under the rate-distortion constraint via end-to-end learning. Experimental results show that introducing the side information greatly improves the ability of the post-processing neural network, and improves the rate-distortion performance.

关键词： Image coding Neural networks Decoding Training Feature extraction Computational modeling Transform coding

Deep Learning-Based Nonlinear Transform for HEVC Intra Coding

学校读者我要写书评

暂无评论

Deep Learning-Based Nonlinear Transform for HEVC Intra Codin...

IEEE Visual Communications and Image processing (VCIP)

作者： Kun Yang Dong Liu Feng Wu CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei China

ISBN: (数字)9781728180687

ISBN: (纸本)9781728180694

In the hybrid video coding framework, transform is adopted to exploit the dependency within the input signal. In this paper, we propose a deep learning-based nonlinear transform for intra coding. Specifically, we incorporate the directional information into the residual domain. Then, a convolutional neural network model is designed to achieve better decorrelation and energy compaction than the conventional discrete cosine transform. This work has two main contributions. First, we propose to use the intra prediction signal to reduce the directionality in the residual. Second, we present a novel loss function to characterize the efficiency of the transform during the training. To evaluate the compression performance of the proposed transform, we implement it into the High Efficiency Video Coding reference software. Experimental results demonstrate that the proposed method achieves up to 1.79% BD-rate reduction for natural videos.

关键词： Transforms Discrete cosine transforms Neural networks Transform coding Video coding Image coding Decoding

Improving resistance to adversarial deformations by regularizing gradients

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Xia, Pengfei Li, Bin CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application Systems Department of Electronic Engineering and Information Science University of Science of Technology of China Hefei China

Improving the resistance of deep neural networks against adversarial attacks is important for deploying models to realistic applications. Currently, most defense methods are designed to defend against additive noise attacks, their performance cannot be guaranteed when against non-additive noise attacks. In this paper, we focus on adversarial deformations, a typical class of non-additive noise attacks, and propose a flow gradient regularization with random start to improve the resistance of models. Theoretically, we prove that, compared with input gradient regularization, regularizing flow gradients is able to get a tighter bound. Across multiple datasets, architectures, and adversarial deformations, our experimental results consistently indicate that models trained with flow gradient regularization can acquire a better resistance than trained with input gradient regularization with a large margin. Moreover, compared with adversarial training, our method can achieve better results in optimization-based and gradient-free attacks, and combining these two methods can improve the resistance against deformation attacks further. Finally, we give a unified form of gradient regularization, which can be used to derive the corresponding form when facing other types of attack. Copyright © 2020, The Authors. All rights reserved.

关键词： Deformation

Chain Code-Based Occupancy Map Coding for Video-Based Point Cloud Compression

学校读者我要写书评

暂无评论

Chain Code-Based Occupancy Map Coding for Video-Based Point ...

IEEE Visual Communications and Image processing (VCIP)

作者： Runyu Yang Ning Yan Li Li Dong Liu Feng Wu CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei China

ISBN: (数字)9781728180687

ISBN: (纸本)9781728180694

In video-based point cloud compression (V-PCC), occupancy map video is utilized to indicate whether a 2-D pixel corresponds to a valid 3-D point or not. In the current design of V-PCC, the occupancy map video is directly compressed losslessly with High Efficiency Video Coding (HEVC). However, the coding tools in HEVC are specifically designed for natural images, thus unsuitable for the occupancy map. In this paper, we present a novel quadtree-based scheme for lossless occupancy map coding. In this scheme, the occupancy map is firstly divided into several coding tree units (CTUs). Then, the CTU is divided into coding units (CUs) recursively using a quadtree. The quadtree partition is terminated when one of the three conditions is satisfied. Firstly, all the pixels have the same value. Secondly, the pixels in the CU only have two kinds of values and they can be separated by a continuous edge whose endpoints lie on the side of the CU. The continuous edge is then coded using chain code. Thirdly, the CU reaches the minimum size. This scheme simplifies the design of block partitioning in HEVC and designs simpler yet more effective coding tools. Experimental results show significant reduction of bit-rate and complexity compared with the occupancy map coding scheme in V-PCC. In addition, this scheme is also very efficient to compress the semantic map.

关键词： Encoding Three-dimensional displays Image coding Semantics Markov processes geometry Copper

Improving Semantic Segmentation via Label Propagation and Temporal Consistency

学校读者我要写书评

暂无评论

Improving Semantic Segmentation via Label Propagation and Te...

2019 IEEE International Conference on Signal, information and Data processing, ICSIDP 2019

作者： Qin, Feiyu Cao, Lumeng Chen, Xuejin CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System University of Science and Technology of China Hefei China

ISBN: (纸本)9781728123455

Semantic segmentation is a fundamental task in indoor scene understanding. Most previous supervised approaches rely on densely annotated image data sets. Due to the limited amount of images with segmentation labels, the performance of existing networks is greatly limited. In this paper, we exploit temporal correlation in video frames to improve the performance and robustness of segmentation networks. Two effective learning strategies are proposed to propagate the information from a few labeled frames to their immediate neighbor frames. First, we scale up training dataset for supervised semantic segmentation networks by generating pseudo ground-truth for neighboring frames from a labeled frame using filtered homography transformation. Furthermore, we introduce a self-supervised loss function to ensure temporal consistency between the segmentation results of adjacent frames. The experimental results demonstrate that our proposed method outperforms state-of-the-art techniques for semantic segmentation on NYU-Depth V2 dataset. © 2019 IEEE.

关键词： Semantic Segmentation

TomoSAR-ALISTA: Efficient TomoSAR Imaging via Deep Unfolded Network

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Muhan Zhang, Zhe Wang, Yue Gao, Silin Qiu, Xiaolan Key Laboratory of Technology in Geo-spatial Information Processing and Application System Chinese Academy of Sciences Beijing100190 China Key Laboratory of Intelligent Aerospace Big Data Application Technology Suzhou215123 China Suzhou Aerospace Information Research Institute Suzhou215123 China School of Electronic Electrical and Communication Engineering University of Chinese Academy of Sciences Beijing100049 China Aerospace Information Research Institute Chinese Academy of Sciences Beijing100094 China Electrical and Computer Engineering Department George Mason University FairfaxVA22030 United States

Synthetic aperture radar (SAR) tomography (TomoSAR) has attracted remarkable interest for its ability in achieving three-dimensional reconstruction along the elevation direction from multiple observations. In recent years, compressed sensing (CS) technique has been introduced into TomoSAR considering for its super-resolution ability with limited samples. Whereas, the CS-based methods suffer from several drawbacks, including weak noise resistance, high computational complexity and complex parameter fine-tuning. Among the different CS algorithms, iterative soft-thresholding algorithm (ISTA) is widely used as a robust reconstruction approach, however, the parameters in the ISTA algorithm are manually chosen, which usually requires a time-consuming fine-tuning process to achieve the best performance. Aiming at efficient TomoSAR imaging, a novel sparse unfolding network named analytic learned ISTA (ALISTA) is proposed towards the TomoSAR imaging problem in this paper, and the key parameters of ISTA are learned from training data via deep learning to avoid complex parameter fine-tuning and significantly relieves the training burden. In addition, experiments verify that it is feasible to use traditional CS algorithms as training labels, which provides a tangible supervised training method to achieve better 3D reconstruction performance even in the absence of labeled data in real applications. Copyright © 2022, The Authors. All rights reserved.

关键词： Compressed sensing

MSight: An Edge-Cloud Infrastructure-based Perception System for Connected Automated Vehicles

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Rusheng Meng, Depu Shen, Shengyin Zou, Zhengxia Li, Houqiang Liu, Henry X. The Department of Civil and Environmental Engineering University of Michigan Ann ArborMI48109 United States The University of Michigan Transportation Research Institude 2901 Baxer Rd Ann ArborMI48109 United States The Department of Guidance Navigation and Control School of Astronautics Beihang University Beijing100191 China The CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei China Mcity University of Michigan Ann ArborMI48109 United States

As vehicular communication and networking technologies continue to advance, infrastructure-based roadside perception emerges as a pivotal tool for connected automated vehicle (CAV) applications. Due to their elevated positioning, roadside sensors, including cameras and lidars, often enjoy unobstructed views with diminished object occlusion. This provides them a distinct advantage over onboard perception, enabling more robust and accurate detection of road objects. This paper presents MSight, a cutting-edge roadside perception system specifically designed for CAVs. MSight offers real-time vehicle detection, localization, tracking, and short-term trajectory prediction. Evaluations underscore the system’s capability to uphold lane-level accuracy with minimal latency, revealing a range of potential applications to enhance CAV safety and efficiency. Presently, MSight operates 24/7 at a two-lane roundabout in the City of Ann Arbor, Michigan. Copyright © 2023, The Authors. All rights reserved.

关键词： Roadsides