检索结果-内蒙古大学图书馆

Efficient structure-preserving superpixel segmentation based on minimum spanning tree

学校读者我要写书评

暂无评论

Efficient structure-preserving superpixel segmentation based...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Yu Bai Xuejin Chen CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System University of Science and Technology of China Hefei Anhui China

ISBN: (纸本)9781467372596

We propose a novel superpixel algorithm based on Minimum Spanning Tree (MST), to generate superpixels efficiently while strictly adhere to object boundaries. The MST, which built by gradually removing strong edges of the image graph extracted from the image, is more sensitive to image local structures. Therefore, an efficient hierarchical clustering strategy is basically employed in our algorithm to segment the input image into superpixels based on the tree distance. To gradually merge the image pixels and remove texture noises, a multi-layer scheme with different resolutions of superpixels is proposed. In each layer, the graph is constructed from the lower layer and segmented into superpixels in a linear complexity with the node number in the graph. Because the node number in each layer is exponentially reduced, the computational time of our method mainly concentrates on the first few layers, which is linear with the number of image pixels. The experimental results conducted on the Berkeley Segmentation Dataset demonstrate that our method outperforms state-of-the-art methods both in terms of structure preservation and computational efficiency.

关键词： Image segmentation Image color analysis Image edge detection Clustering algorithms Partitioning algorithms Joining processes Colored noise

Quality assessment of stereoscopic 360-degree images from multi-viewports

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Xu, Jiahua Luo, Ziyuan Zhou, Wei Zhang, Wenyuan Chen, Zhibo CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei230027 China

Objective quality assessment of stereoscopic panoramic images becomes a challenging problem owing to the rapid growth of 360-degree contents. Different from traditional 2D image quality assessment (IQA), more complex aspects are involved in 3D omnidirectional IQA, especially unlimited field of view (FoV) and extra depth perception, which brings difficulty to evaluate the quality of experience (QoE) of 3D omnidirectional images. In this paper, we propose a multi-viewport based full-reference stereo 360 IQA model. Due to the freely changeable viewports when browsing in the head-mounted display, our proposed approach processes the image inside FoV rather than the projected one such as equirectangular projection (ERP). In addition, since overall QoE depends on both image quality and depth perception, we utilize the features estimated by the difference map between left and right views which can reflect disparity. The depth perception features along with binocular image qualities are employed to further predict the overall QoE of 3D 360 images. The experimental results on our public Stereoscopic OmnidirectionaL Image quality assessment Database (SOLID) show that the proposed method achieves a significant improvement over some well-known IQA metrics and can accurately reflect the overall QoE of perceived images. Copyright © 2019, The Authors. All rights reserved.

关键词： Quality of service

Two-stream action recognition-oriented video super-resolution

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Haochen Liu, Dong Xiong, Zhiwei CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei230027 China

We study the video super-resolution (SR) problem for facilitating video analytics tasks, e.g. action recognition, instead of for visual quality. The popular action recognition methods based on convolutional networks, exemplified by two-stream networks, are not directly applicable on video of low spatial resolution. This can be remedied by performing video SR prior to recognition, which motivates us to improve the SR procedure for recognition accuracy. Tailored for two-stream action recognition networks, we propose two video SR methods for the spatial and temporal streams respectively. On the one hand, we observe that regions with action are more important to recognition, and we propose an optical-flow guided weighted mean-squared-error loss for our spatial-oriented SR (SoSR) network to emphasize the reconstruction of moving objects. On the other hand, we observe that existing video SR methods incur temporal discontinuity between frames, which also worsens the recognition accuracy, and we propose a siamese network for our temporal-oriented SR (ToSR) training that emphasizes the temporal continuity between consecutive frames. We perform experiments using two state-of-the-art action recognition networks and two well-known datasets–UCF101 and HMDB51. Results demonstrate the effectiveness of our proposed SoSR and ToSR in improving recognition accuracy. Copyright © 2019, The Authors. All rights reserved.

关键词： Optical resolving power

Learning for video compression

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Chen, Zhibo He, Tianyu Jin, Xin Wu, Feng CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System University of Science and Technology of China Hefei230027 China

One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis, binarization, etc. Experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding. Copyright © 2018, The Authors. All rights reserved.

关键词： Image compression

No-reference light field image quality assessment based on spatial-angular measurement

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Shi, Likun Zhou, Wei Chen, Zhibo CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei230027 China

Light field image quality assessment (LFI-QA) is a significant and challenging research problem. It helps to better guide light field acquisition, processing and applications. However, only a few objective models have been proposed and none of them completely consider intrinsic factors affecting the LFI quality. In this paper, we propose a No-Reference Light Field image Quality Assessment (NR-LFQA) scheme, where the main idea is to quantify the LFI quality degradation through evaluating the spatial quality and angular consistency. We first measure the spatial quality deterioration by capturing the naturalness distribution of the light field cyclopean image array, which is formed when human observes the LFI. Then, as a transformed representation of LFI, the Epipolar Plane Image (EPI) contains the slopes of lines and involves the angular information. Therefore, EPI is utilized to extract the global and local features from LFI to measure angular consistency degradation. Specifically, the distribution of gradient direction map of EPI is proposed to measure the global angular consistency distortion in the LFI. We further propose the weighted local binary pattern to capture the characteristics of local angular consistency degradation. Extensive experimental results on four publicly available LFI quality datasets demonstrate that the proposed method outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms. Copyright © 2019, The Authors. All rights reserved.

关键词： Deterioration

VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sheng, Xihua Li, Li Liu, Dong Li, Houqiang The CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei230027 China

Almost all digital videos are coded into compact representations before being transmitted. Such compact representations need to be decoded back to pixels before being displayed to humans and – as usual – before being enhanced/analyzed by machine vision algorithms. Intuitively, it is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. Therefore, we propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis, thereby being versatile for both human and machine vision. Our VNVC framework has a feature-based compression loop. In the loop, one frame is encoded into compact representations and decoded to an intermediate feature that is obtained before performing reconstruction. The intermediate feature can be used as reference in motion compensation and motion estimation through feature-based temporal context mining and cross-domain motion encoder-decoder to compress the following frames. The intermediate feature is directly fed into video reconstruction, video enhancement, and video analysis networks to evaluate its effectiveness. The evaluation shows that our framework with the intermediate feature achieves high compression efficiency for video reconstruction and satisfactory task performances with lower complexities. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep neural networks

Stereoscopic omnidirectional image quality assessment based on predictive coding theory

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Chen, Zhibo Xu, Jiahua Lin, Chaoyi Zhou, Wei CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei Anhui230027 China

Objective quality assessment of stereoscopic omnidirectional images is a challenging problem since it is influenced by multiple aspects such as projection deformation, field of view (FoV) range, binocular vision, visual comfort, etc. Existing studies show that classic 2D or 3D image quality assessment (IQA) metrics are not able to perform well for stereoscopic omnidirectional images. However, very few research works have focused on evaluating the perceptual visual quality of omnidirectional images, especially for stereoscopic omnidirectional images. In this paper, based on the predictive coding theory of the human vision system (HVS), we propose a stereoscopic omnidirectional image quality evaluator (SOIQE) to cope with the characteristics of 3D 360-degree images. Two modules are involved in SOIQE: predictive coding theory based binocular rivalry module and multi-view fusion module. In the binocular rivalry module, we introduce predictive coding theory to simulate the competition between high-level patterns and calculate the similarity and rivalry dominance to obtain the quality scores of viewport images. Moreover, we develop the multi-view fusion module to aggregate the quality scores of viewport images with the help of both content weight and location weight. The proposed SOIQE is a parametric model without necessary of regression learning, which ensures its interpretability and generalization performance. Experimental results on our published stereoscopic omnidirectional image quality assessment database (SOLID) demonstrate that our proposed SOIQE method outperforms state-of-the-art metrics. Furthermore, we also verify the effectiveness of each proposed module on both public stereoscopic image datasets and panoramic image datasets. Copyright © 2019, The Authors. All rights reserved.

关键词： Binocular vision

Online filter clustering and pruning for efficient convnets

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhou, Zhengguang Zhou, Wengang Hong, Richang Li, Houqiang CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System EEIS Department University of Science and Technology of China HeFei University of Technology

Pruning filters is an effective method for accelerating deep neural networks (DNNs), but most existing approaches prune filters on a pre-trained network directly which limits in acceleration. Although each filter has its own effect in DNNs, but if two filters are same with each other, we could prune one safely. In this paper, we add an extra cluster loss term in the loss function which can force filters in each cluster to be similar online. After training, we keep one filter in each cluster and prune others and fine-tune the pruned network to compensate the loss. Particularly, the clusters in every layer can be defined firstly which is effective for pruning DNNs within residual blocks. Extensive experiments on CIFAR10 and CIFAR100 benchmarks demonstrate the competitive performance of our proposed filter pruning method. Copyright © 2019, The Authors. All rights reserved.

关键词： Deep neural networks

A Hybrid and Explainable Deep Learning Framework for SAR Images

学校读者我要写书评

暂无评论

A Hybrid and Explainable Deep Learning Framework for SAR Ima...

IEEE International Symposium on geoscience and Remote Sensing (IGARSS)

作者： Zhongling Huang Mihai Datcu Zongxu Pan Bin Lei Key Laboratory of Technology in Geo-spatial Information Processing and Application System CAS Remote Sensing Technology Institute (IMF) German Aerospace Center (DLR)

ISBN: (数字)9781728163741

ISBN: (纸本)9781728163758

Deep learning based patch-wise Synthetic Aperture Radar (SAR) image classification usually requires a large number of labeled data for training. Aiming at understanding SAR images with very limited annotation and taking full advantage of complex-valued SAR data, this paper proposes a general and practical framework for quad-, dual-, and single-polarized SAR data. In this framework, two important elements are taken into consideration: image representation and physical scattering properties. Firstly, a convolutional neural network is applied for SAR image representation. Based on time-frequency analysis and polarimetric decomposition, the scattering labels are extracted from complex SAR data with unsupervised deep learning. Then, a bag of scattering topics for a patch is obtained via topic modeling. By assuming that the generated scattering topics can be regarded as the abstract attributes of SAR images, we propose a soft constraint between scattering topics and image representations to refine the network. Finally, a classifier for land cover and land use semantic labels can be learned with only a few annotated samples. The framework is hybrid for the combination of deep neural network and explainable approaches. Experiments are conducted on Gaofen-3 complex SAR data and the results demonstrate the effectiveness of our proposed framework.

关键词： Deep learning Training Annotations Scattering Image representation Radar polarimetry Synthetic aperture radar