The article focuses on the audio and video analysis for multimedia interactive services. It describes a system that automates home video editing. It automatically extracts a set of highlight segments from a set of raw...
详细信息
The article focuses on the audio and video analysis for multimedia interactive services. It describes a system that automates home video editing. It automatically extracts a set of highlight segments from a set of raw home videos and aligns them with user-supplied incidental music based on the content of the video and incidental music. Finally, it introduces a method for interactive image retrieval using query feedback. It learns the user query as well as the correspondence between high-level user concepts and their low-level machine representation by performing retrievals according to multiple queries supplied by the user during the course of a retrieval session.
作者:
Li, ZDelp, EJPurdue Univ
Video & Image Proc Lab VIPER Sch Elect & Comp Engn W Lafayette IN 47907 USA
In this paper we present a statistical analysis of motion prediction with drift in video coding. The drift effect occurs when the video decoder does not have access to the same reference information used in the encode...
详细信息
ISBN:
(纸本)0819452114
In this paper we present a statistical analysis of motion prediction with drift in video coding. The drift effect occurs when the video decoder does not have access to the same reference information used in the encoder in a hybrid video codec using motion prediction. Although the drift effect has been known to the video research community for a long time, there has not been a systematic theoretical treatment of this mechanism. Generally the performance of motion prediction with drift is evaluated experimentally. In this paper we derive a closed-form expression for the drift error. Based on this result. we derive an efficient rate distortion optimization given the statistical knowledge of the channel.
Scalable video coding via motion-compensated spatio-temporal wavelet decompositions has gained a great interest in transmission over heterogeneous networks due to the flexibility of the resulting bitstream to accomoda...
详细信息
ISBN:
(纸本)0819452114
Scalable video coding via motion-compensated spatio-temporal wavelet decompositions has gained a great interest in transmission over heterogeneous networks due to the flexibility of the resulting bitstream to accomodate various network conditions as well as user capabilities and demands. Meanwhile, the adaptation of the video bitstream to the available bandwidth can lead to discarding the finest detail subbands during the transmission. The loss of these subbands would result in a low quality, oversmoothed reconstructed sequence. In this paper, we present a statistical spatio-temporal model between the wavelet coefficients and we show its efficiency in the prediction of the high frequency subbands and in the quality enhancement of the scalable video.
In recent years, deep learning has achieved significant progress in many respects. However, unlike other research fields with millions of labeled data such as image recognition, only several thousand labeled images ar...
详细信息
ISBN:
(纸本)9781728185514
In recent years, deep learning has achieved significant progress in many respects. However, unlike other research fields with millions of labeled data such as image recognition, only several thousand labeled images are available in image quality assessment (IQA) field for deep learning, which heavily hinders the development and application for IQA. To tackle this problem, in this paper, we proposed an error self-learning semi-supervised method for no-reference (NR) IQA (ESSIQA), which is based on deep learning. We employed an advanced full reference (FR) IQA method to expand databases and supervise the training of network. In addition, the network outputs of expanding images were used as proxy labels replacing errors between subjective scores and objective scores to achieve error self-learning. Two weights of error back propagation were designed to reduce the impact of inaccurate outputs. The experimental results show that the proposed method yielded comparative effect.
With the rapid development of multi-sensor fusion technology in various industrial fields, many composite images closely related to human life have been produced. To meet the rapidly growing needs of various image-bas...
详细信息
ISBN:
(纸本)9781665475921
With the rapid development of multi-sensor fusion technology in various industrial fields, many composite images closely related to human life have been produced. To meet the rapidly growing needs of various image-based applications, we have established the first multi-source composite image (MSCI) database for image quality assessment (IQA). Our MSCI database contains 80 reference images and 1600 distorted images, generated by four advanced compression standards with five distortion levels. In particular, these five distortion levels are determined based on the first five just noticeable difference (JND) levels. Moreover, we verify the IQA performance of some representative methods on our MSCI database. The experimental results show that the performance of the existing methods on the MSCI database needs to be further improved.
Increasing the spatial resolution and frame rate of a video simultaneously has attracted attention in recent years. The current one-stage space-time video super-resolution (STVSR) methods are difficult to deal with la...
详细信息
ISBN:
(纸本)9781728185514
Increasing the spatial resolution and frame rate of a video simultaneously has attracted attention in recent years. The current one-stage space-time video super-resolution (STVSR) methods are difficult to deal with large motion and complex scenes, and are time-consuming and memory intensive. We propose an efficient STVSR framework, which can correctly handle complicated scenes such as occlusion and large motion and generate results with clearer texture. In REDS dataset, our method outperforms all existing one-stage methods. Our method is lightweight and can generate 720p frames at 16fps on a NVIDIA GTX 1080 Ti GPU.
Pixel-wise image quality assessment (IQA) algorithms, such as mean square error (MSE), mean absolute error (MAE) and peak signal-to-noise ratio (PSNR) correlate well with perceptual quality when dealing with images sh...
详细信息
ISBN:
(纸本)9781728180687
Pixel-wise image quality assessment (IQA) algorithms, such as mean square error (MSE), mean absolute error (MAE) and peak signal-to-noise ratio (PSNR) correlate well with perceptual quality when dealing with images sharing the same distortion type but not well when processingimages in different distortion types, which is inconsistent with human visual system (HVS). Although a large number of metrics based on image error has been proposed, there are still difficulties and limitations. To solve this problem, a full reference image quality assessment (FR-IQA) method based on MAE is proposed in this paper. The metric divides the image error (difference between distorted image and reference image) map into smooth region and texture-edge region, calculates their mean values respectively, and then gives them different weights considering the masking effect. The key innovation of this paper is to propose a distortion significance measurement, which is a visual quality coefficient that can effectively indicate the influence of different distortion types on perceptual quality and unify them with HVS. The segmented image error maps are weighted by the distortion significance coefficient. The experimental results on four largest benchmark databases show that the most of the distortions are successfully evaluated and the results are consistent with HVS.
The lack of eye contact in video conference degrades the user experience. This problem has been known and studied for many years. There are hardware-based solutions to the eye gaze problem;however, these specialized s...
详细信息
ISBN:
(纸本)0819452114
The lack of eye contact in video conference degrades the user experience. This problem has been known and studied for many years. There are hardware-based solutions to the eye gaze problem;however, these specialized systems are not generally accessible. This paper assumed the depth of the scene taken from the monocular camera is normal distributed on the XY plane on the real world, the plane parallel to the view plane. The three dimensional normal curve is initially estimated from the face model of the user. By performing rotation operation on the normal curve, the orientation of the face is rectified. The approach suggested in this paper is fast and effective. It has the advantages of 3D modelling, but could save the steps on complex registration, texture mapping and rendering.
VCIP 2022 "Tire pattern image classification based on lightweight network challenge" aims to design lightweight networks that correctly classify tire surface tread patterns and indentation images using less ...
详细信息
ISBN:
(纸本)9781665475921
VCIP 2022 "Tire pattern image classification based on lightweight network challenge" aims to design lightweight networks that correctly classify tire surface tread patterns and indentation images using less overhead. To this end, we present a novel lightweight tire tread classification network. Concretely, we adopt the ShuffleNet-V2-x0.5 network as our backbone. To reduce the computation complexity, we introduce the Space-To-Depth and Anti-Alias Downsampling modules to pre-process the input image. Moreover, to enhance the classification ability of our model, we adopt the knowledge distillation strategy by considering Vision Transformer as the teacher network. To ensure the robustness of our model, we pre-train it on imageNet and fine-tune the training set of the challenge. Experiments on the challenge dataset demonstrate that our model achieves superior performance, with 99.00% classification accuracy, 25.51M FLOPs, and 0.20M parameters.
Recently, deep learning-based video compression algorithms have achieved competitive performance in Bjontegaard delta (BD) rate, especially those adopting super-resolution networks as post-processing modules in downsa...
详细信息
ISBN:
(纸本)9781665475921
Recently, deep learning-based video compression algorithms have achieved competitive performance in Bjontegaard delta (BD) rate, especially those adopting super-resolution networks as post-processing modules in downsampling-based video compression (DBC) frameworks. However, limited by the non-differentiable characteristics of traditional codecs, DBC frameworks mainly focus on improving the performance of super-resolution modules while ignoring optimizing downscaling modules. It is crucial to improve video compression performance without introducing additional modifications to the decoder client in practical application scenarios. We propose a context-aware processing network (CPN) compatible with standard codecs with no computational burden introduced to the client, which preserves the critical information and essential structures during downscaling. The proposed CPN works as a precoder cascaded by standard codecs to improve the compression performance on the server before encoding and transmission. Besides, a surrogate codec is employed to simulate the degradation process of the standard codecs and backpropagate the gradient to optimize the CPN. Experimental results show that the proposed method outperforms latest pre-processing networks and achieves considerable performance compared with the latest DBC frameworks.
暂无评论