检索结果-内蒙古大学图书馆

Restoration of User Videos Shared on Social Media

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Luo, Hongming Zhou, Fei Lam, Kin-Man Qiu, Guoping College of Electronics and Information Engineering Shenzhen University China Department of Electronic and Information Engineering The Hong Kong Polytechnic University China Peng Cheng Laboratory Shenzhen China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen China Shenzhen Key Laboratory of Digital Creative Technology China Shenzhen Institute for Artificial Intelligence and Robotics for Society China Guangdong-Hong Kong Joint Laboratory for Big Data Imaging and Communication China

User videos shared on social media platforms usually suffer from degradations caused by unknown proprietary processing procedures, which means that their visual quality is poorer than that of the originals. This paper presents a new general video restoration framework for the restoration of user videos shared on social media platforms. In contrast to most deep learning-based video restoration methods that perform end-to-end mapping, where feature extraction is mostly treated as a black box, in the sense that what role a feature plays is often unknown, our new method, termed Video restOration through adapTive dEgradation Sensing (VOTES), introduces the concept of a degradation feature map (DFM) to explicitly guide the video restoration process. Specifically, for each video frame, we first adaptively estimate its DFM to extract features representing the difficulty of restoring its different regions. We then feed the DFM to a convolutional neural network (CNN) to compute hierarchical degradation features to modulate an end-to-end video restoration backbone network, such that more attention is paid explicitly to potentially more difficult to restore areas, which in turn leads to enhanced restoration performance. We will explain the design rationale of the VOTES framework and present extensive experimental results to show that the new VOTES method outperforms various state-of-the-art techniques both quantitatively and qualitatively. In addition, we contribute a large scale real-world database of user videos shared on different social media platforms. Codes and datasets are available at https://***/luohongming/***. Copyright © 2022, The Authors. All rights reserved.

关键词： Restoration

Aesthetically Relevant Image Captioning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhong, Zhipeng Zhou, Fei Qiu, Guoping College of Electronics and Information Engineering Shenzhen University China Peng Cheng National Laboratory Shenzhen China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen China Shenzhen Institute for Artificial Intelligence and Robotics for Society China Guangdong-Hong Kong Joint Laboratory for Big Data Imaging and Communication Shenzhen China School of Computer Science The University of Nottingham United Kingdom

Image aesthetic quality assessment (AQA) aims to assign numerical aesthetic ratings to images whilst image aesthetic captioning (IAC) aims to generate textual descriptions of the aesthetic aspects of images. In this paper, we study image AQA and IAC together and present a new IAC method termed Aesthetically Relevant Image Captioning (ARIC). Based on the observation that most textual comments of an image are about objects and their interactions rather than aspects of aesthetics, we first introduce the concept of Aesthetic Relevance Score (ARS) of a sentence and have developed a model to automatically label a sentence with its ARS. We then use the ARS to design the ARIC model which includes an ARS weighted IAC loss function and an ARS based diverse aesthetic caption selector (DACS). We present extensive experimental results to show the soundness of the ARS concept and the effectiveness of the ARIC model by demonstrating that texts with higher ARS’s can predict the aesthetic ratings more accurately and that the new ARIC model can generate more accurate, aesthetically more relevant and more diverse image captions. Furthermore, a large new research database containing 510K images with over 5 million comments and 350K aesthetic scores, and code for implementing ARIC are available at https://***/pengZai/ARIC. © 2022, CC BY-NC-ND.

关键词：

Dynamic aspiration based on win-stay-lose-learn rule in spatial prisoner's dilemma game

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Shi, Zhenyu Wei, Wei Feng, Xiangnan Li, Xing Zheng, Zhiming School of Mathematical Sciences Beihang University Beijing China Key Laboratory of Mathematics Informatics Behavioral Semantics Ministry of Education China Peng Cheng Laboratory ShenzhenGuangdong China Beijing Advanced Innovation Center for Big Data and Brain Computing Beihang China

Prisoner’s dilemma game is the most commonly used model of spatial evolutionary game which is considered as a paradigm to portray competition among selfish individuals. In recent years, Win-Stay-Lose-Learn, a strategy updating rule base on aspiration, has been proved to be an effective model to promote cooperation in spatial prisoner’s dilemma game, which leads aspiration to receive lots of attention. But in many research the assumption that individuals aspiration is fixed is inconsistent with recent results from psychology. In this paper, according to Expected Value Theory and Achievement Motivation Theory, we propose a dynamic aspiration model based on Win-Stay-Lose-Learn rule in which individual’s aspiration is inspired by its payoff. It is found that dynamic aspiration has a significant impact on the evolution process, and different initial aspirations lead to different results, which are called Stable Coexistence under Low Aspiration, Dependent Coexistence under Moderate aspiration and Defection Explosion under High Aspiration respectively. Furthermore, a deep analysis is performed on the local structures which cause cooperator’s existence or defector’s expansion, and the evolution process for different parameters including strategy and aspiration. As a result, the intrinsic structures leading to defectors’ expansion and cooperators’ survival are achieved for different evolution process, which provides a penetrating understanding of the evolution. Compared to fixed aspiration model, dynamic aspiration introduces a more satisfactory explanation on population evolution laws and can promote deeper comprehension for the principle of prisoners dilemma. Copyright © 2020, The Authors. All rights reserved.

关键词： Expansion

COLLABORATIVE AUTO-ENCODING FOR BLIND IMAGE QUALITY ASSESSMENT

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Zehong Zhou, Fei Qiu, Guoping College of Electronic and Information Engineering Shenzhen University Shenzhen China Peng Cheng Laboratory Shenzhen China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen China Shenzhen Institute for Artificial Intelligence and Robotics for Society Shenzhen China School of Computer Science University of Nottingham Nottingham United Kingdom Guangdong-Hong Kong Joint Laboratory for Big Data Imaging and Communication Shenzhen China

Blind image quality assessment (BIQA) is a challenging problem with important real-world applications. Recent efforts attempting to exploit powerful representations by deep neural networks (DNN) are hindered by the lack of subjectively annotated data. This paper presents a novel BIQA method which overcomes this fundamental obstacle. Specifically, we design a pair of collaborative autoencoders (COAE) consisting of a content autoencoder (CAE) and a distortion autoencoder (DAE) that work together to extract content and distortion representations, which are shown to be highly descriptive of image quality. While the CAE follows a standard codec procedure, we introduce the CAE-encoded feature as an extra input to the DAE’s decoder for reconstructing distorted images, thus effectively forcing DAE’s encoder to extract distortion representations. The self-supervised learning framework allows the COAE including two feature extractors to be trained by almost unlimited amount of data, thus leaving limited samples with annotations to finetune a BIQA model. We will show that the proposed BIQA method achieves state-of-the-art performance and has superior generalization capability over other learning based models. The codes are available at: https://***/Macro-Zhou/NRIQA-VISOR/. © 2023, CC BY.

关键词： Supervised learning

Collaborative Auto-encoding for Blind Image Quality Assessment

学校读者我要写书评

暂无评论

Collaborative Auto-encoding for Blind Image Quality Assessme...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Zehong Zhou Fei Zhou Guoping Qiu College of Electronic and Information Engineering Shenzhen University Shenzhen China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen China Shenzhen Institute for Artificial Intelligence and Robotics for Society Shenzhen China Peng Cheng Laboratory Shenzhen China Guangdong-Hong Kong Joint Laboratory for Big Data Imaging and Communication Shenzhen China School of Computer Science University of Nottingham Nottingham U.K

Blind image quality assessment (BIQA) is a challenging problem with important real-world applications. Recent efforts attempting to exploit powerful representations by deep neural networks (DNN) are hindered by the lack of subjectively annotated data. This paper presents a novel BIQA method which overcomes this fundamental obstacle. Specifically, we design a pair of collaborative autoencoders (COAE) consisting of a content autoencoder (CAE) and a distortion autoencoder (DAE) that work together to extract content and distortion representations, which are shown to be highly descriptive of image quality. While the CAE follows a standard codec procedure, we introduce the CAE-encoded feature as an extra input to the DAE's decoder for reconstructing distorted images, thus effectively forcing DAE's encoder to extract distortion representations. The self-supervised learning framework allows the COAE including two feature extractors to be trained by almost unlimited amount of data, thus leaving limited samples with annotations to finetune a BIQA model. We will show that the proposed BIQA method achieves state-of-the-art performance and has superior generalization capability over other learning based models. All the related codes, trained models, and the supplementary material are available at: https://***/Macro-Zhou/NRIQA-VISOR/.

关键词：

FW-GAN: Flow-Navigated Warping GAN for Video Virtual Try-On

学校读者我要写书评

暂无评论

FW-GAN: Flow-Navigated Warping GAN for Video Virtual Try-On

International Conference on Computer Vision (ICCV)

作者： Haoye Dong Xiaodan Liang Xiaohui Shen Bowen Wu Bing-cheng Chen Jian Yin School of Data and Computer Science Sun Yat-sen University Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou P.R.China School of Intelligent Systems Engineering Sun Yat-sen University ByteDance AI Lab

ISBN: (数字)9781728148038

ISBN: (纸本)9781728148045

Beyond current image-based virtual try-on systems that have attracted increasing attention, we move a step forward to developing a video virtual try-on system that precisely transfers clothes onto the person and generates visually realistic videos conditioned on arbitrary poses. Besides the challenges in image-based virtual try-on (e.g., clothes fidelity, image synthesis), video virtual try-on further requires spatiotemporal consistency. Directly adopting existing image-based approaches often fails to generate coherent video with natural and realistic textures. In this work, we propose Flow-navigated Warping Generative Adversarial Network (FW-GAN), a novel framework that learns to synthesize the video of virtual try-on based on a person image, the desired clothes image, and a series of target poses. FW-GAN aims to synthesize the coherent and natural video while manipulating the pose and clothes. It consists of: (i) a flow-guided fusion module that warps the past frames to assist synthesis, which is also adopted in the discriminator to help enhance the coherence and quality of the synthesized video; (ii) a warping net that is designed to warp clothes image for the refinement of clothes textures; (iii) a parsing constraint loss that alleviates the problem caused by the misalignment of segmentation maps from images with different poses and various clothes. Experiments on our newly collected dataset show that FW-GAN can synthesize high-quality video of virtual try-on and significantly outperforms other methods both qualitatively and quantitatively.

关键词： Integrated circuits Generators Gallium nitride Three-dimensional displays Image generation Computational modeling Visualization

Super-resolving Compressed Images via Parallel and Series Integration of Artifact Reduction and Resolution Enhancement

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Luo, Hongming Zhou, Fei Liao, Guangsen Qiu, Guoping College of Electronics and Information Engineering Shenzhen University China Peng Cheng Laboratory Shenzhen China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen China Shenzhen Key Laboratory of Digital Creative Technology China Shenzhen Institute for Artificial Intelligence and Robotics for Society Shenzhen China School of Computer Science University of Nottingham NottinghamNG8 1BB United Kingdom Guangdong-Hong Kong Joint Laboratory for Big Data Imaging and Communication Guangdong Shenzhen China

In real-world applications, such as sharing photos on social media platforms, images are always not only sub-sampled but also heavily compressed thus often containing various artefacts. Simple methods for enhancing the resolution of such images will exacerbate the artefacts, rendering them visually objectionable. In spite of its high practical values, super-resolving compressed images is not well studied in the literature. In this paper, we propose a novel compressed image super resolution (CISR) framework based on parallel and series integration of artefacts removal and resolution enhancement. Based on a mathematical inference model for estimating a clean low-resolution (LR) image and a clean high-resolution (HR) image from a down-sampled and compressed observation, we have designed a CISR architecture consisting of two deep neural network modules: the artefacts removal module (ARM) and the resolution enhancement module (REM). The ARM and the REM work in parallel with both taking the compressed LR image as their inputs, at the same time they also work in series with the REM taking the output of the ARM as one of its inputs and the ARM taking the output of the REM as its other input. A technique called unfolding is introduced to recursively suppress the compression artefacts and restore the image resolution. A unique feature of our CISR system is that it exploits the parallel and series connections between the ARM and the REM, and recursive optimization to reduce the model's dependency on specific types of degradation thus making it possible to train a single model to super-resolve images compressed by different methods to different qualities. Experiments are conducted on a mixture of JPEG and WebP compressed images without assuming apriori compression type and compression quality factor. To demonstrate our technique's real-world application value, we have also applied the trained models directly to restore social media images which have undergone scaling and compre

关键词： Deep neural networks

Global context-aware progressive aggregation network for salient object detection

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Chen, Zuyao Xu, Qianqian Cong, Runmin Huang, Qingming University of Chinese Academy of Sciences Beijing China Key Lab. of Intelligent Information Processing ICT CAS Beijing China Institute of Information Science Beijing Jiaotong University Beijing China Key Lab. of Big Data Mining and Knowledge Management CAS Beijing China Peng Cheng Laboratory Shenzhen Guangdong China

Deep convolutional neural networks have achieved competitive performance in salient object detection, in which how to learn effective and comprehensive features plays a critical role. Most of the previous works mainly adopted multiplelevel feature integration yet ignored the gap between different features. Besides, there also exists a dilution process of high-level features as they passed on the top-down pathway. To remedy these issues, we propose a novel network named GCPANet to effectively integrate low-level appearance features, high-level semantic features, and global context features through some progressive context-aware Feature Interweaved Aggregation (FIA) modules and generate the saliency map in a supervised way. Moreover, a Head Attention (HA) module is used to reduce information redundancy and enhance the top layers features by leveraging the spatial and channel-wise attention, and the Self Refinement (SR) module is utilized to further refine and heighten the input features. Furthermore, we design the Global Context Flow (GCF) module to generate the global context information at different stages, which aims to learn the relationship among different salient regions and alleviate the dilution effect of high-level features. Experimental results on six benchmark datasets demonstrate that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively. Copyright © 2020, The Authors. All rights reserved.

关键词： Deep neural networks

Optimizing video caching at the edge: A hybrid multi-point process approach

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Xianzhi Zhou, Yipeng Wu, Di Hu, Miao Xi Zheng, James Chen, Min Guo, Song Department of Computer Science Sun Yat-sen University Guangzhou510006 China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou510006 China Peng Cheng Laboratory Shenzhen518000 China Department of Computing FSE Macquarie University 2122 Australia School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China Department of Computing The Polytechnic University of Hong Kong Hong Kong

It is always a challenging problem to deliver a huge volume of videos over the Internet. To meet the high bandwidth and stringent playback demand, one feasible solution is to cache video contents on edge servers based on predicted video popularity. Traditional caching algorithms (e.g., LRU, LFU) are too simple to capture the dynamics of video popularity, especially long-tailed videos. Recent learning-driven caching algorithms (e.g., DeepCache) show promising performance, however, such black-box approaches are lack of explainability and interpretability. Moreover, the parameter tuning requires a large number of historical records, which are difficult to obtain for videos with low popularity. In this paper, we optimize video caching at the edge using a white-box approach, which is highly efficient and also completely explainable. To accurately capture the evolution of video popularity, we develop a mathematical model called HRS model, which is the combination of multiple point processes, including Hawkes’ self-exciting, reactive and self-correcting processes. The key advantage of the HRS model is its explainability, and much less number of model parameters. In addition, all its model parameters can be learned automatically through maximizing the Log-likelihood function constructed by past video request events. Next, we further design an online HRS-based video caching algorithm. To verify its effectiveness, we conduct a series of experiments using real video traces collected from Tencent Video, one of the largest online video providers in China. Experiment results demonstrate that our proposed algorithm outperforms the state-of-the-art algorithms, with 12.3% improvement on average in terms of cache hit rate under realistic settings. Copyright © 2021, The Authors. All rights reserved.

关键词： Gradient methods