检索结果-内蒙古大学图书馆

Smallbignet: Integrating core and contextual views for video classification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Xianhang Wang, Yali Zhou, Zhipeng Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Temporal convolution has been widely used for video classification. However, it is performed on spatio-temporal contexts in a limited view, which often weakens its capacity of learning video representation. To alleviate this problem, we propose a concise and novel SmallBig network, with the cooperation of small and big views. For the current time step, the small view branch is used to learn the core semantics, while the big view branch is used to capture the contextual semantics. Unlike traditional temporal convolution, the big view branch can provide the small view branch with the most activated video features from a broader 3D receptive field. Via aggregating such big-view contexts, the small view branch can learn more robust and discriminative spatio-temporal representations for video classification. Furthermore, we propose to share convolution in the small and big view branch, which improves model compactness as well as alleviates overfitting. As a result, our SmallBigNet achieves a comparable model size like 2D CNNs, while boosting accuracy like 3D CNNs. We conduct extensive experiments on the large-scale video benchmarks, e.g., Kinetics400, Something-Something V1 and V2. Our SmallBig network outperforms a number of recent state-of-the-art approaches, in terms of accuracy and/or efficiency. The codes and models will be available on https://***/xhl-video/SmallBigNet. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolution

DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xie, Liangbin Wang, Xintao Chen, Xiangyu Li, Gen Shan, Ying Zhou, Jiantao Dong, Chao State Key Laboratory of Internet of Things for Smart City University of Macau China Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China ARC Lab Tencent PCG China Shanghai Artificial Intelligence Laboratory China Platform Technologies China

Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details. However, it is notorious that GAN-based SR models will inevitably produce unpleasant and undesirable artifacts, especially in practical scenarios. Previous works typically suppress artifacts with an extra loss penalty in the training phase. They only work for in-distribution artifact types generated during training. When applied in real-world scenarios, we observe that those improved methods still generate obviously annoying artifacts during inference. In this paper, we analyze the cause and characteristics of the GAN artifacts produced in unseen test data without ground-truths. We then develop a novel method, namely, DeSRA, to Detect and then "Delete" those SR Artifacts in practice. Specifically, we propose to measure a relative local variance distance from MSE-SR results and GAN-SR results, and locate the problematic areas based on the above distance and semantic-aware thresholds. After detecting the artifact regions, we develop a finetune procedure to improve GAN-based SR models with a few samples, so that they can deal with similar types of artifacts in more unseen real data. Equipped with our DeSRA, we can successfully eliminate artifacts from inference and improve the ability of SR models to be applied in real-world scenarios. The code will be available at https://***/TencentARC/DeSRA. Copyright © 2023, The Authors. All rights reserved.

关键词： Generative adversarial networks

Artist-Aware Image Style Transfer Via Text Guided Contrastive Learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Liu, Zhi-Song Wang, Li-Wen Siu, Wan-Chi Kalogeiton, Vicky Computer Vision and Pattern Recognition Laboratory LUT University Finland Caritas Institute of Higher Education Hong Kong The Hong Kong Polytechnic University Hong Kong LIX Ecole Polytechnique IP Paris France

Image style transfer has attracted widespread attention in the past few years. Despite its remarkable results, it requires additional style images available as references, making it less flexible and inconvenient. Using text is the most natural way to describe the style. More importantly, text can describe implicit abstract styles, like styles of specific artists or art movements. In this paper, we propose a text-driven image style transfer (TxST) that leverages advanced image-text encoders to control arbitrary style transfer. We introduce a contrastive training strategy to effectively extract style descriptions from the image-text model (i.e., CLIP), which aligns stylization with the text description. To this end, we also propose a novel and efficient attention module that explores cross-attentions to fuse style and content features. Finally, we achieve an arbitrary artist-aware image style transfer that can use either image or text for stylization. Extensive experiments demonstrate that our approach outperforms the state-of-the-art methods on both image and text-guided style transfer. Moreover, it can mimic the styles of multiple artists to achieve attractive results, thus highlighting a promising direction in image style transfer. © 2023, The Authors. All rights reserved.

关键词： Abstracting

Auditory filter based broadband MUSIC algorithm for sound source localization

学校读者我要写书评

暂无评论

Chinese Journal of Acoustics 2013年第4期32卷 439-453页

作者： LIAO Fengchai LI Peng LIU Wenju Department of Math and Computer Science Sanming University Sanming Digital Content Technique and Service Research Center Institute of Automation Chinese Academy of Sciences National Key Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences

Based on the analysis of the shortcomings of broadband MUSIC algorithm with short-time Fourier transform （SF-MUSIC） for sound source localization, a broadband MUSIC algorithm with auditory filter （AF-MUSIC） was proposed. The proposed algorithm first em- ploys auditory filter bank to decompose the signals received on the microphone array, and then locates the sound source with MUSIC algorithm over every frequency channel. At last, by combining with the subinterval frequency estimation, the final localization result is gained. Evaluations on the proposed algorithm prove that comparing with the SF-MUSIC algorithm, the AF-MUSIC algorithm decreases the average error of the estimation results with 2.5479 de- gree in different source conditions. The accuracy of sound source DOA estimation is enhanced effectively.

关键词： MUSIC SF DOA

Enhanced quadratic video interpolation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Liu, Yihao Xie, Liangbin Siyao, Li Sun, Wenxiu Qiao, Yu Dong, Chao ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China SenseTime Research

With the prosperity of digital video industry, video frame interpolation has arisen continuous attention in computer vision community and become a new upsurge in industry. Many learning-based methods have been proposed and achieved progressive results. Among them, a recent algorithm named quadratic video interpolation (QVI) achieves appealing performance. It exploits higher-order motion information (e.g. acceleration) and successfully models the estimation of interpolated flow. However, its produced intermediate frames still contain some unsatisfactory ghosting, artifacts and inaccurate motion, especially when large and complex motion occurs. In this work, we further improve the performance of QVI from three facets and propose an enhanced quadratic video interpolation (EQVI) model. In particular, we adopt a rectified quadratic flow prediction (RQFP) formulation with least squares method to estimate the motion more accurately. Complementary with image pixel-level blending, we introduce a residual contextual synthesis network (RCSN) to employ contextual information in high-dimensional feature space, which could help the model handle more complicated scenes and motion patterns. Moreover, to further boost the performance, we devise a novel multi-scale fusion network (MS-Fusion) which can be regarded as a learnable augmentation process. The proposed EQVI model won the first place in the AIM2020 Video Temporal Super-Resolution Challenge. Codes are available at https://***/lyh-18/EQVI. Copyright © 2020, The Authors. All rights reserved.

关键词： Least squares approximations

NLFA: A Non Local Fusion Alignment Module for Multi-Scale Feature in Object Detection 3rd

学校读者我要写书评

暂无评论

NLFA: A Non Local Fusion Alignment Module for Multi-Scale Fe...

3rd International Symposium on Automation, Mechanical and Design Engineering, SAMDE 2022

作者： Xue, Honghui Ma, Jinshan Cai, Zheyi Fu, Junfang Guo, Feng Weng, Wei Dong, Yunxin Zhang, Zhenchang College of Computer and Information Sciences Fujian Agriculture and Forestry University Fuzhou China Fujian Zhongke Zhongxin Intelligent Technology Co. Ltd Fuzhou China Fujian Newland Auto-ID Tech. Co. Ltd Fuzhou China Department of Computer and Information Engineering Xiamen University of Technology Xiamen China Fujian Key Laboratory of Pattern Recognition and Image Understanding Xiamen China

ISBN: (纸本)9783031400698

Recently, in order to pursue better detection results, more convolutional layers and deeper networks are a direction pursued by everyone. However, more and more down-sampling convolution or up-sampling operations generate feature maps of different scales, which makes it difficult to avoid the loss of detailed information of the image, and the distribution of different scales features will be misaligned. In particular, the loss and dislocation of the target boundary information will affect the features learned by the model and reduce the accuracy. This paper proposes a feature alignment method based on non-local idea, and designed two modules—Non Local Align Module (NLA) and Channel Fusion Augment Module (CFA). At the same time, the neighborhood calculation algorithm is also designed for it, which strengthens the binding force on the calculation of boundary information. These two modules can be easily embedded into the current mainstream object detection network to improve the detection effect of the model. Compared to the previous model, the network using our NLA module and CFA module achieves better results than the original model. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Object detection

Visual compositional learning for Human-Object interaction detection

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Hou, Zhi Peng, Xiaojiang Qiao, Yu Tao, Dacheng UBTECH Sydney AI Centre School of Computer Science Faculty of Engineering University of Sydney DarlingtonNSW2008 Australia Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China

Human-Object interaction (HOI) detection aims to localize and infer relationships between human and objects in an image. It is challenging because an enormous number of possible combinations of objects and verbs types forms a long-tail distribution. We devise a deep Visual Compositional Learning (VCL) framework, which is a simple yet efficient framework to effectively address this problem. VCL first decomposes an HOI representation into object and verb specific features, and then composes new interaction samples in the feature space via stitching the decomposed features. The integration of decomposition and composition enables VCL to share object and verb features among different HOI samples and images, and to generate new interaction samples and new types of HOI, and thus largely alleviates the long-tail distribution problem and benefits low-shot or zero-shot HOI detection. Extensive experiments demonstrate that the proposed VCL can effectively improve the generalization of HOI detection on HICO-DET and V-COCO and outperforms the recent state-of-the-art methods on HICO-DET. Code is available at https://***/zhihou7/VCL. Copyright © 2020, The Authors. All rights reserved.

关键词： Deep learning

A New Journey from SDRTV to HDRTV

学校读者我要写书评

暂无评论

A New Journey from SDRTV to HDRTV

International Conference on computer vision (ICCV)

作者： Xiangyu Chen Zhengwen Zhang Jimmy S. Ren Lynhoo Tian Yu Qiao Chao Dong ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institute of Advanced Technology Chinese Academy of Sciences SenseTime Research Qing Yuan Research Institute Shanghai Jiao Tong University Shanghai AI Laboratory Shanghai

ISBN: (纸本)9781665428132

Nowadays modern displays are capable to render video content with high dynamic range (HDR) and wide color gamut (WCG). However, most available resources are still in standard dynamic range (SDR). Therefore, there is an urgent demand to transform existing SDR-TV contents into their HDR-TV versions. In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step solution pipeline including adaptive global color mapping, local enhancement and highlight generation. Moreover, the above analysis inspires us to present a lightweight network that utilizes global statistics as guidance to conduct image-adaptive color mapping. In addition, we construct a dataset using HDR videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results achieve state-of-the-art performance in quantitative comparisons and visual quality. The code and dataset are available at https://***/chxy95/HDRTVNet.

关键词： Visualization computer vision Analytical models Codes Image color analysis Pipelines Transforms

Natural image matting based on image inpainting

学校读者我要写书评

暂无评论

Natural image matting based on image inpainting

computer Graphics, Image and Virtualization (ICCGIV), International Conference on

作者： Yuan Zhang Mian Tan Zhulian Zhou Yuan Yang Yihui Liang Fujian Feng Guizhou Key Laboratory of Pattern Recognition and Intelligent System Guizhou Minzu University Guiyang China School of Computer Science University of Electronic Science and Technology China Zhongshan China

Deep image matting is a hot problem with applications in computer vision and image processing. It has been widely used in image composition, film production and video editing etc. The current matting method based on image inpainting uses a deep neural network to inpaint the foreground target and background region to further improve the accuracy of alpha matte. However, when the trimap contains large unknown regions, the excessive inpainting of foreground and background produce a lot of redundant information, which leads to a degradation of alpha matte quality. Therefore, to address this issue, a natural image matting method based on image inpainting is designed. This method involves the refinement process for trimap, which improves the quality of the trimap, enlarges the foreground and background regions providing additional information for image matting. Extensive experimental results on the composition-1k dataset demonstrate that the presented method provide high-quality alpha mattes not only in the case that the trimap contains small unknown regions, but also in the case that the trimap contains large unknown regions.

关键词： Degradation Deep learning computer vision Image processing Refining Neural networks Production