检索结果-内蒙古大学图书馆

Smallbignet: Integrating core and contextual views for video classification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Xianhang Wang, Yali Zhou, Zhipeng Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Temporal convolution has been widely used for video classification. However, it is performed on spatio-temporal contexts in a limited view, which often weakens its capacity of learning video representation. To alleviate this problem, we propose a concise and novel SmallBig network, with the cooperation of small and big views. For the current time step, the small view branch is used to learn the core semantics, while the big view branch is used to capture the contextual semantics. Unlike traditional temporal convolution, the big view branch can provide the small view branch with the most activated video features from a broader 3D receptive field. Via aggregating such big-view contexts, the small view branch can learn more robust and discriminative spatio-temporal representations for video classification. Furthermore, we propose to share convolution in the small and big view branch, which improves model compactness as well as alleviates overfitting. As a result, our SmallBigNet achieves a comparable model size like 2D CNNs, while boosting accuracy like 3D CNNs. We conduct extensive experiments on the large-scale video benchmarks, e.g., Kinetics400, Something-Something V1 and V2. Our SmallBig network outperforms a number of recent state-of-the-art approaches, in terms of accuracy and/or efficiency. The codes and models will be available on https://***/xhl-video/SmallBigNet. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolution

Learning to predict context-adaptive convolution for semantic segmentation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Liu, Jianbo He, Junjun Ren, Jimmy S. Qiao, Yu Li, Hongsheng CUHK-SenseTime Joint Laboratory Chinese University of Hong Kong Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SenseTime Research

Long-range contextual information is essential for achieving high-performance semantic segmentation. Previous feature re-weighting methods [34] demonstrate that using global context for re-weighting feature channels can effectively improve the accuracy of semantic segmentation. However, the globally-sharing feature re-weighting vector might not be optimal for regions of different classes in the input image. In this paper, we propose a Context-adaptive Convolution Network (CaC-Net) to predict a spatially-varying feature weighting vector for each spatial location of the semantic feature maps. In CaC-Net, a set of context-adaptive convolution kernels are predicted from the global contextual information in a parameter-efficient manner. When used for convolution with the semantic feature maps, the predicted convolutional kernels can generate the spatially-varying feature weighting factors capturing both global and local contextual information. Comprehensive experimental results show that our CaC-Net achieves superior segmentation performance on three public datasets, PASCAL Context, PASCAL VOC 2012 and ADE20K. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolution

Learning Attentive Pairwise Interaction for Fine-Grained Classification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhuang, Peiqin Wang, Yali Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Fine-grained classification is a challenging problem, due to subtle differences among highly-confused categories. Most approaches address this difficulty by learning discriminative representation of individual input image. On the other hand, humans can effectively identify contrastive clues by comparing image pairs. Inspired by this fact, this paper proposes a simple but effective Attentive Pairwise Interaction Network (API-Net), which can progressively recognize a pair of fine-grained images by interaction. Specifically, API-Net first learns a mutual feature vector to capture semantic differences in the input pair. It then compares this mutual vector with individual vectors to generate gates for each input image. These distinct gate vectors inherit mutual context on semantic differences, which allow API-Net to attentively capture contrastive clues by pairwise interaction between two images. Additionally, we train API-Net in an end-to-end manner with a score ranking regularization, which can further generalize API-Net by taking feature priorities into account. We conduct extensive experiments on five popular benchmarks in fine-grained classification. API-Net outperforms the recent SOTA methods, i.e., CUB-200-2011 (90.0%), Aircraft (93.9%), Stanford Cars (95.3%), Stanford Dogs (90.3%), and NABirds (88.1%). Copyright © 2020, The Authors. All rights reserved.

关键词： Vectors

Learning dynamical human-joint affinity for 3D pose estimation in videos

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Junhao Wang, Yali Zhou, Zhipeng Luan, Tianyu Wang, Zhe Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences University of California Irvine United States Shanghai AI Laboratory Shanghai China

Graph Convolution Network (GCN) has been successfully used for 3D human pose estimation in videos. However, it is often built on the fixed human-joint affinity, according to human skeleton. This may reduce adaptation capacity of GCN to tackle complex spatio-temporal pose variations in videos. To alleviate this problem, we propose a novel Dynamical Graph Network (DG-Net), which can dynamically identify human-joint affinity, and estimate 3D pose by adaptively learning spatial/temporal joint relations from videos. Different from traditional graph convolution, we introduce Dynamical Spatial/Temporal Graph convolution (DSG/DTG) to discover spatial/temporal human-joint affinity for each video exemplar, depending on spatial distance/temporal movement similarity between human joints in this video. Hence, they can effectively understand which joints are spatially closer and/or have consistent motion, for reducing depth ambiguity and/or motion uncertainty when lifting 2D pose to 3D pose. We conduct extensive experiments on three popular benchmarks, e.g., Human3.6M, HumanEva-I, and MPI-INF-3DHP, where DG-Net outperforms a number of recent SOTA approaches with fewer input frames and model size. Copyright © 2021, The Authors. All rights reserved.

关键词： Convolution

DegAE: A New Pretraining Paradigm for Low-Level vision

学校读者我要写书评

暂无评论

DegAE: A New Pretraining Paradigm for Low-Level Vision

Conference on computer vision and pattern recognition (CVPR)

作者： Yihao Liu Jingwen He Jinjin Gu Xiangtao Kong Yu Qiao Chao Dong Shanghai Artificial Intelligence Laboratory ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences University of Chinese Academy of Sciences The University of Sydney

Self-supervised pretraining has achieved remarkable success in high-level vision, but its application in low-level vision remains ambiguous and not well-established. What is the primitive intention of pretraining? What is the core problem of pretraining in low-level vision? In this paper, we aim to answer these essential questions and establish a new pretraining scheme for low-level vision. Specifically, we examine previous pretraining methods in both high-level and low-level vision, and categorize current low-level vision tasks into two groups based on the difficulty of data acqui-sition: low-cost and high-cost tasks. Existing literature has mainly focused on pretraining for low-cost tasks, where the observed performance improvement is often limited. However, we argue that pretraining is more significant for high-cost tasks, where data acquisition is more challenging. To learn a general low-level vision representation that can improve the performance of various tasks, we propose a new pretraining paradigm called degradation autoencoder (De-gAE). DegAE follows the philosophy of designing pretext task for self-supervised pretraining and is elaborately tai-lored to low-level vision. With DegAE pretraining, SwinIR achieves a 6.88dB performance gain on image dehaze task, while Uformer obtains 3.22dB and 0.54dB improvement on dehaze and derain tasks, respectively.

关键词：

Enhanced quadratic video interpolation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Liu, Yihao Xie, Liangbin Siyao, Li Sun, Wenxiu Qiao, Yu Dong, Chao ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China SenseTime Research

With the prosperity of digital video industry, video frame interpolation has arisen continuous attention in computer vision community and become a new upsurge in industry. Many learning-based methods have been proposed and achieved progressive results. Among them, a recent algorithm named quadratic video interpolation (QVI) achieves appealing performance. It exploits higher-order motion information (e.g. acceleration) and successfully models the estimation of interpolated flow. However, its produced intermediate frames still contain some unsatisfactory ghosting, artifacts and inaccurate motion, especially when large and complex motion occurs. In this work, we further improve the performance of QVI from three facets and propose an enhanced quadratic video interpolation (EQVI) model. In particular, we adopt a rectified quadratic flow prediction (RQFP) formulation with least squares method to estimate the motion more accurately. Complementary with image pixel-level blending, we introduce a residual contextual synthesis network (RCSN) to employ contextual information in high-dimensional feature space, which could help the model handle more complicated scenes and motion patterns. Moreover, to further boost the performance, we devise a novel multi-scale fusion network (MS-Fusion) which can be regarded as a learnable augmentation process. The proposed EQVI model won the first place in the AIM2020 Video Temporal Super-Resolution Challenge. Codes are available at https://***/lyh-18/EQVI. Copyright © 2020, The Authors. All rights reserved.

关键词： Least squares approximations

Compressed sensing ensemble classifier for human detection

学校读者我要写书评

暂无评论

Compressed sensing ensemble classifier for human detection

4th International Conference on Intelligence Science and Big Data Engineering, IScIDE 2013

作者： Zhang, Baochang Liu, Juan Gao, Yongsheng Liu, Jianzhuang Science and Technology on Aircraft Control Laboratory School of Automation Science and Electrical Engineering BeiHang University Beijing 100191 China School of Engineering Griffith University Australia Shenzhen Key Lab for Computer Vision and Pattern Recognition Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China Department of Information Engineering Chinese University of Hong Kong Hong Kong Hong Kong

ISBN: (纸本)9783642420566

This paper proposes a novel Compressed Sensing Ensemble Classifier (CSEC) for human detection. The proposed CSEC employs the compressed sensing technique to get a more sparse model with a more reasonable selection of base classifiers. The major contributions of this paper are: 1) a novel principled framework for ensemble classifier design based on compressed sensing;2) a new concept of considering both the simplicity of ensemble classifier and irrelevance of base classifiers towards optimal classifier design;and 3) a quadratic function for CSEC optimization which includes a new optimizable positive semi-definite relevance matrix to simultaneously select appropriate base classifiers with minimized relevance. Experimental results on INRIA and SDL databases show that the performance of CSEC is better than two most popular classifiers SVM and AdaBoost, as well as a most recent method CLML. © 2013 Springer-Verlag Berlin Heidelberg.

关键词： Compressed sensing

Digging into Uncertainty in Self-supervised Multi-view Stereo

学校读者我要写书评

暂无评论

Digging into Uncertainty in Self-supervised Multi-view Stere...

International Conference on computer vision (ICCV)

作者： Hongbin Xu Zhipeng Zhou Yali Wang Wenxiong Kang Baigui Sun Hao Li Yu Qiao ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences South China University of Technology Alibaba Group Pazhou Laboratory Shanghai AI Laboratory

ISBN: (纸本)9781665428132

Self-supervised Multi-view stereo (MVS) with a pretext task of image reconstruction has achieved significant progress recently. However, previous methods are built upon intuitions, lacking comprehensive explanations about the effectiveness of the pretext task in self-supervised MVS. To this end, we propose to estimate epistemic uncertainty in self-supervised MVS, accounting for what the model ignores. Specially, the limitations can be categorized into two types: ambiguious supervision in foreground and invalid supervision in background. To address these issues, we propose a novel Uncertainty reduction Multi-view Stereo (U-MVS) framework for self-supervised learning. To alleviate ambiguous supervision in foreground, we involve extra correspondence prior with a flow-depth consistency loss. The dense 2D correspondence of optical flows is used to regularize the 3D stereo correspondence in MVS. To handle the invalid supervision in background, we use Monte-Carlo Dropout to acquire the uncertainty map and further filter the unreliable supervision signals on invalid regions. Extensive experiments on DTU and Tank&Temples benchmark show that our U-MVS framework 1 achieves the best performance among unsupervised MVS methods, with competitive performance with its supervised opponents.

关键词： Optical losses Optical filters computer vision Uncertainty Three-dimensional displays Monte Carlo methods Benchmark testing

Multiple Transfer Learning and Multi-label Balanced Training Strategies for Facial AU Detection In the Wild

学校读者我要写书评

暂无评论

Multiple Transfer Learning and Multi-label Balanced Training...

IEEE computer Society Conference on computer vision and pattern recognition Workshops (CVPRW)

作者： Sijie Ji Kai Wang Xiaojiang Peng Jianfei Yang Zhaoyang Zeng Yu Qiao Nanyang Technological University Singapore ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Science Sun Yat-Sen University China

ISBN: (数字)9781728193601

ISBN: (纸本)9781728193618

This paper 1 presents SIAT-NTU solution and results of facial action unit (AU) detection in the EmotiNet Challenge 2020. The task aims to detect 23 AUs from facial images in the wild, and its main difficulties lie in the imbalanced AU distribution and discriminative feature learning. We tackle these difficulties from the following aspects. First, to address the unconstrained heterogeneity of in-the-wild images, we detect and align faces with multi-task convolutional neural networks (MTCNN). Second, by using multiple transfer strategies, we pre-train large CNNs on multiple related datasets, e.g. face recognition datasets and facial expression datasets, and fine-tune them on the EmotiNetdataset. Third, we employ a multi-label balanced sampling strategy and a weighted loss to mitigate the imbalance problem. Last but not the least, to further improve performance, we ensemble multiple models and optimize the thresholds for each AU. Our proposed solution achieves an accuracy of 90.13% and F1 of 44.10% in the final test phase. Our Code is available at: https://***/kaiwang960112/enc2020_au_detection.

关键词： Gold Task analysis Feature extraction Face recognition Training Mathematical model Muscles