检索结果-内蒙古大学图书馆

Interactive grounded language acquisition and generalization in a 2D world

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yu, Haonan Zhang, Haichao Xu, Wei Baidu Research Sunnyvale United States National Engineering Laboratory for Deep Learning Technology and Applications Beijing China

We build a virtual agent for learning language in a 2D maze-like world. The agent sees images of the surrounding environment, listens to a virtual teacher, and takes actions to receive rewards. It interactively learns the teacher's language from scratch based on two language use cases: sentence-directed navigation and question answering. It learns simultaneously the visual representations of the world, the language, and the action control. By disentangling language grounding from other computational routines and sharing a concept detection function between language grounding and prediction, the agent reliably interpolates and extrapolates to interpret sentences that contain new word combinations or new words missing from training sentences. The new words are transferred from the answers of language prediction. Such a language ability is trained and evaluated on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words. The proposed model significantly outperforms five comparison methods for interpreting zero-shot sentences. In addition, we demonstrate human-interpretable intermediate outputs of the model in the appendix. Copyright © 2018, The Authors. All rights reserved.

关键词： Visual languages

Gradient descent meets shift-and-invert preconditioning for eigenvector computation 18

学校读者我要写书评

暂无评论

Gradient descent meets shift-and-invert preconditioning for ...

Proceedings of the 32nd International Conference on Neural Information Processing Systems

作者： Zhiqiang Xu Big Data Lab (BDL-US) Baidu Research National Engineering Laboratory for Deep Learning Technology and Applications

Shift-and-invert preconditioning, as a classic acceleration technique for the leading eigenvector computation, has received much attention again recently, owing to fast least-squares solvers for efficiently approximating matrix inversions in power iterations. In this work, we adopt an inexact Riemannian gradient descent perspective to investigate this technique on the effect of the step-size scheme. The shift-and-inverted power method is included as a special case with adaptive step-sizes. Particularly, two other step-size settings, i.e., constant step-sizes and Barzilai-Borwein (BB) step-sizes, are examined theoretically and/or empirically. We present a novel convergence analysis for the constant step-size setting that achieves a rate at Õ(√λ1/λ1 – λp+1), where λi represents the i-th largest eigenvalue of the given real symmetric matrix and p is the multiplicity of λ1. Our experimental studies show that the proposed algorithm can be significantly faster than the shift-and-inverted power method in practice.

关键词：

IAFA: Instance-Aware Feature Aggregation for 3D Object Detection from a Single Image 15th

学校读者我要写书评

暂无评论

IAFA: Instance-Aware Feature Aggregation for 3D Object Detec...

15th Asian Conference on Computer Vision, ACCV 2020

作者： Zhou, Dingfu Song, Xibin Dai, Yuchao Yin, Junbo Lu, Feixiang Liao, Miao Fang, Jin Zhang, Liangjun Baidu Research Beijing China National Engineering Laboratory of Deep Learning Technology and Application Beijing China Northwestern Polytechnical University Xi’an China Beijing Institute of Technology Beijing China

ISBN: (纸本)9783030695248

3D object detection from a single image is an important task in Autonomous Driving (AD), where various approaches have been proposed. However, the task is intrinsically ambiguous and challenging as single image depth estimation is already an ill-posed problem. In this paper, we propose an instance-aware approach to aggregate useful information for improving the accuracy of 3D object detection with the following contributions. First, an instance-aware feature aggregation (IAFA) module is proposed to collect local and global features for 3D bounding boxes regression. Second, we empirically find that the spatial attention module can be well learned by taking coarse-level instance annotations as a supervision signal. The proposed module has significantly boosted the performance of the baseline method on both 3D detection and 2D bird-eye’s view of vehicle detection among all three categories. Third, our proposed method outperforms all single image-based approaches (even these methods trained with depth as auxiliary inputs) and achieves state-of-the-art 3D detection performance on the KITTI benchmark. © 2021, Springer Nature Switzerland AG.

关键词： Object detection

CASIA-SURF: A Large-Scale Multi-Modal Benchmark for Face Anti-Spoofing

学校读者我要写书评

暂无评论

IEEE Transactions on Biometrics, Behavior, and Identity Scie...

IEEE Transactions on Biometrics, Behavior, and Identity Science 2020年第2期2卷 182-193页

作者： Zhang, Shifeng Liu, Ajian Wan, Jun Liang, Yanyan Guo, Guodong Escalera, Sergio Escalante, Hugo Jair Li, Stan Z. National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing100190 China Faculty of Information Technology Macau University of Science and Technology 999078 China Baidu Research and National Engineering Laboratory for Deep Learning Technology and Application Institute of Deep Learning Beijing100085 China Óptica y Electrónica Instituto Nacional de Astrofísica Puebla08007 Mexico Computer Science Department CINVESTAV-Zacatenco Mexico City07360 Mexico University of Chinese Academy of Sciences Beijing100049 China Westlake University Hangzhou310024 China

Face anti-spoofing is essential to prevent face recognition systems from a security breach. Much of the progresses have been made by the availability of face anti-spoofing benchmark datasets in recent years. However, existing face anti-spoofing benchmarks have limited number of subjects (≤170) and modalities (≤2), which hinder the further development of the academic community. To facilitate face anti-spoofing research, we introduce a large-scale multi-modal dataset, namely CASIA-SURF, which is the largest publicly available dataset for face anti-spoofing in terms of both subjects and modalities. Specifically, it consists of 1,000 subjects with 21,000 videos and each sample has 3 modalities (i.e., RGB, Depth and IR). We also provide comprehensive evaluation metrics, diverse evaluation protocols, training/validation/testing subsets and a measurement tool, developing a new benchmark for face anti-spoofing. Moreover, we present a novel multi-modal multi-scale fusion method as a strong baseline, which performs feature re-weighting to select the more informative channel features while suppressing the less useful ones for each modality across different scales. Extensive experiments have been conducted on the proposed dataset to verify its significance and generalization capability. The dataset is available at https://***/***/face-anti-spoofing/welcome/challengecvpr2019?authuser=0. © 2019 IEEE.

关键词： Face recognition

Investigate the relationship between learners' social characteristics and academic achievements 5

学校读者我要写书评

暂无评论

Investigate the relationship between learners' social charac...

5th International Workshop on Statistical Physics and Mathematics for Complex Systems, SPMCS 2017

作者： Liu, Zhi Kang, Lingyun Su, Zhu Liu, Sannyuya Sun, Jianwen National Engineering Research Center for E-Learning Central China Normal University Wuhan China National Engineering Laboratory for Technology of Big Data Applications in Education Central China Normal University Wuhan China

The social interaction behaviors of group learning have obtained a lot of attention from researches, and social network analysis (SNA) plays a critical role in exploring collective interactive patterns. In this paper, we investigated the experimental data from the discussion forum of "Discourse Analysis" course in a university learning platform. By calculating learners social characteristics, we explored the relationship between social characteristics and academic achievements, as well as the differences between various achieving learners. The results show that there exist strong positive correlations between most social characteristics and academic achievements, and the majority of high-achieving learners are located at the core position of the network. In addition, the high-achieving learners exhibited more active participations in discussions and received more posts compared with middle- and low-achieving learners. © Published under licence by IOP Publishing Ltd.

关键词：

Sparse to dense motion transfer for face image animation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhao, Ruiqi Wu, Tianyi Guo, Guodong Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China

Face image animation from a single image has achieved remarkable progress. However, it remains challenging when only sparse landmarks are available as the driving signal. Given a source face image and a sequence of sparse face landmarks, our goal is to generate a video of the face imitating the motion of landmarks. We develop an efficient and effective method for motion transfer from sparse landmarks to the face image. We then combine global and local motion estimation in a unified model to faithfully transfer the motion. The model can learn to segment the moving foreground from the background and generate not only global motion, such as rotation and translation of the face, but also subtle local motion such as the gaze change. We further improve face landmark detection on videos. With temporally better aligned landmark sequences for training, our method can generate temporally coherent videos with higher visual quality. Experiments suggest we achieve results comparable to the state-of-the-art image driven method on the same identity testing and better results on cross identity testing. © 2021, CC BY-NC-ND.

关键词： Animation

Feature Selective Transformer for Semantic Image Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Lin, Fangjian Wu, Tianyi Wu, Sitong Tian, Shengwei Guo, Guodong Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China

Recently, it has attracted more and more attentions to fuse multi-scale features for semantic image segmentation. Various works were proposed to employ progressive local or global fusion, but the feature fusions are not rich enough for modeling multi-scale context features. In this work, we focus on fusing multi-scale features from Transformer-based backbones for semantic segmentation, and propose a Feature Selective Transformer (FeSeFormer), which aggregates features from all scales (or levels) for each query feature. Specifically, we first propose a Scale-level Feature Selection (SFS) module, which can choose an informative subset from the whole multi-scale feature set for each scale, where those features that are important for the current scale (or level) are selected and the redundant are discarded. Furthermore, we propose a Full-scale Feature Fusion (FFF) module, which can adaptively fuse features of all scales for queries. Based on the proposed SFS and FFF modules, we develop a Feature Selective Transformer (FeSeFormer), and evaluate our FeSeFormer on four challenging semantic segmentation benchmarks, including PASCAL Context, ADE20K, COCO-Stuff 10K, and Cityscapes, outperforming the state-of-the-art. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantic Segmentation

FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction

学校读者我要写书评

暂无评论

FaceScape: A Large-Scale High Quality 3D Face Dataset and De...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Haotian Yang Hao Zhu Yanru Wang Mingkai Huang Qiu Shen Ruigang Yang Xun Cao Nanjing University Baidu Research National Engineering Laboratory for Deep Learning Technology and Applications China University of Kentucky Inceptio Inc.

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

In this paper, we present a large-scale detailed 3D face dataset, FaceScape, and propose a novel algorithm that is able to predict elaborate riggable 3D face models from a single image input. FaceScape dataset provides 18,760 textured 3D faces, captured from 938 subjects and each with 20 specific expressions. The 3D models contain the pore-level facial geometry that is also processed to be topologically uniformed. These fine 3D facial models can be represented as a 3D morphable model for rough shapes and displacement maps for detailed geometry. Taking advantage of the large-scale and high-accuracy dataset, a novel algorithm is further proposed to learn the expression-specific dynamic details using a deep neural network. The learned relationship serves as the foundation of our 3D face prediction system from a single image input. Different than the previous methods, our predicted 3D models are riggable with highly detailed geometry under different expressions. The unprecedented dataset and code will be released to public for research purpose.

关键词： Face Three-dimensional displays Solid modeling Shape Geometry Predictive models Cameras

POEM: 1-bit point-wise operations based on expectation-maximization for efficient point cloud processing

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Xu, Sheng Li, Yanjing Zhao, Junhe Zhang, Baochang Guo, Guodong Beihang University Beijing China National Engineering Laboratory for Deep Learning Technology and Application Institute of Deep Learning Baidu Research Beijing China

Real-time point cloud processing is fundamental for lots of computer vision tasks, while still challenged by the computational problem on resource-limited edge devices. To address this issue, we implement XNOR-Net-based binary neural networks (BNNs) for an efficient point cloud processing, but its performance is severely suffered due to two main drawbacks, Gaussian-distributed weights and non-learnable scale factor. In this paper, we introduce point-wise operations based on Expectation-Maximization (POEM) into BNNs for efficient point cloud processing. The EM algorithm can efficiently constrain weights for a robust bi-modal distribution. We lead a well-designed reconstruction loss to calculate learnable scale factors to enhance the representation capacity of 1-bit fully-connected (Bi-FC) layers. Extensive experiments demonstrate that our POEM surpasses existing the state-of-the-art binary point cloud networks by a significant margin, up to 6.7%. © 2021, CC BY.

关键词： Maximum principle