检索结果-内蒙古大学图书馆

Few-Shot learning with Complex-valued Neural Networks 31

学校读者我要写书评

暂无评论

Few-Shot Learning with Complex-valued Neural Networks

31st British Machine Vision Conference, BMVC 2020

作者： Liu, Zhen Zhang, Baochang Guo, Guodong Beihang University Beijing China Institute of Deep Learning Baidu Research National Engineering Laboratory for Deep Learning Technology and Application China

Feature representation is fundamental and attracts much attention in few-shot learning. Convolutional neural networks (CNNs) are among the best feature extractors so far in this field, which are successfully combined with metric learning, leading to the state-of-the-art performance. However, the subtle difference among inter-class samples challenges existing CNN based methods, which only use real-valued CNNs that fail to extract more detailed information. In this paper, we introduce complex metric module (CMM) into metric learning, aiming to better measure the inter- and intra-class relations based on both amplitude and phase information. Specifically, building upon the recent episodic training mechanism, our CMM can enhance the representation capacity by extracting robust complex-valued features to facilitate modeling subtle relationships among samples, which can enhance the performance of the few-shot classification task when only few samples are available. Moreover, we introduce a new transductive method into CMM, by considering not only query and support but also query and query relationships to predict classes of unlabeled samples. Experiments on two benchmark datasets show that the proposed CMM significantly improves the performance over other approaches and achieves the state-of-the-art results. © 2020. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.

关键词： Benchmarking

POEM: 1-bit Point-wise Operations based on Expectation-Maximization for Efficient Point Cloud Processing 32

学校读者我要写书评

暂无评论

POEM: 1-bit Point-wise Operations based on Expectation-Maxim...

32nd British Machine Vision Conference, BMVC 2021

作者： Xu, Sheng Li, Yanjing Zhao, Junhe Zhang, Baochang Guo, Guodong Beihang University Beijing China National Engineering Laboratory for Deep Learning Technology and Application Institute of Deep Learning Baidu Research Beijing China

Real-time point cloud processing is fundamental for lots of computer vision tasks, while still challenged by the computational problem on resource-limited edge devices. To address this issue, we implement XNOR-Net-based binary neural networks (BNNs) for an efficient point cloud processing, but its performance is severely suffered due to two main drawbacks, Gaussian-distributed weights and non-learnable scale factor. In this paper, we introduce point-wise operations based on Expectation-Maximization (POEM) into BNNs for efficient point cloud processing. The EM algorithm can efficiently constrain weights for a robust bi-modal distribution. We lead a well-designed reconstruction loss to calculate learnable scale factors to enhance the representation capacity of 1-bit fully-connected (Bi-FC) layers. Extensive experiments demonstrate that our POEM surpasses existing the state-of-the-art binary point cloud networks by a significant margin, up to 6.7%. © 2021. The copyright of this document resides with its authors.

关键词： Computer vision

A new method of region embedding for text classification 6

学校读者我要写书评

暂无评论

A new method of region embedding for text classification

6th International Conference on learning Representations, ICLR 2018

作者： Qiao, Chao Huang, Bo Niu, Guocheng Li, Daren Dong, Daxiang He, Wei Yu, Dianhai Wu, Hua Baidu Inc. Beijing China National Engineering Laboratory of Deep Learning Technology and Application China

To represent a text as a bag of properly identified "phrases" and use the representation for processing the text is proved to be useful. The key question here is how to identify the phrases and represent them. The traditional method of utilizing n-grams can be regarded as an approximation of the approach. Such a method can suffer from data sparsity, however, particularly when the length of n-gram is large. In this paper, we propose a new method of learning and utilizing task-specific distributed representations of n-grams, referred to as "region embeddings". Without loss of generality we address text classification. We specifically propose two models for region embeddings. In our models, the representation of a word has two parts, the embedding of the word itself, and a weighting matrix to interact with the local context, referred to as local context unit. The region embeddings are learned and used in the classification task, as parameters of the neural network classifier. Experimental results show that our proposed method outperforms existing methods in text classification on several benchmark datasets. The results also indicate that our method can indeed capture the salient phrasal expressions in the texts. © learning Representations, ICLR 2018 - Conference Track *** right reserved.

关键词： Embeddings

GINet: Graph Interaction Network for Scene Parsing 1

学校读者我要写书评

暂无评论

16th European Conference on Computer Vision, ECCV 2020

作者： Wu, Tianyi Lu, Yu Zhu, Yu Zhang, Chuang Wu, Ming Ma, Zhanyu Guo, Guodong Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China Beijing University of Posts and Telecommunications Beijing China

ISBN: (数字)9783030585204

ISBN: (纸本)9783030585198

Recently, context reasoning using image regions beyond local convolution has shown great potential for scene parsing. In this work, we explore how to incorperate the linguistic knowledge to promote context reasoning over image regions by proposing a Graph Interaction unit (GI unit) and a Semantic Context Loss (SC-loss). The GI unit is capable of enhancing feature representations of convolution networks over high-level semantics and learning the semantic coherency adaptively to each sample. Specifically, the dataset-based linguistic knowledge is first incorporated in the GI unit to promote context reasoning over the visual graph, then the evolved representations of the visual graph are mapped to each local representation to enhance the discriminated capability for scene parsing. GI unit is further improved by the SC-loss to enhance the semantic representations over the exemplar-based semantic graph. We perform full ablation studies to demonstrate the effectiveness of each component in our approach. Particularly, the proposed GINet outperforms the state-of-the-art approaches on the popular benchmarks, including Pascal-Context and COCO Stuff. © 2020, Springer Nature Switzerland AG.

关键词： Convolution

DeLS-3D: deep localization and segmentation with a 3D semantic map

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Wang, Peng Yang, Ruigang Cao, Binbin Xu, Wei Lin, Yuanqing Baidu Research National Engineering Laboratory for Deep Learning Technology and Applications

For applications such as augmented reality, autonomous driving, self-localization/camera pose estimation and scene parsing are crucial technologies. In this paper, we propose a unified framework to tackle these two problems simultaneously. The uniqueness of our design is a sensor fusion scheme which integrates camera videos, motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robustness and efficiency of the system. Specifically, we first have an initial coarse camera pose obtained from consumer-grade GPS/IMU, based on which a label map can be rendered from the 3D semantic map. Then, the rendered label map and the RGB image are jointly fed into a pose CNN, yielding a corrected camera pose. In addition, to incorporate temporal information, a multi-layer recurrent neural network (RNN) is further deployed improve the pose accuracy. Finally, based on the pose from RNN, we render a new label map, which is fed together with the RGB image into a segment CNN which produces per-pixel semantic label. In order to validate our approach, we build a dataset with registered 3D point clouds and video camera images. Both the point clouds and the images are semantically-labeled. Each video frame has ground truth pose from highly accurate motion sensors. We show that practically, pose estimation solely relying on images like PoseNet [25] may fail due to street view confusion, and it is important to fuse multiple sensors. Finally, various ablation studies are performed, which demonstrate the effectiveness of the proposed system. In particular, we show that scene parsing and pose estimation are mutually beneficial to achieve a more robust and accurate system. Copyright © 2018, The Authors. All rights reserved.

关键词： Rendering (computer graphics)

Adaptive cross-fusion learning for multi-modal gesture recognition

学校读者我要写书评

暂无评论

Virtual Reality & Intelligent Hardware 2021年第3期3卷 235-247页

作者： Benjia ZHOU Jun WAN Yanyan LIANG Guodong GUO Macao University of Science and Technology Macao 999078China National Laboratory of Pattern Recognition Institute of AutomationChinese Academy of SciencesBeijing 100190China Baidu Research Beijing 100193Chinaand National Engineering Laboratory for Deep Learning Technology and ApplicationBeijing 100193China

Background Gesture recognition has attracted significant attention because of its wide range of potential *** multi-modal gesture recognition has made significant progress in recent years,a popular method still is simply fusing prediction scores at the end of each branch,which often ignores complementary features among different modalities in the early stage and does not fuse the complementary features into a more discriminative *** This paper proposes an Adaptive Cross-modal Weighting(ACmW)scheme to exploit complementarity features from RGB-D data in this *** scheme learns relations among different modalities by combining the features of different data *** proposed ACmW module contains two key functions:(1)fusing complementary features from multiple streams through an adaptive one-dimensional convolution;and(2)modeling the correlation of multi-stream complementary features in the time *** the effective combination of these two functional modules,the proposed ACmW can automatically analyze the relationship between the complementary features from different streams,and can fuse them in the spatial and temporal *** Extensive experiments validate the effectiveness of the proposed method,and show that our method outperforms state-of-the-art methods on IsoGD and NVGesture.

关键词： Gesture recognition Multi-modal fusion RGB-D

Interactive language acquisition with one-shot visual concept learning through a conversational game

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Zhang, Haichao Yu, Haonan Xu, Wei Baidu Research - Institue of Deep Learning Sunnyvale United States National Engineering Laboratory for Deep Learning Technology and Applications Beijing China

Building intelligent agents that can communicate with and learn from humans in natural language is of great value. Supervised language learning is limited by the ability of capturing mainly the statistics of training data, and is hardly adaptive to new scenarios or flexible for acquiring new knowledge without inefficient retraining or catastrophic forgetting. We highlight the perspective that conversational interaction serves as a natural interface both for language learning and for novel knowledge acquisition and propose a joint imitation and reinforcement approach for grounded language learning through an interactive conversational game. The agent trained with this approach is able to actively acquire information by asking questions about novel objects and use the just-learned knowledge in subsequent conversations in a one-shot fashion. Results compared with other methods verified the effectiveness of the proposed approach. Copyright © 2018, The Authors. All rights reserved.

关键词： Intelligent agents

Every Pixel Counts: Unsupervised geometry learning with holistic 3d motion understanding

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yang, Zhenheng Wang, Peng Wang, Yang Xu, Wei Nevatia, Ram University of Southern California Baidu Research National Engineering Laboratory for Deep Learning Technology and Applications

learning to estimate 3D geometry in a single image by watching unlabeled videos via deep convolutional network has made significant process recently. Current state-of-the-art (SOTA) methods, are based on the learning framework of rigid structure-from-motion, where only 3D camera ego motion is modeled for geometry estimation. However, moving objects also exist in many videos, e.g. moving cars in a street scene. In this paper, we tackle such motion by additionally incorporating per-pixel 3D object motion into the learning framework, which provides holistic 3D scene flow understanding and helps single image geometry estimation. Specifically, given two consecutive frames from a video, we adopt a motion network to predict their relative 3D camera pose and a segmentation mask distinguishing moving objects and rigid background. An optical flow network is used to estimate dense 2D per-pixel correspondence. A single image depth network predicts depth maps for both images. The four types of information, i.e. 2D flow, camera pose, segment mask and depth maps, are integrated into a differentiable holistic 3D motion parser (HMP), where per-pixel 3D motion for rigid background and moving objects are recovered. We design various losses w.r.t. the two types of 3D motions for training the depth and motion networks, yielding further error reduction for estimated geometry. Finally, in order to solve the 3D motion confusion from monocular videos, we combine stereo images into joint training. Experiments on KITTI 2015 dataset show that our estimated geometry, 3D motion and moving object masks, not only are constrained to be consistent, but also significantly outperforms other SOTA algorithms, demonstrating the benefits of our approach. Copyright © 2018, The Authors. All rights reserved.

关键词： Pixels

Occlusion aware unsupervised learning of optical flow

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Wang, Yang Yang, Yi Yang, Zhenheng Zhao, Liang Wang, Peng Xu, Wei Baidu Research University of Southern California National Engineering Laboratory for Deep Learning Technology and Applications

It has been recently shown that a convolutional neural network can learn optical flow estimation with unsuper-vised learning. However, the performance of the unsuper-vised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsuper-vised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Espe-cially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning. Copyright © 2017, The Authors. All rights reserved.

关键词： Optical flows