检索结果-内蒙古大学图书馆

Few-Shot Learning of Part-Specific Probability Space for 3D Shape Segmentation

学校读者我要写书评

暂无评论

Few-Shot Learning of Part-Specific Probability Space for 3D ...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Lingjing Wang Xiang Li Yi Fang NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi UAE New York University New York USA

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Recently, deep neural networks are introduced as supervised discriminative models for the learning of 3D point cloud segmentation. Most previous supervised methods require a large number of training data with human annotation part labels to guide the training process to ensure the model's generalization abilities on test data. In comparison, we propose a novel 3D shape segmentation method that requires few labeled data for training. Given an input 3D shape, the training of our model starts with identifying a similar 3D shape with part annotations from a mini-pool of shape templates (e.g. 10 shapes). With the selected template shape, a novel Coherent Point Transformer is proposed to fully leverage the power of a deep neural network to smoothly morph the template shape towards the input shape. Then, based on the transformed template shapes with part labels, a newly proposed Part-specific Density Estimator is developed to learn a continuous part-specific probability distribution function on the entire 3D space with a batch consistency regularization term. With the learned part-specific probability distribution, our model is able to predict the part labels of a new input 3D shape in an end-to-end manner. We demonstrate that our proposed method can achieve remarkable segmentation results on the ShapeNet dataset with few shots, compared to previous supervised learning approaches.

关键词： Shape Three-dimensional displays Solid modeling Neural networks Training Semantics Supervised learning

Unsupervised Deep Shape Descriptor With Point Distribution Learning

学校读者我要写书评

暂无评论

Unsupervised Deep Shape Descriptor With Point Distribution L...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yi Shi Mengchen Xu Shuaihang Yuan Yi Fang NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi UAE New York University New York USA

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Deep learning models have achieved great success in supervised shape descriptor learning for 3D shape retrieval, classification, and correspondence. However, the unsupervised shape descriptor calculated via deep learning is less studied than that of supervised ones due to the design challenges of unsupervised neural network architecture. This paper proposes a novel probabilistic framework for the learning of unsupervised deep shape descriptors with point distribution learning. In our approach, we firstly associate each point with a Gaussian, and the point clouds are modeled as the distribution of the points. We then use deep neural networks (DNNs) to model a maximum likelihood estimation process that is traditionally solved with an iterative Expectation-Maximization (EM) process. Our key novelty is that ``training'' these DNNs with unsupervised self-correspondence L2 distance loss will elegantly reveal the statically significant deep shape descriptor representation for the distribution of the point clouds. We have conducted experiments over various 3D datasets. Qualitative and quantitative comparisons demonstrate that our proposed method achieves superior classification performance over existing unsupervised 3D shape descriptors. In addition, we verified the following attractive properties of our shape descriptor through experiments: multi-scale shape representation, robustness to shape rotation, and robustness to noise.

关键词： Three-dimensional displays Shape Robustness Decoding Solid modeling Training Neural networks

3DMotion-Net: Learning Continuous Flow Function for 3D Motion Prediction

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yuan, Shuaihang Li, Xiang Tzes, Anthony Fang, Yi NYU Multimedia and Visual Computing Lab United States New York University Abu Dhabi United Arab Emirates New York University United States

In this paper, we deal with the problem to predict the future 3D motions of 3D object scans from previous two consecutive frames. Previous methods mostly focus on sparse motion prediction in the form of skeletons. While in this paper we focus on predicting dense 3D motions in the from of 3D point clouds. To approach this problem, we propose a self-supervised approach that leverages the power of the deep neural network to learn a continuous flow function of 3D point clouds that can predict temporally consistent future motions and naturally bring out the correspondences among consecutive point clouds at the same time. More specifically, in our approach, to eliminate the unsolved and challenging process of defining a discrete point convolution on 3D point cloud sequences to encode spatial and temporal information, we introduce a learnable latent code to represent the temporal-aware shape descriptor which is optimized during model training. Moreover, a temporally consistent motion Morpher is proposed to learn a continuous flow field which deforms a 3D scan from the current frame to the next frame. We perform extensive experiments on D-FAUST, SCAPE and TOSCA benchmark data sets and the results demonstrate that our approach is capable of handling temporally inconsistent input and produces consistent future 3D motion while requiring no ground truth supervision. Copyright © 2020, The Authors. All rights reserved.

关键词： Motion estimation

DeepTracking-Net: 3D Tracking with Unsupervised Learning of Continuous Flow

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yuan, Shuaihang Li, Xiang Fang, Yi NYU Multimedia and Visual Computing Lab United States New York University Abu Dhabi United Arab Emirates New York University United States

This paper deals with the problem of 3D tracking, i.e., to find dense correspondences in a sequence of time-varying 3D shapes. Despite deep learning approaches have achieved promising performance for pairwise dense 3D shapes matching, it is a great challenge to generalize those approaches for the tracking of 3D time-varying geometries. In this paper, we aim at handling the problem of 3D tracking, which provides the tracking of the consecutive frames of 3D shapes. We propose a novel unsupervised 3D shape registration framework named DeepTracking-Net, which uses the deep neural networks (DNNs) as auxiliary functions to produce spatially and temporally continuous displacement fields for 3D tracking of objects in a temporal order. Our key novelty is that we present a novel temporal-aware correspondence descriptor (TCD) that captures spatio-temporal essence from consecutive 3D point cloud frames. Specifically, our DeepTracking-Net starts with optimizing a randomly initialized latent TCD. The TCD is then decoded to regress a continuous flow (i.e. a displacement vector field) which assigns a motion vector to every point of time-varying 3D shapes. Our DeepTracking-Net jointly optimizes TCDs and DNNs’ weights towards the minimization of an unsupervised alignment loss. Experiments on both simulated and real data sets demonstrate that our unsupervised DeepTracking-Net outperforms the current supervised state-of-the-art method. In addition, we prepare a new synthetic 3D data, named SynMotions, to the 3D tracking and recognition community. Copyright © 2020, The Authors. All rights reserved.

关键词： Deep neural networks

Robust Image Matching by Dynamic Feature Selection 0

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Huang, Hao Fang, Yi Wang, Linging Li, Xiang Chen, Jianchun NYU Multimedia and Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates

Estimating dense correspondences between images is a long-standing image understanding task. Recent works introduce convolutional neural networks (CNNs) to extract high-level feature maps and find correspondences through feature matching. However, high-level feature maps are in low spatial resolution and therefore insufficient to provide accurate and fine-grained features to distinguish intra-class variations for correspondence matching. To address this problem, we generate robust features by dynamically selecting features at different scales. To resolve two critical issues in feature selection, i.e., how many and which scales of features to be selected, we frame the feature selection process as a sequential Markov decision-making process (MDP) and introduce an optimal selection strategy using reinforcement learning (RL). We define an RL environment for image matching in which each individual action either requires new features or terminates the selection episode by referring a matching score. Deep neural networks are incorporated into our method and trained for decision making. Experimental results show that our method achieves comparable/superior performance with state-of-the-art methods on three benchmarks, demonstrating the effectiveness of our feature selection strategy. Copyright © 2020, The Authors. All rights reserved.

关键词： Feature extraction

Active Crowd Analysis for Pandemic Risk Mitigation for Blind or visually Impaired Persons 16th

学校读者我要写书评

暂无评论

Active Crowd Analysis for Pandemic Risk Mitigation for Blind...

Workshops held at the 16th European Conference on Computer Vision, ECCV 2020

作者： Shrestha, Samridha Lu, Daohan Tian, Hanlin Cao, Qiming Liu, Julie Rizzo, John-Ross Seiple, William H. Porfiri, Maurizio Fang, Yi NYU Multimedia and Visual Computing Lab Abu Dhabi and New York United Arab Emirates New York University Abu Dhabi Abu Dhabi129188 United Arab Emirates New York University New YorkNY10012 United States Lighthouse Guild New YorkNY10023 United States

ISBN: (纸本)9783030668228

During pandemics like COVID-19, social distancing is essential to combat the rise of infections. However, it is challenging for the visually impaired to practice social distancing as their low vision hinders them from maintaining a safe physical distance from other humans. In this paper, we propose a smartphone-based computationally-efficient deep neural network to detect crowds and relay the associated risks to the Blind or visually Impaired (BVI) user through directional audio alerts. The system first detects humans and estimates their distances from the smartphone’s monocular camera feed. Then, the system clusters humans into crowds to generate density and distance maps from the crowd centers. Finally, the system tracks detections in previous frames creating motion maps predicting the motion of crowds to generate an appropriate audio alert. Active Crowd Analysis is designed for real-time smartphone use, utilizing the phone’s native hardware to ensure the BVI can safely maintain social distancing. © 2020, Springer Nature Switzerland AG.

关键词： Smartphones

3D meta-registration: Learning to learn registration of 3D point clouds

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Lingjing Hao, Yu Li, Xiang Fang, Yi NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi United Arab Emirates New York University New York United States

Deep learning-based point cloud registration models are often generalized from extensive training over a large volume of data to learn the ability to predict the desired geometric transformation to register 3D point clouds. In this paper, we propose a meta-learning based 3D registration model, named 3D Meta-Registration, that is capable of rapidly adapting and well generalizing to new 3D registration tasks for unseen 3D point clouds. Our 3D Meta-Registration gains a competitive advantage by training over a variety of 3D registration tasks, which leads to an optimized model for the best performance on the distribution of registration tasks including potentially unseen tasks. Specifically, the proposed 3D Meta-Registration model consists of two modules: 3D registration learner and 3D registration meta-learner. During the training, the 3D registration learner is trained to complete a specific registration task aiming to determine the desired geometric transformation that aligns the source point cloud with the target one. In the meantime, the 3D registration meta-learner is trained to provide the optimal parameters to update the 3D registration learner based on the learned task distribution. After training, the 3D registration meta-learner, which is learned with the optimized coverage of distribution of 3D registration tasks, is able to dynamically update 3D registration learners with desired parameters to rapidly adapt to new registration tasks. We tested our model on synthesized dataset ModelNet and FlyingThings3D, as well as real-world dataset KITTI. Experimental results demonstrate that 3D Meta-Registration achieves superior performance over other previous techniques (e.g. FlowNet3D). © 2020, CC0.

关键词： Competition

Unsupervised partial point set registration via joint shape completion and registration

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Xiang Wang, Lingjing Fang, Yi NYU Multimedia and Visual Computing Lab NYU Tandon Abu Dhabi United Arab Emirates Department of Electrical and Computer Engineering NYU Tandon United States Department of Electrical and Computer Engineering NYU Abu Dhabi United Arab Emirates

We propose a self-supervised method for partial point set registration. While recent proposed learning-based methods have achieved impressive registration performance on the full shape observations, these methods mostly suffer from performance degradation when dealing with partial shapes. To bridge the performance gaps between partial point set registration with full point set registration, we proposed to incorporate a shape completion network to benefit the registration process. To achieve this, we design a latent code for each pair of shapes, which can be regarded as a geometric encoding of the target shape. By doing so, our model does need an explicit feature embedding network to learn the feature encodings. More importantly, both our shape completion network and the point set registration network take the shared latent codes as input, which are optimized along with the parameters of two decoder networks in the training process. Therefore, the point set registration process can thus benefit from the joint optimization process of latent codes, which are enforced to represent the information of full shape instead of partial ones. In the inference stage, we fix the network parameter and optimize the latent codes to get the optimal shape completion and registration results. Our proposed method is pure unsupervised and does not need any ground truth supervision. Experiments on the ModelNet40 dataset demonstrate the effectiveness of our model for partial point set registration. Copyright © 2020, The Authors. All rights reserved.

关键词： Encoding (symbols)

Learning Object-Specific Distance From a Monocular Image

学校读者我要写书评

暂无评论

Learning Object-Specific Distance From a Monocular Image

International Conference on Computer Vision (ICCV)

作者： Jing Zhu Yi Fang NYU Multimedia and Visual Computing Lab USA New York University USA New York University Abu Dhabi UAE

ISBN: (数字)9781728148038

ISBN: (纸本)9781728148045

Environment perception, including object detection and distance estimation, is one of the most crucial tasks for autonomous driving. Many attentions have been paid on the object detection task, but distance estimation only arouse few interests in the computer vision community. Observing that the traditional inverse perspective mapping algorithm performs poorly for objects far away from the camera or on the curved road, in this paper, we address the challenging distance estimation problem by developing the first end-to-end learning-based model to directly predict distances for given objects in the images. Besides the introduction of a learning-based base model, we further design an enhanced model with a keypoint regressor, where a projection loss is defined to enforce a better distance estimation, especially for objects close to the camera. To facilitate the research on this task, we construct the extented KITTI and nuScenes (mini) object detection datasets with a distance for each object. Our experiments demonstrate that our proposed methods outperform alternative approaches (e.g., the traditional IPM, SVR) on object-specific distance estimation, particularly for the challenging cases that objects are on a curved road. Moreover, the performance margin implies the effectiveness of our enhanced method.

关键词： Estimation Feature extraction Cameras Predictive models Task analysis Roads Training