检索结果-内蒙古大学图书馆

31st British Machine Vision Conference, BMVC 2020

作者： Zhu, Jing Shi, Yunxiao Ren, Mengwei Fang, Yi NYU Multimedia and Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates

Monocular depth estimation is a challenging task that aims to predict a corresponding depth map from a given single RGB image. Recent deep learning models have been proposed to predict the depth from the image by learning the alignment of deep features between the RGB image and the depth domains. In this paper, we present a novel approach, named Memorable Domain Adaptation Network (MDA-Net), to more effectively transfer domain features for monocular depth estimation by taking into account the common structure regularities (e.g., repetitive structure patterns, planar surfaces, symmetries) in domain adaptation. To this end, we introduce a new Structure-Oriented Memory (SOM) module to learn and memorize the structure-specific information between RGB image domain and the depth domain. More specifically, in the SOM module, we develop a Memorable Bank of Filters (MBF) unit to learn a set of filters that memorize the structure-aware image-depth residual pattern, and also an Attention Guided Controller (AGC) unit to control the filter selection in the MBF given image features queries. Given the query image feature, the trained SOM module is able to adaptively select the best customized filters for cross-domain feature transferring with an optimal structural disparity between image and depth. In summary, we focus on addressing this structure-specific domain adaption challenge by proposing a novel end-to-end multi-scale memorable network for monocular depth estimation. The experiments show that our MDA-Net demonstrates the superior performance compared to the existing supervised monocular depth estimation approaches on the challenging KITTI and nyu Depth V2 benchmarks. © 2020. The copyright of this document resides with its authors.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

Robust Image Matching By Dynamic Feature Selection 31

Robust Image Matching By Dynamic Feature Selection

引用

31st British Machine Vision Conference, BMVC 2020

作者： Huang, Hao Chen, Jianchun Li, Xiang Wang, Linging Fang, Yi NYU Multimedia and Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates

Estimating dense correspondences between images is a long-standing image understanding task. Recent works introduce convolutional neural networks (CNNs) to extract high-level feature maps and find correspondences through feature matching. However, high-level feature maps are in low spatial resolution and therefore insufficient to provide accurate and fine-grained features to distinguish intra-class variations for correspondence matching. To address this problem, we generate robust features by dynamically selecting features at different scales. To resolve two critical issues in feature selection, i.e., how many and which scales of features to be selected, we frame the feature selection process as a sequential Markov decision-making process (MDP) and introduce an optimal selection strategy using reinforcement learning (RL). We define an RL environment for image matching in which each individual action either requires new features or terminates the selection episode by referring a matching score. Deep neural networks are incorporated into our method and trained for decision making. Experimental results show that our method achieves comparable/superior performance with state-of-the-art methods on three benchmarks, demonstrating the effectiveness of our feature selection strategy. © 2020. The copyright of this document resides with its authors.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Geometry-Aware Segmentation of Remote Sensing Images via implicit height estimation

arXiv

引用

arXiv 2020年

作者： Li, Xiang Fang, Yi NYU Multimedia and Visual Computing Lab NYU Tandon Abu DhabiUAE United States

Deep learning-based methods, especially deep convolutional neural networks (CNNs), have made significant breakthroughs in the field of remote sensing and greatly advanced the performance of the semantic segmentation of remote sensing images. Recent studies have shown the benefits of using additional elevation data (e.g., DSM or nDSM) for enhancing the performance of the semantic segmentation of aerial images. However, previous methods mostly adopt 3D elevation information as additional inputs. While in many real-world applications, one does not have the corresponding DSM information at hand and the spatial resolution of acquired DSM images usually do not match the aerial images. To alleviate this data constraint and also take the advantage of 3D elevation information, in this paper, we propose a geometry-aware segmentation model that achieves accurate semantic segmentation of aerial images via implicit height estimation. Instead of using a single-stream encoder-decoder network for semantic labeling, we design a separate decoder branch to predict the height map and use the DSM images as side supervision to train this newly designed decoder branch. In this way, our model does not require DSM as model input and still benefits from the helpful 3D geometric information during training. With the newly designed decoder branch, our model can distill the 3D geometric features from 2D appearance features under the supervision of ground truth DSM images. Moreover, we develop a new geometry-aware convolution module that fuses the 3D geometric features from the height decoder branch and the 2D contextual features from the semantic segmentation branch. The fused feature embeddings can produce geometry-aware segmentation maps with enhanced performance. Our model is trained with DSM images as side supervision, while in the inference stage, it does not require DSM data and can directly predict the semantic labels in an end-to-end fashion. Experiments on ISPRS Vaihingen and Potsdam d

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

3D UNSUPERVISED REGION-AWARE REGISTRATION TRANSFORMER

arXiv

引用

arXiv 2021年

作者： Hao, Yu Fang, Yi NYU Multimedia and Visual Computing Lab United States NYUAD Center for Artificial Intelligence and Robotics New York University Abu Dhabi Abu Dhabi United Arab Emirates

This paper concerns the research problem of point cloud registration to find the rigid transformation to optimally align the source point set with the target one. Learning robust point cloud registration models with deep neural networks has emerged as a powerful paradigm, offering promising performance in predicting the global geometric transformation for a pair of point sets. Existing methods first leverage an encoder to regress the global shape descriptor, which is then decoded into a shape-conditioned transformation via concatenation-based conditioning. However, different regions of a 3D shape vary in their geometric structures which makes it more sense that we have a region-conditioned transformation instead of the shape-conditioned one. In this paper, we define our 3D registration function through the introduction of a new design of 3D region partition module that is able to divide the input shape to different regions with a self-supervised 3D shape reconstruction loss without the need for ground truth labels. We further propose the 3D shape transformer module to efficiently and effectively capture short- and long-range geometric dependencies for regions on the 3D shape Consequently, the region-aware decoder module is proposed to predict the transformations for different regions respectively. The global geometric transformation from the source point set to the target one is then formed by the weighted fusion of region-aware transformation. Compared to the state-of-the-art approaches, our experiments show that our 3D-URRT achieves superior registration performance over various benchmark datasets (e.g. ModelNet40). Copyright © 2021, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

3D meta-segmentation neural network

arXiv

引用

arXiv 2021年

作者： Hao, Yu Fang, Yi Nyu Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi United Arab Emirates New York University New YorkNY United States

Though deep learning methods have shown great success in 3D point cloud part segmentation, they generally rely on a large volume of labeled training data, which makes the model suffer from unsatisfied generalization abilities to unseen classes with limited data. To address this problem, we present a novel meta-learning strategy that regards the 3D shape segmentation function as a task. By training over a number of 3D part segmentation tasks, our method is capable to learn the prior over the respective 3D segmentation function space which leads to an optimal model that is rapidly adapting to new part segmentation tasks. To implement our meta-learning strategy, we propose two novel modules: meta part segmentation learner and part segmentation learner. During the training process, the part segmentation learner is trained to complete a specific part segmentation task in the few-shot scenario. In the meantime, the meta part segmentation learner is trained to capture the prior from multiple similar part segmentation tasks. Based on the learned information of task distribution, our meta part segmentation learner is able to dynamically update the part segmentation learner with optimal parameters which enable our part segmentation learner to rapidly adapt and have great generalization ability on new part segmentation tasks. We demonstrate that our model achieves superior part segmentation performance with the few-shot setting on the widely used dataset: ShapeNet. Copyright © 2021, The Authors. All rights reserved.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation

arXiv

引用

arXiv 2024年

作者： Wen, Congcong Liu, Yifan Bethala, Geeta Chandra Raju Peng, Zheng Lin, Hui Liu, Yu-Shen Fang, Yi NYUAD Center for Artificial Intelligence and Robotics New York University Abu Dhabi United Arab Emirates NYU Multimedia and Visual Computing Lab New York University United States New York University Abu Dhabi United Arab Emirates Tandon School of Engineering New York University New York United States School of Cyber Science and Technology University of Science and Technology of China Anhui China School of Software Tsinghua University Beijing China

Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSACLLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at https://***/. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Height estimation from single aerial images using a deep ordinal regression network

arXiv

引用

arXiv 2020年

作者： Li, Xiang Wang, Mingyang Fang, Yi NYU Multimedia and Visual Computing Lab NYU Tandon and Abu Dhabi United Arab Emirates NYU Multimedia and Visual Computing Lab NYU Tandon United States Department of Electrical and Computer Engineering New York University Abu Dhabi United Arab Emirates

Understanding the 3D geometric structure of the Earth’s surface has been an active research topic in photogrammetry and remote sensing community for decades, serving as an essential building block for various applications such as 3D digital city modeling, change detection, and city management. Previous researches have extensively studied the problem of height estimation from aerial images based on stereo or multiview image matching. These methods require two or more images from different perspectives to reconstruct 3D coordinates with camera information provided. In this paper, we deal with the ambiguous and unsolved problem of height estimation from a single aerial image. Driven by the great success of deep learning, especially deep convolution neural networks (CNNs), some researches have proposed to estimate height information from a single aerial image by training a deep CNN model with large-scale annotated datasets. These methods treat height estimation as a regression problem and directly use an encoder-decoder network to regress the height values. In this paper, we proposed to divide height values into spacing-increasing intervals and transform the regression problem into an ordinal regression problem, using an ordinal loss for network training. To enable multi-scale feature extraction, we further incorporate an Atrous Spatial Pyramid Pooling (ASPP) module to extract features from multiple dilated convolution layers. After that, a post-processing technique is designed to transform the predicted height map of each patch into a seamless height map. Finally, we conduct extensive experiments on ISPRS Vaihingen and Potsdam datasets. Experimental results demonstrate significantly better performance of our method compared to the state-of-the-art methods. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolutional neural networks

来源：评论

学校读者我要写书评

暂无评论

Few-shot Object Detection on Remote Sensing Images

arXiv

引用

arXiv 2020年

作者： Deng, Jingyu Li, Xiang Fang, Yi Multimedia and Visual Computing Lab NYU Tandon and Abu Dhabi Department of Electrical and Computer Engineering NYU Tandon United States Department of Electrical and Computer Engineering NYU Abu Dhabi United Arab Emirates Multimedia and Visual Computing Lab NYU Tandon and Abu Dhabi Department of Electrical and Computer Engineering Abu Dhabi United Arab Emirates

In this paper, we deal with the problem of object detection on remote sensing images. Previous methods have developed numerous deep CNN-based methods for object detection on remote sensing images and the report remarkable achievements in detection performance and efficiency. However, current CNN-based methods mostly require a large number of annotated samples to train deep neural networks and tend to have limited generalization abilities for unseen object categories. In this paper, we introduce a few-shot learning-based method for object detection on remote sensing images where only a few annotated samples are provided for the unseen object categories. More specifically, our model contains three main components: a meta feature extractor that learns to extract feature representations from input images, a reweighting module that learn to adaptively assign different weights for each feature representation from the support images, and a bounding box prediction module that carries out object detection on the reweighted feature maps. We build our few-shot object detection model upon YOLOv3 architecture and develop a multi-scale object detection framework. Experiments on two benchmark datasets demonstrate that with only a few annotated samples our model can still achieve a satisfying detection performance on remote sensing images and the performance of our model is significantly better than the well-established baseline models. Copyright © 2020, The Authors. All rights reserved.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

3DMotion-Net: Learning Continuous Flow Function for 3D Motion Prediction

3DMotion-Net: Learning Continuous Flow Function for 3D Motio...

引用

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Shuaihang Yuan Xiang Li Anthony Tzes Yi Fang NYU Multimedia and Visual Computing Lab USA New York University Abu Dhabi UAE New York University USA

ISBN: (数字)9781728162126

ISBN: (纸本)9781728162133

This paper deals with predicting future 3D motions of 3D object scans from the previous two consecutive frames. Previous methods mostly focus on sparse motion prediction in the form of skeletons. While in this paper, we focus on predicting dense 3D motions in the form of 3D point clouds. To approach this problem, we propose a self-supervised approach that leverages the power of the deep neural network to learn a continuous flow function of 3D point clouds that can predict temporally consistent future motions and naturally bring out the correspondences among consecutive point clouds at the same time. More specifically, in our approach, to eliminate the unsolved and challenging process of defining a discrete point convolution on 3D point cloud sequences to encode spatial and temporal information, we introduce a learnable latent code to represent the temporal-aware shape descriptor, which is optimized during the model training. Moreover, a temporally consistent motion Morpher is proposed to learn a continuous flow field which deforms a 3D scan from the current frame to the next frame. We perform extensive experiments on D-FAUST, SCAPE, and TOSCA benchmark data sets. The results demonstrate that our approach is capable of handling temporally inconsistent input and produces consistent future 3D motion while requiring no ground truth supervision.

关键词： Training Solid modeling Three-dimensional displays Shape Convolution Benchmark testing Decoding

来源：评论

学校读者我要写书评

暂无评论

Few-Shot Learning of Part-Specific Probability Space for 3D Shape Segmentation

Few-Shot Learning of Part-Specific Probability Space for 3D ...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Lingjing Wang Xiang Li Yi Fang NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi UAE New York University New York USA

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Recently, deep neural networks are introduced as supervised discriminative models for the learning of 3D point cloud segmentation. Most previous supervised methods require a large number of training data with human annotation part labels to guide the training process to ensure the model's generalization abilities on test data. In comparison, we propose a novel 3D shape segmentation method that requires few labeled data for training. Given an input 3D shape, the training of our model starts with identifying a similar 3D shape with part annotations from a mini-pool of shape templates (e.g. 10 shapes). With the selected template shape, a novel Coherent Point Transformer is proposed to fully leverage the power of a deep neural network to smoothly morph the template shape towards the input shape. Then, based on the transformed template shapes with part labels, a newly proposed Part-specific Density Estimator is developed to learn a continuous part-specific probability distribution function on the entire 3D space with a batch consistency regularization term. With the learned part-specific probability distribution, our model is able to predict the part labels of a new input 3D shape in an end-to-end manner. We demonstrate that our proposed method can achieve remarkable segmentation results on the ShapeNet dataset with few shots, compared to previous supervised learning approaches.

关键词： Shape Three-dimensional displays Solid modeling Neural networks Training Semantics Supervised learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：