检索结果-内蒙古大学图书馆

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Wenjun Gui Bingyu Li Shuaihang Yuan John-Ross Rizzo Lakshay Sharma Chen Feng Anthony Tzes Yi Fang NYU Multimedia and Visual Computing Lab NYU Tandon Langone Medical Center NYU NYU Abu Dhabi NYU Tandon

Spatial cognition, as gained through the sense of vision, is one of the most important capabilities of human beings. However, for the visually impaired (VI), lack of this perceptual capability poses great challenges in their life. Therefore, we have designed Point-to-Tell-and-Touch, a wearable system with an ergonomic human-machine interface, for assisting the VI with active environmental exploration, with a particular focus on spatial intelligence and navigation to objects of interest in an alien environment. Our key idea is to link visual signals, as decoded synthetically, to the VI's proprioception for more intelligible guidance, in addition to vision-to-audio assistance, i.e., finger pose, as indicated by pointing, is used as “proprioceptive laser pointer” to target an object in that line of sight. The whole system consists of two features, Point-to-Tell and Point-to-Touch, both of which can work independently or cooperatively. The Point-to-Tell feature contains a camera with a novel one-stage neural network tailored for blind-centered object detection and recognition, and a headphone telling the VI the semantic label and distance from the pointed object. the Point-to-Touch, the second feature, leverages a vibrating wrist band to create a haptic feedback tool that supplements the initial vectorial guidance provided by the first stage (hand pose being direction and the distance being the extent, offered through audio cues). Both platform features utilize proprioception or joint position sense. Through hand pose, the VI end user knows where he or she is pointing relative to their egocentric coordinate system and we are able to use this foundation to build spatial intelligence. Our successful indoor experiments demonstrate the proposed system to be effective and reliable in helping the VI gain spatial cognition and explore the world in a more intuitive way.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Meta deformation network: Meta functionals for shape correspondence

arXiv

引用

arXiv 2020年

作者： Lu, Daohan Fang, Yi Multimedia and Visual Computing Lab New York University New York United States Tandon School of Engineering New York University New York United States Department of Electrical and Computer Engineering New York University Abu Dhabi United Arab Emirates College of Arts and Science New York University New York United States

We present a new technique named Meta Deformation Network for 3D shape matching via deformation, in which a deep neural network maps a reference shape onto the parameters of a second neural network whose task is to give the correspondence between a learned template and query shape via deformation. We categorize the second neural network as a meta-function, or a function generated by another function, as its parameters are dynamically given by the first network on a per-input basis. This leads to a straightforward overall architecture and faster execution speeds, without loss in the quality of the deformation of the template. We show in our experiments that Meta Deformation Network leads to improvements on the MPI-FAUST Inter Challenge over designs that utilized a conventional decoder design that has non-dynamic Copyright © 2020, The Authors. All rights reserved.

关键词： Deformation

来源：评论

学校读者我要写书评

暂无评论

Learning object-specific distance from a monocular image

arXiv

引用

arXiv 2019年

作者： Zhu, Jing Fang, Yi Abu-Haimed, Husam Lien, Kuo-Chin Fu, Dongdong Gu, Junli NYU Multimedia and Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates XMotors.ai

Environment perception, including object detection and distance estimation, is one of the most crucial tasks for autonomous driving. Many attentions have been paid on the object detection task, but distance estimation only arouse few interests in the computer vision community. Observing that the traditional inverse perspective mapping algorithm performs poorly for objects far away from the camera or on the curved road, in this paper, we address the challenging distance estimation problem by developing the first end-to-end learning-based model to directly predict distances for given objects in the images. Besides the introduction of a learning-based base model, we further design an enhanced model with a keypoint regressor, where a projection loss is defined to enforce a better distance estimation, especially for objects close to the camera. To facilitate the research on this task, we construct the extented KITTI and nuScenes (mini) object detection datasets with a distance for each object. Our experiments demonstrate that our proposed methods outperform alternative approaches (e.g., the traditional IPM, SVR) on object-specific distance estimation, particularly for the challenging cases that objects are on a curved road. Moreover, the performance margin implies the effectiveness of our enhanced method. Copyright © 2019, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

Structure-Attentioned memory network for monocular depth estimation

arXiv

引用

arXiv 2019年

作者： Zhu, Jing Shi, Yunxiao Ren, Mengwei Fang, Yi Lien, Kuo-Chin Gu, Junli NYU Multimedia and Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates XMotors.ai

Monocular depth estimation is a challenging task that aims to predict a corresponding depth map from a given single RGB image. Recent deep learning models have been proposed to predict the depth from the image by learning the alignment of deep features between the RGB image and the depth domains. In this paper, we present a novel approach, named Structure-Attentioned Memory Network, to more effectively transfer domain features for monocular depth estimation by taking into account the common structure regularities (e.g., repetitive structure patterns, planar surfaces, symmetries) in domain adaptation. To this end, we introduce a new Structure-Oriented Memory (SOM) module to learn and memorize the structure-specific information between RGB image domain and the depth domain. More specifically, in the SOM module, we develop a Memorable Bank of Filters (MBF) unit to learn a set of filters that memorize the structure-aware image-depth residual pattern, and also an Attention Guided Controller (AGC) unit to control the filter selection in the MBF given image features queries. Given the query image feature, the trained SOM module is able to adaptively select the best customized filters for cross-domain feature transferring with an optimal structural disparity between image and depth. In summary, we focus on addressing this structure-specific domain adaption challenge by proposing a novel end-to-end multi-scale memorable network for monocular depth estimation. The experiments show that our proposed model demonstrates the superior performance compared to the existing supervised monocular depth estimation approaches on the challenging KITTI and NYU Depth V2 benchmarks. Copyright © 2019, The Authors. All rights reserved.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Self-supervised learning of depth and ego-motion with differentiable bundle adjustment

arXiv

引用

arXiv 2019年

作者： Shi, Yunxiao Zhu, Jing Fang, Yi Lien, Kuochin Gu, Junli NYU Multimedia Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates XMotors.ai United States

Learning to predict scene depth and camera motion from RGB inputs only is a challenging task. Most existing learning based methods deal with this task in a supervised manner which require ground-truth data that is expensive to acquire. More recent approaches explore the possibility of estimating scene depth and camera pose in a self-supervised learning framework. Despite encouraging results are shown, current methods either learn from monocular videos for depth and pose and typically do so without enforcing multi-view geometry constraints between scene structure and camera motion, or require stereo sequences as input where the ground-truth between-frame motion parameters need to be known. In this paper we propose to jointly optimize the scene depth and camera motion via incorporating differentiable Bundle Adjustment (BA) layer by minimizing the feature-metric error, and then form the photometric consistency loss with view synthesis as the final supervisory signal. The proposed approach only needs unlabeled monocular videos as input, and extensive experiments on the KITTI and Cityscapes dataset show that our method achieves state-of-the-art results in self-supervised approaches using monocular videos as input, and even gains advantage to the line of methods that learns from calibrated stereo sequences (i.e. with pose supervision). Copyright © 2019, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

Density-aware convolutional networks with context encoding for airborne lidar point cloud classification

arXiv

引用

arXiv 2019年

作者： Li, Xiang Wang, Mingyang Wen, Congcong Wang, Lingjing Zhou, Nan Fang, Yi Multimedia and Visual Computing Lab New York University New York United States Tandon School of Engineering New York University New York United States Aerospace Information Research Institute Chinese Academy of Sciences

To better address challenging issues of the irregularity and inhomogeneity inherently present in 3D point clouds, researchers have shifted their focus from designing hand-crafted point features toward learning 3D point signatures using deep neural networks for 3D point cloud classification. Recently proposed deep learning-based point cloud classification methods either apply 2D CNNs on projected feature images or apply 1D convolutional layers directly on raw point sets. These methods cannot adequately recognize fine-grained local structures caused by the uneven density distribution of the point cloud data. In this paper, to address this challenging issue, we introduced a density-aware convolutional module that uses the pointwise density to reweight the learnable weights of convolution kernels. The proposed convolution module can fully approximate the 3D continuous convolution on unevenly distributed 3D point sets. Based on this convolution module, we further developed a multiscale fully convolutional neural network with downsampling and upsampling blocks to enable hierarchical point feature learning. In addition, to regularize the global semantic context, we implemented a context encoding module to predict a global context encoding and formulated a context encoding regularizer to enforce the predicted context encoding to align with the ground truth encoding. The overall network can be trained in an end-to-end fashion with the raw 3D coordinates and the height above ground as inputs. Experiments on the International Society for Photogrammetry and Remote Sensing (ISPRS) 3D labeling benchmark demonstrated the superiority of the proposed method for point cloud classification. Our model achieved a new state-of-the-art performance with an average F1 score of 71.2% and improved the performance by a large margin on several categories (such as powerline, impervious surface, car, and facade). Copyright © 2019, The Authors. All rights reserved.

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

Unsupervised 3D reconstruction from a single image via adversarial learning

arXiv

引用

arXiv 2017年

作者： Wang, Lingjing Fang, Yi NYU Multimedia and Visual Computing Lab Courant Institute of Mathematical Science NYU Tandon School of Engineering United States NYU Multimedia and Visual Computing Lab Dept. of ECE NYU Abu Dhabi United Arab Emirates

Recent advancements in deep learning opened new opportunities for learning a high-quality 3D model from a single 2D image given sufficient training on large-scale data sets. However, the significant imbalance between available amount of images and 3D models, and the limited availability of labeled 2D image data (i.e. manually annotated pairs between images and their corresponding 3D models), severely impacts the training of most supervised deep learning methods in practice. In this paper, driven by a novel design of adversarial networks, we have developed an unsupervised learning paradigm to reconstruct 3D models from a single 2D image, which is free of manually annotated pairwise input image and its associated 3D model. Particularly, the paradigm begins with training an adaption network via autoencoder with adversarial loss, which embeds unpaired 2D synthesized image domain with real world image domain to a shared latent vector space. Then, we jointly train a 3D deconvolutional network to transform the latent vector space to the 3D object space together with the embedding process. Our experiments verify our network's robust and superior performance in handling 3D volumetric object generation from a single 2D image. Copyright © 2017, The Authors. All rights reserved.

关键词： Vector spaces

来源：评论

学校读者我要写书评

暂无评论

Paintings alive: A virtual reality-based approach for enhancing the user experience of art gallery visitors 6th

Paintings alive: A virtual reality-based approach for enhanc...

引用

6th International European-Mediterranean Conference on Digital Heritage, EuroMed 2016

作者： Panayiotou, Stavros Lanitis, Andreas Visual Media Computing Research Lab Department of Multimedia and Graphic Arts Cyprus University of Technology Lemesos Cyprus

ISBN: (纸本)9783319489735

Visits to art museums are not appealing to a wide sector of the public, limiting in that way the exposure of art works to the wider community and especially the younger generation. In this paper we investigate the use of Virtual Reality (VR) for displaying animated artworks as a means for creating an enhanced user experience that could make visits to art-galleries more appealing. A key point of the proposed technique is the analysis of the art-works that enables the determination of a most appropriate setting and animation that best promotes the original message of the painter. The promise of this approach has been validated through an experimental user evaluation where users clearly indicated the benefits of using interactive virtual environments for presenting animated art paintings. © Springer International Publishing AG 2016.

关键词： Virtual reality

来源：评论

学校读者我要写书评

暂无评论

3D-A-nets: 3D deep dense descriptor for volumetric shapes with adversarial networks

arXiv

引用

arXiv 2017年

作者： Ren, Mengwei Niu, Liang Fang, Yi NYU Multimedia and Visual Computing Lab NYU Abu Dhabi United Arab Emirates New York University BrooklynNY United States 5 Metrotech Center LC024 BrooklynNY11201 United States

Recently researchers have been shifting their focus towards learned 3D shape descriptors from hand-craft ones to better address challenging issues of the deformation and structural variation inherently present in 3D objects. 3D geometric data are often transformed to 3D Voxel grids with regular format in order to be better fed to a deep neural net architecture. However, the computational intractability of direct application of 3D convolutional nets to 3D volumetric data severely limits the efficiency (i.e. slow processing) and effectiveness (i.e. unsatisfied accuracy) in processing 3D geometric data. In this paper, powered with a novel design of adversarial networks (3D-A-Nets), we have developed a novel 3D deep dense shape descriptor (3D-DDSD) to address the challenging issues of efficient and effective 3D volumetric data processing. We developed new definition of 2D multilayer dense representation (MDR) of 3D volumetric data to extract concise but geometrically informative shape description and a novel design of adversarial networks that jointly train a set of convolution neural network (CNN), recurrent neural network (RNN) and an adversarial discriminator. More specifically, the generator network produces 3D shape features that encourages the clustering of samples from the same category with correct class label, whereas the discriminator network discourages the clustering by assigning them misleading adversarial class labels. By addressing the challenges posed by the computational inefficiency of direct application of CNN to 3D volumetric data, 3D-A-Nets can learn high-quality 3DDSDD which demonstrates superior performance on 3D shape classification and retrieval over other state-of-the-art techniques by a great margin. Copyright © 2017, The Authors. All rights reserved.

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

ArbiText: Arbitrary-oriented text detection in unconstrained scene

arXiv

引用

arXiv 2017年

作者： Xing, Daitao Li, Zichen Chen, Xin Fang, Yi NYU Multimedia and Visual Computing Lab NYU Abu Dhabi United Arab Emirates New York University BrooklynNY United States HERE Technologies 5 Metrotech Center LC024 BrooklynNY11201 United States

Arbitrary-oriented text detection in the wild is a very challenging task, due to the aspect ratio, scale, orientation, and illumination variations. In this paper, we propose a novel method, namely Arbitrary-oriented Text (or ArbText for short) detector, for efficient text detection in unconstrained natural scene images. Specifically, we first adopt the circle anchors rather than the rectangular ones to represent bounding boxes, which is more robust to orientation variations. Subsequently, we incorporate a pyramid pooling module into the Single Shot MultiBox Detector framework, in order to simultaneously explore the local and global visual information, which can therefore generate more confidential detection results. Experiments on established scene-text datasets, such as the ICDAR 2015 and MSRA-TD500 datasets, have demonstrated the superior performance of the proposed method, compared to the state-of-the-art approaches. Copyright © 2017, The Authors. All rights reserved.

关键词： Aspect ratio

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：