检索结果-内蒙古大学图书馆

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Wenjun Gui Bingyu Li Shuaihang Yuan John-Ross Rizzo Lakshay Sharma Chen Feng Anthony Tzes Yi Fang NYU Multimedia and Visual Computing Lab NYU Tandon Langone Medical Center NYU NYU Abu Dhabi NYU Tandon

Spatial cognition, as gained through the sense of vision, is one of the most important capabilities of human beings. However, for the visually impaired (VI), lack of this perceptual capability poses great challenges in their life. Therefore, we have designed Point-to-Tell-and-Touch, a wearable system with an ergonomic human-machine interface, for assisting the VI with active environmental exploration, with a particular focus on spatial intelligence and navigation to objects of interest in an alien environment. Our key idea is to link visual signals, as decoded synthetically, to the VI's proprioception for more intelligible guidance, in addition to vision-to-audio assistance, i.e., finger pose, as indicated by pointing, is used as “proprioceptive laser pointer” to target an object in that line of sight. The whole system consists of two features, Point-to-Tell and Point-to-Touch, both of which can work independently or cooperatively. The Point-to-Tell feature contains a camera with a novel one-stage neural network tailored for blind-centered object detection and recognition, and a headphone telling the VI the semantic label and distance from the pointed object. the Point-to-Touch, the second feature, leverages a vibrating wrist band to create a haptic feedback tool that supplements the initial vectorial guidance provided by the first stage (hand pose being direction and the distance being the extent, offered through audio cues). Both platform features utilize proprioception or joint position sense. Through hand pose, the VI end user knows where he or she is pointing relative to their egocentric coordinate system and we are able to use this foundation to build spatial intelligence. Our successful indoor experiments demonstrate the proposed system to be effective and reliable in helping the VI gain spatial cognition and explore the world in a more intuitive way.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Learned binary spectral shape descriptor for 3D shape correspondence

Learned binary spectral shape descriptor for 3D shape corres...

引用

2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016

作者： Xie, Jin Wang, Meng Fang, Yi NYU Multimedia and Visual Computing Lab United Arab Emirates Department of Electrical and Computer Engineering New York University Abu Dhabi United Arab Emirates Department of Electrical and Computer Engineering NYU Tandon School of Engineering United States

ISBN: (纸本)9781467388511

Dense 3D shape correspondence is an important problem in computer vision and computer graphics. Recently, the local shape descriptor based 3D shape correspondence approaches have been widely studied, where the local shape descriptor is a real-valued vector to characterize the geometrical structure of the shape. Different from these realvalued local shape descriptors, in this paper, we propose to learn a novel binary spectral shape descriptor with the deep neural network for 3D shape correspondence. The binary spectral shape descriptor can require less storage space and enable fast matching. First, based on the eigenvectors of the Laplace-Beltrami operator, we construct a neural network to form a nonlinear spectral representation to characterize the shape. Then, for the defined positive and negative points on the shapes, we train the constructed neural network by minimizing the errors between the outputs and their corresponding binary descriptors, minimizing the variations of the outputs of the positive points and maximizing the variations of the outputs of the negative points, simultaneously. Finally, we binarize the output of the neural network to form the binary spectral shape descriptor for shape correspondence. The proposed binary spectral shape descriptor is evaluated on the SCAPE and TOSCA 3D shape datasets for shape correspondence. The experimental results demonstrate the effectiveness of the proposed binary shape descriptor for the shape correspondence task.

关键词： Object recognition

来源：评论

学校读者我要写书评

暂无评论

AN INTEGRATED FRAMEWORK FOR DEVELOPING AND EVALUATING AN AUTOMATED LECTURE STYLE ASSESSMENT SYSTEM

arXiv

引用

arXiv 2023年

作者： Dimitriadou, Eleni Lanitis, Andreas Visual Media Computing Lab Department of Multimedia and Graphic Arts Cyprus University of Technology Limassol Cyprus CYENS Centre of Excellence Nicosia Cyprus

The aim of the work presented in this paper is to develop and evaluate an integrated system that provides automated lecture style evaluation, allowing teachers to get instant feedback related to the goodness of their lecturing style. The proposed system aims to promote improvement of lecture quality, that could upgrade the overall student learning experience. The proposed application utilizes specific measurable biometric characteristics, such as facial expressions, body activity, speech rate and intonation, hand movement, and facial pose, extracted from a video showing the lecturer from the audience point of view. Measurable biometric features extracted during a lecture are combined to provide teachers with a score reflecting lecture style quality both at frame rate and by providing lecture quality metrics for the whole lecture. The acceptance of the proposed lecture style evaluation system was evaluated by chief education officers, teachers and students regarding the functionality, usefulness of the application, and possible improvements. The results indicate that participants found the application novel and useful in providing automated feedback regarding lecture quality. Furthermore, the performance evaluation of the proposed system was compared with the performance of humans in the task of lecture style evaluation. Results indicate that the proposed system not only achieves similar performance to human observers, but in some cases, it outperforms them. Copyright © 2023, The Authors. All rights reserved.

关键词： Biometrics

来源：评论

学校读者我要写书评

暂无评论

Detect and Approach: Close-Range Navigation Support for People with Blindness and Low Vision 17th

Detect and Approach: Close-Range Navigation Support for Pe...

引用

17th European Conference on Computer Vision, ECCV 2022

作者： Hao, Yu Feng, Junchi Rizzo, John-Ross Wang, Yao Fang, Yi NYU Multimedia and Visual Computing Lab New York United States NYU Tandon School of Engineering New York University New York United States New York University Abu Dhabi Abu Dhabi United Arab Emirates NYU Langone Health New York United States

ISBN: (纸本)9783031250743

People with blindness and low vision (pBLV) experience significant challenges when locating final destinations or targeting specific objects in unfamiliar environments. Furthermore, besides initially locating and orienting oneself to a target object, approaching the final target from one’s present position is often frustrating and challenging, especially when one drifts away from the initial planned path to avoid obstacles. In this paper, we develop a novel wearable navigation solution to provide real-time guidance for a user to approach a target object of interest efficiently and effectively in unfamiliar environments. Our system contains two key visual computing functions: initial target object localization in 3D and continuous estimation of the user’s trajectory, both based on the 2D video captured by a low-cost monocular camera mounted on in front of the chest of the user. These functions enable the system to suggest an initial navigation path, continuously update the path as the user moves, and offer timely recommendation about the correction of the user’s path. Our experiments demonstrate that our system is able to operate with an error of less than 0.5 m both outdoor and indoor. The system is entirely vision-based and does not need other sensors for navigation, and the computation can be run with the Jetson processor in the wearable system to facilitate real-time navigation assistance. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Eye protection

来源：评论

学校读者我要写书评

暂无评论

FairCLIP: Harnessing Fairness in Vision-Language Learning

FairCLIP: Harnessing Fairness in Vision-Language Learning

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yan Luo Min Shi Muhammad Osama Khan Muhammad Muneeb Afzal Hao Huang Shuaihang Yuan Yu Tian Luo Song Ava Kouhana Tobias Elze Yi Fang Mengyu Wang Harvard Ophthalmology AI Lab Harvard University Tandon School of Engineering New York University Multimedia and Visual Computing Lab New York University Abu Dhabi

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Fairness is a critical concern in deep learning, especially in healthcare, where these models influence diagnoses and treatment decisions. Although fairness has been investigated in the vision-only domain, the fairness of medical vision-language (VL) models remains unexplored due to the scarcity of medical VL datasets for studying fairness. To bridge this research gap, we introduce the first fair vision-language medical dataset (Harvard-FairVLMed) that provides detailed demographic attributes, ground-truth labels, and clinical notes to facilitate an in-depth examination of fairness within VL foundation models. Using Harvard-FairVLMed, we conduct a comprehensive fairness analysis of two widely-used VL models (CLIP and BLIP2), pre-trained on both natural and medical domains, across four different protected attributes. Our results highlight significant biases in all VL models, with Asian, Male, Non-Hispanic, and Spanish being the preferred subgroups across the protected attributes of race, gender, ethnicity, and language, respectively. In order to alleviate these biases, we propose FairCLIP an optimal-transport-based approach that achieves a favorable trade-off between performance and fairness by reducing the Sinkhorn distance between the overall sample distribution and the distributions corresponding to each demographic group. As the first VL dataset of its kind, Harvard-FairVLMed holds the potential to catalyze advancements in the development of machine learning models that are both ethically aware and clinically effective. Our dataset and code are available at https://***/datasets/harvard-fairvlmed10k.

关键词： Deep learning Bridges Analytical models Ethics Computer vision Codes Computational modeling

来源：评论

学校读者我要写书评

暂无评论

Few-shot Object Detection on Remote Sensing Images

arXiv

引用

arXiv 2020年

作者： Deng, Jingyu Li, Xiang Fang, Yi Multimedia and Visual Computing Lab NYU Tandon and Abu Dhabi Department of Electrical and Computer Engineering NYU Tandon United States Department of Electrical and Computer Engineering NYU Abu Dhabi United Arab Emirates Multimedia and Visual Computing Lab NYU Tandon and Abu Dhabi Department of Electrical and Computer Engineering Abu Dhabi United Arab Emirates

In this paper, we deal with the problem of object detection on remote sensing images. Previous methods have developed numerous deep CNN-based methods for object detection on remote sensing images and the report remarkable achievements in detection performance and efficiency. However, current CNN-based methods mostly require a large number of annotated samples to train deep neural networks and tend to have limited generalization abilities for unseen object categories. In this paper, we introduce a few-shot learning-based method for object detection on remote sensing images where only a few annotated samples are provided for the unseen object categories. More specifically, our model contains three main components: a meta feature extractor that learns to extract feature representations from input images, a reweighting module that learn to adaptively assign different weights for each feature representation from the support images, and a bounding box prediction module that carries out object detection on the reweighted feature maps. We build our few-shot object detection model upon YOLOv3 architecture and develop a multi-scale object detection framework. Experiments on two benchmark datasets demonstrate that with only a few annotated samples our model can still achieve a satisfying detection performance on remote sensing images and the performance of our model is significantly better than the well-established baseline models. Copyright © 2020, The Authors. All rights reserved.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

Few-Shot Learning of Part-Specific Probability Space for 3D Shape Segmentation

Few-Shot Learning of Part-Specific Probability Space for 3D ...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Lingjing Wang Xiang Li Yi Fang NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi UAE New York University New York USA

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Recently, deep neural networks are introduced as supervised discriminative models for the learning of 3D point cloud segmentation. Most previous supervised methods require a large number of training data with human annotation part labels to guide the training process to ensure the model's generalization abilities on test data. In comparison, we propose a novel 3D shape segmentation method that requires few labeled data for training. Given an input 3D shape, the training of our model starts with identifying a similar 3D shape with part annotations from a mini-pool of shape templates (e.g. 10 shapes). With the selected template shape, a novel Coherent Point Transformer is proposed to fully leverage the power of a deep neural network to smoothly morph the template shape towards the input shape. Then, based on the transformed template shapes with part labels, a newly proposed Part-specific Density Estimator is developed to learn a continuous part-specific probability distribution function on the entire 3D space with a batch consistency regularization term. With the learned part-specific probability distribution, our model is able to predict the part labels of a new input 3D shape in an end-to-end manner. We demonstrate that our proposed method can achieve remarkable segmentation results on the ShapeNet dataset with few shots, compared to previous supervised learning approaches.

关键词： Shape Three-dimensional displays Solid modeling Neural networks Training Semantics Supervised learning

来源：评论

学校读者我要写书评

暂无评论

Unsupervised Deep Shape Descriptor With Point Distribution Learning

Unsupervised Deep Shape Descriptor With Point Distribution L...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yi Shi Mengchen Xu Shuaihang Yuan Yi Fang NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi UAE New York University New York USA

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Deep learning models have achieved great success in supervised shape descriptor learning for 3D shape retrieval, classification, and correspondence. However, the unsupervised shape descriptor calculated via deep learning is less studied than that of supervised ones due to the design challenges of unsupervised neural network architecture. This paper proposes a novel probabilistic framework for the learning of unsupervised deep shape descriptors with point distribution learning. In our approach, we firstly associate each point with a Gaussian, and the point clouds are modeled as the distribution of the points. We then use deep neural networks (DNNs) to model a maximum likelihood estimation process that is traditionally solved with an iterative Expectation-Maximization (EM) process. Our key novelty is that ``training'' these DNNs with unsupervised self-correspondence L2 distance loss will elegantly reveal the statically significant deep shape descriptor representation for the distribution of the point clouds. We have conducted experiments over various 3D datasets. Qualitative and quantitative comparisons demonstrate that our proposed method achieves superior classification performance over existing unsupervised 3D shape descriptors. In addition, we verified the following attractive properties of our shape descriptor through experiments: multi-scale shape representation, robustness to shape rotation, and robustness to noise.

关键词： Three-dimensional displays Shape Robustness Decoding Solid modeling Training Neural networks

来源：评论

学校读者我要写书评

暂无评论

3DMotion-Net: Learning Continuous Flow Function for 3D Motion Prediction

arXiv

引用

arXiv 2020年

作者： Yuan, Shuaihang Li, Xiang Tzes, Anthony Fang, Yi NYU Multimedia and Visual Computing Lab United States New York University Abu Dhabi United Arab Emirates New York University United States

In this paper, we deal with the problem to predict the future 3D motions of 3D object scans from previous two consecutive frames. Previous methods mostly focus on sparse motion prediction in the form of skeletons. While in this paper we focus on predicting dense 3D motions in the from of 3D point clouds. To approach this problem, we propose a self-supervised approach that leverages the power of the deep neural network to learn a continuous flow function of 3D point clouds that can predict temporally consistent future motions and naturally bring out the correspondences among consecutive point clouds at the same time. More specifically, in our approach, to eliminate the unsolved and challenging process of defining a discrete point convolution on 3D point cloud sequences to encode spatial and temporal information, we introduce a learnable latent code to represent the temporal-aware shape descriptor which is optimized during model training. Moreover, a temporally consistent motion Morpher is proposed to learn a continuous flow field which deforms a 3D scan from the current frame to the next frame. We perform extensive experiments on D-FAUST, SCAPE and TOSCA benchmark data sets and the results demonstrate that our approach is capable of handling temporally inconsistent input and produces consistent future 3D motion while requiring no ground truth supervision. Copyright © 2020, The Authors. All rights reserved.

关键词： Motion estimation

来源：评论

学校读者我要写书评

暂无评论

DeepTracking-Net: 3D Tracking with Unsupervised Learning of Continuous Flow

arXiv

引用

arXiv 2020年

作者： Yuan, Shuaihang Li, Xiang Fang, Yi NYU Multimedia and Visual Computing Lab United States New York University Abu Dhabi United Arab Emirates New York University United States

This paper deals with the problem of 3D tracking, i.e., to find dense correspondences in a sequence of time-varying 3D shapes. Despite deep learning approaches have achieved promising performance for pairwise dense 3D shapes matching, it is a great challenge to generalize those approaches for the tracking of 3D time-varying geometries. In this paper, we aim at handling the problem of 3D tracking, which provides the tracking of the consecutive frames of 3D shapes. We propose a novel unsupervised 3D shape registration framework named DeepTracking-Net, which uses the deep neural networks (DNNs) as auxiliary functions to produce spatially and temporally continuous displacement fields for 3D tracking of objects in a temporal order. Our key novelty is that we present a novel temporal-aware correspondence descriptor (TCD) that captures spatio-temporal essence from consecutive 3D point cloud frames. Specifically, our DeepTracking-Net starts with optimizing a randomly initialized latent TCD. The TCD is then decoded to regress a continuous flow (i.e. a displacement vector field) which assigns a motion vector to every point of time-varying 3D shapes. Our DeepTracking-Net jointly optimizes TCDs and DNNs’ weights towards the minimization of an unsupervised alignment loss. Experiments on both simulated and real data sets demonstrate that our unsupervised DeepTracking-Net outperforms the current supervised state-of-the-art method. In addition, we prepare a new synthetic 3D data, named SynMotions, to the 3D tracking and recognition community. Copyright © 2020, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：