检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Luo, Yan Shi, Min Khan, Muhammad Osama Afzal, Muhammad Muneeb Huang, Hao Yuan, Shuaihang Tian, Yu Song, Luo Kouhana, Ava Elze, Tobias Fang, Yi Wang, Mengyu Harvard Ophthalmology AI Lab Harvard University United States Tandon School of Engineering New York University United States Multimedia and Visual Computing Lab New York University Abu Dhabi United States

Fairness is a critical concern in deep learning, especially in healthcare, where these models influence diagnoses and treatment decisions. Although fairness has been investigated in the vision-only domain, the fairness of medical vision-language (VL) models remains unexplored due to the scarcity of medical VL datasets for studying fairness. To bridge this research gap, we introduce the first fair vision-language medical dataset (Harvard-FairVLMed) that provides detailed demographic attributes, ground-truth labels, and clinical notes to facilitate an in-depth examination of fairness within VL foundation models. Using Harvard-FairVLMed, we conduct a comprehensive fairness analysis of two widely-used VL models (CLIP and BLIP2), pre-trained on both natural and medical domains, across four different protected attributes. Our results highlight significant biases in all VL models, with Asian, Male, Non-Hispanic, and Spanish being the preferred subgroups across the protected attributes of race, gender, ethnicity, and language, respectively. In order to alleviate these biases, we propose FairCLIP, an optimal-transport-based approach that achieves a favorable trade-off between performance and fairness by reducing the Sinkhorn distance between the overall sample distribution and the distributions corresponding to each demographic group. As the first VL dataset of its kind, Harvard-FairVLMed holds the potential to catalyze advancements in the development of machine learning models that are both ethically aware and clinically effective. Our dataset and code are available at https://***/datasets/ harvard-fairvlmed10k. © 2024, CC BY-NC-ND.

关键词： Population statistics

来源：评论

学校读者我要写书评

暂无评论

Self-supervised learning of depth and ego-motion with differentiable bundle adjustment

arXiv

引用

arXiv 2019年

作者： Shi, Yunxiao Zhu, Jing Fang, Yi Lien, Kuochin Gu, Junli NYU Multimedia Visual Computing Lab United States New York University United States New York University Abu Dhabi United Arab Emirates XMotors.ai United States

Learning to predict scene depth and camera motion from RGB inputs only is a challenging task. Most existing learning based methods deal with this task in a supervised manner which require ground-truth data that is expensive to acquire. More recent approaches explore the possibility of estimating scene depth and camera pose in a self-supervised learning framework. Despite encouraging results are shown, current methods either learn from monocular videos for depth and pose and typically do so without enforcing multi-view geometry constraints between scene structure and camera motion, or require stereo sequences as input where the ground-truth between-frame motion parameters need to be known. In this paper we propose to jointly optimize the scene depth and camera motion via incorporating differentiable Bundle Adjustment (BA) layer by minimizing the feature-metric error, and then form the photometric consistency loss with view synthesis as the final supervisory signal. The proposed approach only needs unlabeled monocular videos as input, and extensive experiments on the KITTI and Cityscapes dataset show that our method achieves state-of-the-art results in self-supervised approaches using monocular videos as input, and even gains advantage to the line of methods that learns from calibrated stereo sequences (i.e. with pose supervision). Copyright © 2019, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

3D Unsupervised Region-Aware Registration Transformer

3D Unsupervised Region-Aware Registration Transformer

引用

IEEE International Conference on Image Processing

作者： Yu Hao Yi Fang NYU Multimedia and Visual Computing Lab New York University Abu Dhabi Abu Dhabi UAE NYUAD Center for Artificial Intelligence and Robotics New York University Abu Dhabi Abu Dhabi UAE

This paper concerns the research problem of point cloud registration to find the rigid transformation to optimally align the source point set with the target one. Learning robust point cloud registration models with deep neural networks has emerged as a powerful paradigm, offering promising performance in predicting the global geometric transformation for a pair of point sets. Existing methods first leverage an encoder to regress the global shape descriptor, which is then decoded into a shape-conditioned transformation via concatenation-based conditioning. However, different regions of a 3D shape vary in their geometric structures which makes it more sense that we have a region-conditioned transformation instead of the shape-conditioned one. In this paper, we define our 3D registration function through the introduction of a new design of 3D region partition module that is able to divide the input shape to different regions with a self-supervised 3D shape reconstruction loss without the need for ground truth labels. We further propose the 3D shape transformer module to efficiently and effectively capture short-and long-range geometric dependencies for regions on the 3D shape Consequently, the region-aware decoder module is proposed to predict the transformations for different regions respectively. The global geometric transformation from the source point set to the target one is then formed by the weighted fusion of region-aware transformation. Compared to the state-of-the-art approaches, our experiments show that our 3D-URRT achieves superior registration performance over various benchmark datasets (e.g. ModelNet40).

关键词：

来源：评论

学校读者我要写书评

暂无评论

Detect and Approach: Close-Range Navigation Support for People with Blindness and Low Vision

arXiv

引用

arXiv 2022年

作者： Hao, Yu Feng, Junchi Rizzo, John-Ross Wang, Yao Fang, Yi Nyu Multimedia and Visual Computing Lab Nyu Tandon School of Engineering New York University United States New York University Abu Dhabi United Arab Emirates Nyu Langone Health United States

People with blindness and low vision (pBLV) experience significant challenges when locating final destinations or targeting specific objects in unfamiliar environments. Furthermore, besides initially locating and orienting oneself to a target object, approaching the final target from one's present position is often frustrating and challenging, especially when one drifts away from the initial planned path to avoid obstacles. In this paper, we develop a novel wearable navigation solution to provide real-time guidance for a user to approach a target object of interest efficiently and effectively in unfamiliar environments. Our system contains two key visual computing functions: initial target object localization in 3D and continuous estimation of the user's trajectory, both based on the 2D video captured by a low-cost monocular camera mounted on in front of the chest of the user. These functions enable the system to suggest an initial navigation path, continuously update the path as the user moves, and offer timely recommendation about the correction of the user's path. Our experiments demonstrate that our system is able to operate with an error of less than 0.5 meter both outdoor and indoor. The system is entirely vision-based and does not need other sensors for navigation, and the computation can be run with the Jetson processor in the wearable system to facilitate real-time navigation assistance. © 2022, CC BY.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

Density-aware convolutional networks with context encoding for airborne lidar point cloud classification

arXiv

引用

arXiv 2019年

作者： Li, Xiang Wang, Mingyang Wen, Congcong Wang, Lingjing Zhou, Nan Fang, Yi Multimedia and Visual Computing Lab New York University New York United States Tandon School of Engineering New York University New York United States Aerospace Information Research Institute Chinese Academy of Sciences

To better address challenging issues of the irregularity and inhomogeneity inherently present in 3D point clouds, researchers have shifted their focus from designing hand-crafted point features toward learning 3D point signatures using deep neural networks for 3D point cloud classification. Recently proposed deep learning-based point cloud classification methods either apply 2D CNNs on projected feature images or apply 1D convolutional layers directly on raw point sets. These methods cannot adequately recognize fine-grained local structures caused by the uneven density distribution of the point cloud data. In this paper, to address this challenging issue, we introduced a density-aware convolutional module that uses the pointwise density to reweight the learnable weights of convolution kernels. The proposed convolution module can fully approximate the 3D continuous convolution on unevenly distributed 3D point sets. Based on this convolution module, we further developed a multiscale fully convolutional neural network with downsampling and upsampling blocks to enable hierarchical point feature learning. In addition, to regularize the global semantic context, we implemented a context encoding module to predict a global context encoding and formulated a context encoding regularizer to enforce the predicted context encoding to align with the ground truth encoding. The overall network can be trained in an end-to-end fashion with the raw 3D coordinates and the height above ground as inputs. Experiments on the International Society for Photogrammetry and Remote Sensing (ISPRS) 3D labeling benchmark demonstrated the superiority of the proposed method for point cloud classification. Our model achieved a new state-of-the-art performance with an average F1 score of 71.2% and improved the performance by a large margin on several categories (such as powerline, impervious surface, car, and facade). Copyright © 2019, The Authors. All rights reserved.

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

3D-A-nets: 3D deep dense descriptor for volumetric shapes with adversarial networks

arXiv

引用

arXiv 2017年

作者： Ren, Mengwei Niu, Liang Fang, Yi NYU Multimedia and Visual Computing Lab NYU Abu Dhabi United Arab Emirates New York University BrooklynNY United States 5 Metrotech Center LC024 BrooklynNY11201 United States

Recently researchers have been shifting their focus towards learned 3D shape descriptors from hand-craft ones to better address challenging issues of the deformation and structural variation inherently present in 3D objects. 3D geometric data are often transformed to 3D Voxel grids with regular format in order to be better fed to a deep neural net architecture. However, the computational intractability of direct application of 3D convolutional nets to 3D volumetric data severely limits the efficiency (i.e. slow processing) and effectiveness (i.e. unsatisfied accuracy) in processing 3D geometric data. In this paper, powered with a novel design of adversarial networks (3D-A-Nets), we have developed a novel 3D deep dense shape descriptor (3D-DDSD) to address the challenging issues of efficient and effective 3D volumetric data processing. We developed new definition of 2D multilayer dense representation (MDR) of 3D volumetric data to extract concise but geometrically informative shape description and a novel design of adversarial networks that jointly train a set of convolution neural network (CNN), recurrent neural network (RNN) and an adversarial discriminator. More specifically, the generator network produces 3D shape features that encourages the clustering of samples from the same category with correct class label, whereas the discriminator network discourages the clustering by assigning them misleading adversarial class labels. By addressing the challenges posed by the computational inefficiency of direct application of CNN to 3D volumetric data, 3D-A-Nets can learn high-quality 3DDSDD which demonstrates superior performance on 3D shape classification and retrieval over other state-of-the-art techniques by a great margin. Copyright © 2017, The Authors. All rights reserved.

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

Unsupervised partial point set registration via joint shape completion and registration

arXiv

引用

arXiv 2020年

作者： Li, Xiang Wang, Lingjing Fang, Yi NYU Multimedia and Visual Computing Lab NYU Tandon Abu Dhabi United Arab Emirates Department of Electrical and Computer Engineering NYU Tandon United States Department of Electrical and Computer Engineering NYU Abu Dhabi United Arab Emirates

We propose a self-supervised method for partial point set registration. While recent proposed learning-based methods have achieved impressive registration performance on the full shape observations, these methods mostly suffer from performance degradation when dealing with partial shapes. To bridge the performance gaps between partial point set registration with full point set registration, we proposed to incorporate a shape completion network to benefit the registration process. To achieve this, we design a latent code for each pair of shapes, which can be regarded as a geometric encoding of the target shape. By doing so, our model does need an explicit feature embedding network to learn the feature encodings. More importantly, both our shape completion network and the point set registration network take the shared latent codes as input, which are optimized along with the parameters of two decoder networks in the training process. Therefore, the point set registration process can thus benefit from the joint optimization process of latent codes, which are enforced to represent the information of full shape instead of partial ones. In the inference stage, we fix the network parameter and optimize the latent codes to get the optimal shape completion and registration results. Our proposed method is pure unsupervised and does not need any ground truth supervision. Experiments on the ModelNet40 dataset demonstrate the effectiveness of our model for partial point set registration. Copyright © 2020, The Authors. All rights reserved.

关键词： Encoding (symbols)

来源：评论

学校读者我要写书评

暂无评论

Understanding the Impact of Image Quality and Distance of Objects to Object Detection Performance

Understanding the Impact of Image Quality and Distance of Ob...

引用

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Yu Hao Haoyang Pei Yixuan Lyu Zhongzheng Yuan John-Ross Rizzo Yao Wang Yi Fang NYU Multimedia and Visual Computing Lab NYU Tandon and NYU Abu Dhabi NYU Tandon School of Engineering New York University USA NYU Langone Health USA NYUAD Center for Artificial Intelligence and Robotics NYU Abu Dhabi UAE

Object detection is a fundamental task for autonomous driving, which aim to identify and localize objects within an image. Deep learning has made great strides for object detection, with popular models including Faster R-CNN, YOLO, and SSD. The detection accuracy and computational cost of object detection depend on the spatial resolution of an image, which may be constrained by both the camera and storage considerations. Furthermore, original images are often compressed and uploaded to a remote server for object detection. Compression is often achieved by reducing either spatial or amplitude resolution or, at times, both, both of which have well-known effects on performance. Detection accuracy also depends on the distance of the object of interest from the camera. Our work examines the impact of spatial and amplitude resolution, as well as object distance, on object detection accuracy and computational cost. As existing models are optimized for uncompressed (or lightly compressed) images over a narrow range of spatial resolution, we develop a resolution-adaptive variant of YOLOv5 (RA-YOLO), which varies the number of scales in the feature pyramid and detection head based on the spatial resolution of the input image. To train and evaluate this new method, we created a dataset of images with diverse spatial and amplitude resolutions by combining images from the TJU and Eurocity datasets and generating different resolutions by applying spatial resizing and compression. We first show that RA-YOLO achieves a good trade-off between detection accuracy and inference time over a large range of spatial resolutions. We then evaluate the impact of spatial and amplitude resolutions on object detection accuracy using the proposed RA-YOLO model. We demonstrate that the optimal spatial resolution that leads to the highest detection accuracy depends on the ‘tolerated’ image size (constrained by the available bandwidth or storage). We further assess the impact of the distance of an o

关键词：

来源：评论

学校读者我要写书评

暂无评论

ArbiText: Arbitrary-oriented text detection in unconstrained scene

arXiv

引用

arXiv 2017年

作者： Xing, Daitao Li, Zichen Chen, Xin Fang, Yi NYU Multimedia and Visual Computing Lab NYU Abu Dhabi United Arab Emirates New York University BrooklynNY United States HERE Technologies 5 Metrotech Center LC024 BrooklynNY11201 United States

Arbitrary-oriented text detection in the wild is a very challenging task, due to the aspect ratio, scale, orientation, and illumination variations. In this paper, we propose a novel method, namely Arbitrary-oriented Text (or ArbText for short) detector, for efficient text detection in unconstrained natural scene images. Specifically, we first adopt the circle anchors rather than the rectangular ones to represent bounding boxes, which is more robust to orientation variations. Subsequently, we incorporate a pyramid pooling module into the Single Shot MultiBox Detector framework, in order to simultaneously explore the local and global visual information, which can therefore generate more confidential detection results. Experiments on established scene-text datasets, such as the ICDAR 2015 and MSRA-TD500 datasets, have demonstrated the superior performance of the proposed method, compared to the state-of-the-art approaches. Copyright © 2017, The Authors. All rights reserved.

关键词： Aspect ratio

来源：评论

学校读者我要写书评

暂无评论

Pyramid Learnable Tokens for 3D LiDAR Place Recognition

Pyramid Learnable Tokens for 3D LiDAR Place Recognition

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： Congcong Wen Hao Huang Yu-Shen Liu Yi Fang NYUAD Center for Artificial Intelligence and Robotics New York University Abu Dhabi UAE NYU Multimedia and Visual Computing Lab New York University USA New York University Abu Dhabi UAE School of Software Tsinghua University Beijing P. R. China

3D LiDAR place recognition plays a vital role in various robot applications' including robotic navigation, autonomous driving, and simultaneous localization and mapping. However, most previous studies evaluated their models on accumulated 2D scans instead of real-world 3D LiDAR scans with a larger number of points, which limits the application in real scenarios. To address this limitation, we propose a point transformer network with pyramid learnable tokens (PTNet-PLT) to learn global descriptors for an actual scanned 3D LiDAR place recognition. Specifically, we first present a novel shifted cube attention module that consists of a self-attention module for local feature extraction and a cross-attention module for regional feature aggregation. The self-attention module constrains attention computation on a locally partitioned cube and builds connections across cubes based on the shifted cube scheme. In addition, the cross-attention module introduces several learnable tokens to separately aggregate features of points with similar features but spatially distant into an arbitrarily shaped region, which enables the model to capture long-term dependencies of the points. Next, we build a pyramid architecture network to learn multi-scale features and involve a decreasing number of tokens at each layer to aggregate features over a larger region. Finally, we obtain the global descriptor by concatenating learned region tokens of all layers. Experiments on three datasets, including USyd Campus, Oxford Robot-Car, and KITTI, demonstrate the effectiveness and generalization of the proposed model for large-scale 3D LiDAR place recognition.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：