检索结果-内蒙古大学图书馆

Learning superpoint graph cut for 3D instance segmentation 22

学校读者我要写书评

暂无评论

Learning superpoint graph cut for 3D instance segmentation

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Le Hui Linghua Tang Yaqi Shen Jin Xie Jian Yang PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education and Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China

ISBN: (纸本)9781713871088

3D instance segmentation is a challenging task due to the complex local geometric structures of objects in point clouds. In this paper, we propose a learning-based superpoint graph cut method that explicitly learns the local geometric structures of the point cloud for 3D instance segmentation. Specifically, we first oversegment the raw point clouds into superpoints and construct the superpoint graph. Then, we propose an edge score prediction network to predict the edge scores of the superpoint graph, where the similarity vectors of two adjacent nodes learned through cross-graph attention in the coordinate and feature spaces are used for regressing edge scores. By forcing two adjacent nodes of the same instance to be close to the instance center in the coordinate and feature spaces, we formulate a geometry-aware edge loss to train the edge score prediction network. Finally, we develop a superpoint graph cut network that employs the learned edge scores and the predicted semantic classes of nodes to generate instances, where bilateral graph attention is proposed to extract discriminative features on both the coordinate and feature spaces for predicting semantic labels and scores of instances. Extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, show that our method achieves new state-of-the-art performance on 3D instance segmentation.

关键词：

Customized Generation Reimagined: Fidelity and Editability Harmonized

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jin, Jian Shen, Yang Fu, Zhenyong Yang, Jian PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China

Customized generation aims to incorporate a novel concept into a pre-trained text-to-image model, enabling new generations of the concept in novel contexts guided by textual prompts. However, customized generation suffers from an inherent trade-off between concept fidelity and editability, i.e., between precisely modeling the concept and faithfully adhering to the prompts. Previous methods reluctantly seek a compromise and struggle to achieve both high concept fidelity and ideal prompt alignment simultaneously. In this paper, we propose a "Divide, Conquer, then Integrate" (DCI) framework, which performs a surgical adjustment in the early stage of denoising to liberate the fine-tuned model from the fidelity-editability trade-off at inference. The two conflicting components in the trade-off are decoupled and individually conquered by two collaborative branches, which are then selectively integrated to preserve high concept fidelity while achieving faithful prompt adherence. To obtain a better fine-tuned model, we introduce an image-specific Context Optimization (ICO) strategy for model customization. ICO replaces manual prompt templates with learnable image-specific contexts, providing an adaptive and precise fine-tuning direction to promote the overall performance. Extensive experiments demonstrate the effectiveness of our method in reconciling the fidelity-editability trade-off. Code is available at https://***/jinjianRick/DCI_ICO. Copyright © 2024, The Authors. All rights reserved.

关键词： Economic and social effects

3D Siamese Transformer Network for Single Object Tracking on Point Clouds

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hui, Le Wang, Lingpeng Tang, Linghua Lan, Kaihao Xie, Jin Yang, Jian Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security PCA Lab School of Computer Science and Engineering Nanjing University of Science and Technology China

Siamese network based trackers formulate 3D single object tracking as cross-correlation learning between point features of a template and a search area. Due to the large appearance variation between the template and search area during tracking, how to learn the robust cross correlation between them for identifying the potential target in the search area is still a challenging problem. In this paper, we explicitly use Transformer to form a 3D Siamese Transformer network for learning robust cross correlation between the template and the search area of point clouds. Specifically, we develop a Siamese point Transformer network to learn shape context information of the target. Its encoder uses self-attention to capture non-local information of point clouds to characterize the shape information of the object, and the decoder utilizes cross-attention to upsample discriminative point features. After that, we develop an iterative coarse-to-fine correlation network to learn the robust cross correlation between the template and the search area. It formulates the cross-feature augmentation to associate the template with the potential target in the search area via cross attention. To further enhance the potential target, it employs the ego-feature augmentation that applies self-attention to the local k-NN graph of the feature space to aggregate target features. Experiments on the KITTI, nuScenes, and Waymo datasets show that our method achieves state-of-the-art performance on the 3D single object tracking task. Source code is available at https://***/fpthink/STNet. Copyright © 2022, The Authors. All rights reserved.

关键词： Iterative methods

Generative Subgraph Contrast for Self-Supervised Graph Representation Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Yuehui Hui, Le Jiang, Haobo Qian, Jianjun Xie, Jin Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security PCA Lab School of Computer Science and Engineering Nanjing University of Science and Technology China

Contrastive learning has shown great promise in the field of graph representation learning. By manually constructing positive/negative samples, most graph contrastive learning methods rely on the vector inner product based similarity metric to distinguish the samples for graph representation. However, the handcrafted sample construction (e.g., the perturbation on the nodes or edges of the graph) may not effectively capture the intrinsic local structures of the graph. Also, the vector inner product based similarity metric cannot fully exploit the local structures of the graph to characterize the graph difference well. To this end, in this paper, we propose a novel adaptive subgraph generation based contrastive learning framework for efficient and robust self-supervised graph representation learning, and the optimal transport distance is utilized as the similarity metric between the subgraphs. It aims to generate contrastive samples by capturing the intrinsic structures of the graph and distinguish the samples based on the features and structures of subgraphs simultaneously. Specifically, for each center node, by adaptively learning relation weights to the nodes of the corresponding neighborhood, we first develop a network to generate the interpolated subgraph. We then construct the positive and negative pairs of subgraphs from the same and different nodes, respectively. Finally, we employ two types of optimal transport distances (i.e., Wasserstein distance and Gromov-Wasserstein distance) to construct the structured contrastive loss. Extensive node classification experiments on benchmark datasets verify the effectiveness of our graph contrastive learning method. Source code is available at https://***/yh-han/***. Copyright © 2022, The Authors. All rights reserved.

关键词： Classification (of information)

3D siamese voxel-to-BEV tracker for sparse point clouds

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hui, Le Wang, Lingpeng Cheng, Mingmei Xie, Jin Yang, Jian PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China

3D object tracking in point clouds is still a challenging problem due to the sparsity of LiDAR points in dynamic environments. In this work, we propose a Siamese voxel-to-BEV tracker, which can significantly improve the tracking performance in sparse 3D point clouds. Specifically, it consists of a Siamese shape-aware feature learning network and a voxel-to-BEV target localization network. The Siamese shape-aware feature learning network can capture 3D shape information of the object to learn the discriminative features of the object so that the potential target from the background in sparse point clouds can be identified. To this end, we first perform template feature embedding to embed the template’s feature into the potential target and then generate a dense 3D shape to characterize the shape information of the potential target. For localizing the tracked target, the voxel-to-BEV target localization network regresses the target’s 2D center and the z-axis center from the dense bird’s eye view (BEV) feature map in an anchor-free manner. Concretely, we compress the voxelized point cloud along z-axis through max pooling to obtain a dense BEV feature map, where the regression of the 2D center and the z-axis center can be performed more effectively. Extensive evaluation on the KITTI and nuScenes datasets shows that our method significantly outperforms the current state-of-the-art methods by a large margin. Code is available at https://***/fpthink/V2B. Copyright © 2021, The Authors. All rights reserved.

关键词： Large dataset

Planning with learned dynamic model for unsupervised point cloud registration

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jiang, Haobo Qian, Jianjun Xie, Jin Yang, Jian PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information Ministry of Education Jiangsu Key Lab of Image Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China

Point cloud registration is a fundamental problem in 3D computer vision. In this paper, we cast point cloud registration into a planning problem in reinforcement learning, which can seek the transformation between the source and target point clouds through trial and error. By modeling the point cloud registration process as a Markov decision process (MDP), we develop a latent dynamic model of point clouds, consisting of a transformation network and evaluation network. The transformation network aims to predict the new transformed feature of the point cloud after performing a rigid transformation (i.e., action) on it while the evaluation network aims to predict the alignment precision between the transformed source point cloud and target point cloud as the reward signal. Once the dynamic model of the point cloud is trained, we employ the cross-entropy method (CEM) to iteratively update the planning policy by maximizing the rewards in the point cloud registration process. Thus, the optimal policy, i.e., the transformation between the source and target point clouds, can be obtained via gradually narrowing the search space of the transformation. Experimental results on ModelNet40 and 7Scene benchmark datasets demonstrate that our method can yield good registration performance in an unsupervised manner. Copyright © 2021, The Authors. All rights reserved.

关键词： Dynamic models

Road detection through CRF based LiDAR-camera fusion

学校读者我要写书评

暂无评论

Road detection through CRF based LiDAR-camera fusion

2019 International Conference on Robotics and Automation, ICRA 2019

作者： Gu, Shuo Zhang, Yigong Tang, Jinhui Yang, Jian Kong, Hui PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing 210094 China School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing 210094 China Institute of Advanced Artifical Intelligence in Nanjing Horizon Robotics China

ISBN: (纸本)9781538681763

In this paper, we propose a road detection method with LiDAR-camera fusion in a novel conditional random field (CRF) framework to exploit both range and color information. In the LiDAR based part, a fast height-difference based scanning strategy is applied in the 2D LiDAR range-image domain and a dense road detection result in camera image domain can be obtained through geometric upsampling given the LiDAR-camera calibration parameters. In the camera based part, a fully convolutional network is applied in the camera image domain. Finally, we fuse the dense and binary road detection results from both LiDAR and camera in a single CRF framework. Experiments show that using a single thread of CPU, the proposed LiDAR based part can operate at a frequency of over 250Hz with sparse output in range image and 40Hz with dense result in camera image for the 64-beam Velodyne scanner. Our CRF fusion method achieves very promising road detection performance on the KITTI-Road dataset. © 2019 IEEE.

关键词： Laser radar Roads Cameras Three-dimensional displays Two dimensional displays image segmentation Transforms

Sequential 3D Human Pose and Shape Estimation From Point Clouds

学校读者我要写书评

暂无评论

Sequential 3D Human Pose and Shape Estimation From Point Clo...

Conference on computer vision and Pattern Recognition (CVPR)

作者： Kangkan Wang Jin Xie Guofeng Zhang Lei Liu Jian Yang Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China State Key Laboratory of CAD&CG Zhejiang University China ZJU-SenseTime Joint Lab of 3D Vision

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

This work addresses the problem of 3D human pose and shape estimation from a sequence of point clouds. Existing sequential 3D human shape estimation methods mainly focus on the template model fitting from a sequence of depth images or the parametric model regression from a sequence of RGB images. In this paper, we propose a novel sequential 3D human pose and shape estimation framework from a sequence of point clouds. Specifically, the proposed framework can regress 3D coordinates of mesh vertices at different resolutions from the latent features of point clouds. Based on the estimated 3D coordinates and features at the low resolution, we develop a spatial-temporal mesh attention convolution (MAC) to predict the 3D coordinates of mesh vertices at the high resolution. By assigning specific attentional weights to different neighboring points in the spatial and temporal domains, our spatial-temporal MAC can capture structured spatial and temporal features of point clouds. We further generalize our framework to the real data of human bodies with a weakly supervised fine-tuning method. The experimental results on SURREAL, Human3.6M, DFAUST and the real detailed data demonstrate that the proposed approach can accurately recover the 3D body model sequence from a sequence of point clouds.

关键词： Three-dimensional displays Solid modeling Shape Biological system modeling Estimation Spatial resolution Feature extraction

Multi-Scale Dynamic Feature Encoding Network for image Demoiréing

学校读者我要写书评

暂无评论

Multi-Scale Dynamic Feature Encoding Network for Image Demoi...

International Conference on computer vision Workshops (ICCV Workshops)

作者： Xi Cheng Zhenyong Fu Jian Yang PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education and Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing China

ISBN: (数字)9781728150239

ISBN: (纸本)9781728150246

The prevalence of digital sensors, such as digital cameras and mobile phones, simplifies the acquisition of photos. Digital sensors, however, suffer from producing Moire when photographing objects having complex textures, which deteriorates the quality of photos. Moire spreads across various frequency bands of images and is a dynamic texture with varying colors and shapes, which pose two main challenges in demoireing - an important task in image restoration. In this paper, towards addressing the first challenge, we design a multi-scale network to process images at different spatial resolutions, obtaining features in different frequency bands, and thus our method can jointly remove moire in different frequency bands. Towards solving the second challenge, we propose a dynamic feature encoding module (DFE), embedded in each scale, for dynamic texture. Moire pattern can be eliminated more effectively via *** proposed method, termed Multi-scale convolutional network with Dynamic feature encoding for image DeMoireing (MDDM), can outperform the state of the arts in fidelity as well as perceptual on benchmarks.

关键词： Spatial resolution Task analysis Encoding image restoration image coding Visualization