检索结果-内蒙古大学图书馆

RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Xiao, Junjin Zhang, Qing Nie, Yonewei Zhu, Lei Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China South China University of Technology China China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

This paper presents RoGSplat, a novel approach for synthesizing high-fidelity novel views of unseen human from sparse multi-view images, while requiring no cumbersome per-subject optimization. Unlike previous methods that typically struggle with sparse views with few overlappings and are less effective in reconstructing complex human geometry, the proposed method enables robust reconstruction in such challenging conditions. Our key idea is to lift SMPL vertices to dense and reliable 3D prior points representing accurate human body geometry, and then regress human Gaussian parameters based on the points. To account for possible misalignment between SMPL model and images, we propose to predict image-aligned 3D prior points by leveraging both pixel-level features and voxel-level features, from which we regress the coarse Gaussians. To enhance the ability to capture high-frequency details, we further render depth maps from the coarse 3D Gaussians to help regress fine-grained pixel-wise Gaussians. Experiments on several benchmark datasets demonstrate that our method outperforms state-of-the-art methods in novel view synthesis and cross-dataset generalization. Our code is available at https: //***/iSEE-laboratory/RoGSplat. Copyright © 2025, The Authors. All rights reserved.

关键词： Pixels

Differentiable Gated Autoencoders for Unsupervised Feature Selection

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Chen, Zebin Bian, Jintang Qiao, Bo Xie, Xiaohua School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Unsupervised feature selection (UFS) aims to identify a subset of the most informative features from high-dimensional data without labels. However, most existing UFS methods cannot adequately capture the intricate nonlinear relationships present in high-dimensional data, limiting their ability to obtain meaningful features. In this letter, we propose a novel UFS model that leverages the autoencoder with a differentiable gating function to address this limitation. Our model optimizes a bi-level loss function to automatically select a subset of informative and stable features by capturing the nonlinear interactions between features. Furthermore, we leverage the continuous relaxation of the Bernoulli distribution to parameterize stochastic gates, thereby enabling the learning of a relaxed model through a low-variance gradient estimator. The proposed model does not necessitate prior knowledge or domain-specific tuning, making it a versatile and widely applicable method. Extensive experiments on 13 benchmark datasets demonstrate the effectiveness and superiority of the proposed method. © 2024, The Authors. All rights reserved.

关键词： Feature Selection

Diversifying Spatial-Temporal Perception for Video Domain Generalization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lin, Kun-Yu Du, Jia-Run Gao, Yipeng Zhou, Jiaming Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Video domain generalization aims to learn generalizable video classification models for unseen target domains by training in a source domain. A critical challenge of video domain generalization is to defend against the heavy reliance on domain-specific cues extracted from the source domain when recognizing target videos. To this end, we propose to perceive diverse spatial-temporal cues in videos, aiming to discover potential domain-invariant cues in addition to domain-specific cues. We contribute a novel model named Spatial-Temporal Diversification Network (STDN), which improves the diversity from both space and time dimensions of video data. First, our STDN proposes to discover various types of spatial cues within individual frames by spatial grouping. Then, our STDN proposes to explicitly model spatial-temporal dependencies between video contents at multiple space-time scales by spatial-temporal relation modeling. Extensive experiments on three benchmarks of different types demonstrate the effectiveness and versatility of our approach. © 2023, CC BY-SA.

关键词： Video recording

Progressive Pretext Task Learning for Human Trajectory Prediction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Xiaotong Liang, Tianming Lai, Jianhuang Hu, Jian-Fang School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Human trajectory prediction is a practical task of predicting the future positions of pedestrians on the road, which typically covers all temporal ranges from short-term to long-term within a trajectory. However, existing works attempt to address the entire trajectory prediction with a singular, uniform training paradigm, neglecting the distinction between short-term and long-term dynamics in human trajectories. To overcome this limitation, we introduce a novel Progressive Pretext Task learning (PPT) framework, which progressively enhances the model's capacity of capturing short-term dynamics and long-term dependencies for the final entire trajectory prediction. Specifically, we elaborately design three stages of training tasks in the PPT framework. In the first stage, the model learns to comprehend the short-term dynamics through a stepwise next-position prediction task. In the second stage, the model is further enhanced to understand long-term dependencies through a destination prediction task. In the final stage, the model aims to address the entire future trajectory task by taking full advantage of the knowledge from previous stages. To alleviate the knowledge forgetting, we further apply a cross-task knowledge distillation. Additionally, we design a Transformer-based trajectory predictor, which is able to achieve highly efficient two-step reasoning by integrating a destination-driven prediction strategy and a group of learnable prompt embeddings. Extensive experiments on popular benchmarks have demonstrated that our proposed approach achieves state-of-the-art performance with high efficiency. Code is available at https://***/iSEE-laboratory/PPT. Copyright © 2024, The Authors. All rights reserved.

关键词： Trajectories

Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding

学校读者我要写书评

暂无评论

Siamese Learning with Joint Alignment and Regression for Wea...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Chaolei Tan Jianhuang Lai Wei-Shi Zheng Jian-Fang Hu School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Video Paragraph Grounding (VPG) is an emerging task in video-language understanding, which aims at localizing multiple sentences with semantic relations and temporal or-der from an untrimmed video. However, existing VPG approaches are heavily reliant on a considerable number of temporal labels that are laborious and time-consuming to acquire. In this work, we introduce and explore Weakly-Supervised Video Paragraph Grounding (WSVPG) to elim-inate the need of temporal annotations. Different from pre-vious weakly-supervised grounding frameworks based on multiple instance learning or reconstruction learning for two-stage candidate ranking, we propose a novel siamese learning framework that jointly learns the cross-modal feature alignment and temporal coordinate regression without timestamp labels to achieve concise one-stage localization for WSVPG. Specifically, we devise a Siamese Grounding TRansformer (SiamGTR) consisting of two weight-sharing branches for learning complementary supervision. An Aug-mentation Branch is utilized for directly regressing the tem-poral boundaries of a complete paragraph within a pseudo video, and an Inference Branch is designed to capture the order-guided feature correspondence for localizing multi-ple sentences in a normal video. We demonstrate by exten-sive experiments that our paradigm has superior practica-bility and flexibility to achieve efficient weakly-supervised or semi-supervised learning, outperforming state-of-the-art methods trained with the same or stronger supervision.

关键词： Location awareness Computer vision Grounding Annotations Semantics Semisupervised learning Transformers

Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tan, Chaolei Lai, Jianhuang Zheng, Wei-Shi Hu, Jian-Fang School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Video Paragraph Grounding (VPG) is an emerging task in video-language understanding, which aims at localizing multiple sentences with semantic relations and temporal order from an untrimmed video. However, existing VPG approaches are heavily reliant on a considerable number of temporal labels that are laborious and time-consuming to acquire. In this work, we introduce and explore Weakly-Supervised Video Paragraph Grounding (WSVPG) to eliminate the need of temporal annotations. Different from previous weakly-supervised grounding frameworks based on multiple instance learning or reconstruction learning for two-stage candidate ranking, we propose a novel siamese learning framework that jointly learns the cross-modal feature alignment and temporal coordinate regression without timestamp labels to achieve concise one-stage localization for WSVPG. Specifically, we devise a Siamese Grounding TRansformer (SiamGTR) consisting of two weight-sharing branches for learning complementary supervision. An Augmentation Branch is utilized for directly regressing the temporal boundaries of a complete paragraph within a pseudo video, and an Inference Branch is designed to capture the order-guided feature correspondence for localizing multiple sentences in a normal video. We demonstrate by extensive experiments that our paradigm has superior practicability and flexibility to achieve efficient weakly-supervised or semi-supervised learning, outperforming state-of-the-art methods trained with the same or stronger supervision. © 2024, CC BY.

关键词： Semantics

Self-Supervised Rotation Learning for 3D Segmentation on Nasopharyngeal Carcinoma MRI Images

学校读者我要写书评

暂无评论

Self-Supervised Rotation Learning for 3D Segmentation on Nas...

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Li, Changtai Jiang, Ruohui Yin, Shihua Yang, Jinzhu Ban, Xiaojuan University of Science and Technology Beijing Beijing Advanced Innovation Center for Materials Genome Engineering School of Intelligence Science and Technology Beijing China University of Science and Technology Beijing School of Intelligence Science and Technology Beijing China Second Affiliated Hospital of Guangxi Medical University Nanning China Northeastern University Key Laboratory of Intelligent Computing in Medical Image Ministry of Education Shenyang China

ISBN: (纸本)9798350337488

Segmenting the tumors of nasopharyngeal carcinoma (NPC) in Magnetic Resonance Imaging (MRI) images is critical for its diagnosis and treatment. In medical image segmentation, it is vital to exploit the rich information contained in 3D sliced images. However, the precise identification and location of lesion regions often require plenty of labels. To alleviate this situation, in this article, we propose 3DRotNPC, a framework based on a tailored Self-Supervised Learning (SSL) strategy, to accurately segment tumor regions for NPC under the circumstance of label limitation. To learn the rich 3D spatial and geometric nature of MRI images in a Self-Supervised way, the proxy task of randomly selecting and rotating images in consecutively sliced data is designed. After the SSL pre-training stage, the learned parameters of the Convolutional Neural Networks (CNNs) based model are transferred to adapt the downstream segmentation task. We verify the capability of 3DRotNPC on the NPC tumor dataset which is collected and curated from the clinical treatment in the representative hospital. Extensive experiments demonstrate that our approach delivers considerable gains in downstream 3D voxel segmentation, especially with a 10% to 50% number of labels. © 2023 IEEE.

关键词： 3D tumor segmentation MRI image nasopharyngeal carcinoma Self-Supervised Learning

Class-specific Prompts in Vision Transformer for Continual Learning of New Diseases

学校读者我要写书评

暂无评论

Class-specific Prompts in Vision Transformer for Continual L...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Defeng Zhao Zejun Ye Wei-Shi Zheng Ruixuan Wang School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Key Laboratory of Machine Intelligence and Advanced Computing MOE China Department of Network Intelligence Peng Cheng Laboratory Shenzhen China

Current intelligent diagnosis systems are often trained to diagnose a small number of diseases and lack the ability of continually learning new disease knowledge. To have such continual learning ability, the deployed intelligent system needs to be continually updated based on training data of only new diseases, without accessing old data of previously learned diseases due to privacy concerns and challenges in data sharing across medical centers. In this study, a novel and effective prompt learning strategy is proposed to help a pretrained and fixed Vision Transformer (ViT) continually learn new diseases. In particular, a set of unique prompts for each disease are effectively learned such that discriminative disease features can be well extracted from the fixed ViT under the instructions of prompts during feature extraction, even though the ViT feature extractor is pretrained in the natural image domain. Extensive empirical evaluations on two medical image datasets and one natural image dataset demonstrate the superior performance of the proposed method. The source code is available at https://***/zhaoedf/CSPrompt.

关键词：

Automatic Detection and Characterization of Human Veins Using Infra-Red Image Processing

学校读者我要写书评

暂无评论

Journal of Computer and Communications 2024年第9期12卷 141-159页

作者： Jean Ndoumbe Brice Ekobo Akoa Gaelle Patricia Talotsing Frederic Franck Kounga Samuel Kaissassou Bertin Chouanmo Njo Laboratory of Computer Engineering Data Science and Artificial Intelligence Department of Computer & Telecommunications Engineering National Higher Polytechnic School Douala Cameroon Laboratory of Electrical Engineering Mechatronics and Signal Processing Department of Electrical & Telecommunications Engineering National Advanced School of Engineering Yaound Cameroon Mobile Computing and Networking Research Laboratory Department of Computer and Software Engineering Polytechnique Montreal Montreal Canada DigiPlus SARL Bonamoussadi Douala Cameroon

The detection and characterization of human veins using infrared (IR) image processing have gained significant attention due to its potential applications in biometric identification, medical diagnostics, and vein-based authentication systems. This paper presents a low-cost approach for automatic detection and characterization of human veins from IR images. The proposed method uses image processing techniques including segmentation, feature extraction, and, pattern recognition algorithms. Initially, the IR images are preprocessed to enhance vein structures and reduce noise. Subsequently, a CLAHE algorithm is employed to extract vein regions based on their unique IR absorption properties. Features such as vein thickness, orientation, and branching patterns are extracted using mathematical morphology and directional filters. Finally, a classification framework is implemented to categorize veins and distinguish them from surrounding tissues or artifacts. A setup based on Raspberry Pi was used. Experimental results of IR images demonstrate the effectiveness and robustness of the proposed approach in accurately detecting and characterizing human. The developed system shows promising for integration into applications requiring reliable and secure identification based on vein patterns. Our work provides an effective and low-cost solution for nursing staff in low and middle-income countries to perform a safe and accurate venipuncture.

关键词： Vein Detection Blood Radiation Infrared Image CLAHE Algorithm Raspberry Pi