检索结果-内蒙古大学图书馆

Progressive Pretext Task Learning for Human Trajectory Prediction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Xiaotong Liang, Tianming Lai, Jianhuang Hu, Jian-Fang School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Human trajectory prediction is a practical task of predicting the future positions of pedestrians on the road, which typically covers all temporal ranges from short-term to long-term within a trajectory. However, existing works attempt to address the entire trajectory prediction with a singular, uniform training paradigm, neglecting the distinction between short-term and long-term dynamics in human trajectories. To overcome this limitation, we introduce a novel Progressive Pretext Task learning (PPT) framework, which progressively enhances the model's capacity of capturing short-term dynamics and long-term dependencies for the final entire trajectory prediction. Specifically, we elaborately design three stages of training tasks in the PPT framework. In the first stage, the model learns to comprehend the short-term dynamics through a stepwise next-position prediction task. In the second stage, the model is further enhanced to understand long-term dependencies through a destination prediction task. In the final stage, the model aims to address the entire future trajectory task by taking full advantage of the knowledge from previous stages. To alleviate the knowledge forgetting, we further apply a cross-task knowledge distillation. Additionally, we design a Transformer-based trajectory predictor, which is able to achieve highly efficient two-step reasoning by integrating a destination-driven prediction strategy and a group of learnable prompt embeddings. Extensive experiments on popular benchmarks have demonstrated that our proposed approach achieves state-of-the-art performance with high efficiency. Code is available at https://***/iSEE-laboratory/PPT. Copyright © 2024, The Authors. All rights reserved.

关键词： Trajectories

Diversifying Spatial-Temporal Perception for Video Domain Generalization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lin, Kun-Yu Du, Jia-Run Gao, Yipeng Zhou, Jiaming Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Video domain generalization aims to learn generalizable video classification models for unseen target domains by training in a source domain. A critical challenge of video domain generalization is to defend against the heavy reliance on domain-specific cues extracted from the source domain when recognizing target videos. To this end, we propose to perceive diverse spatial-temporal cues in videos, aiming to discover potential domain-invariant cues in addition to domain-specific cues. We contribute a novel model named Spatial-Temporal Diversification Network (STDN), which improves the diversity from both space and time dimensions of video data. First, our STDN proposes to discover various types of spatial cues within individual frames by spatial grouping. Then, our STDN proposes to explicitly model spatial-temporal dependencies between video contents at multiple space-time scales by spatial-temporal relation modeling. Extensive experiments on three benchmarks of different types demonstrate the effectiveness and versatility of our approach. © 2023, CC BY-SA.

关键词： Video recording

Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding

学校读者我要写书评

暂无评论

Siamese Learning with Joint Alignment and Regression for Wea...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Chaolei Tan Jianhuang Lai Wei-Shi Zheng Jian-Fang Hu School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Video Paragraph Grounding (VPG) is an emerging task in video-language understanding, which aims at localizing multiple sentences with semantic relations and temporal or-der from an untrimmed video. However, existing VPG approaches are heavily reliant on a considerable number of temporal labels that are laborious and time-consuming to acquire. In this work, we introduce and explore Weakly-Supervised Video Paragraph Grounding (WSVPG) to elim-inate the need of temporal annotations. Different from pre-vious weakly-supervised grounding frameworks based on multiple instance learning or reconstruction learning for two-stage candidate ranking, we propose a novel siamese learning framework that jointly learns the cross-modal feature alignment and temporal coordinate regression without timestamp labels to achieve concise one-stage localization for WSVPG. Specifically, we devise a Siamese Grounding TRansformer (SiamGTR) consisting of two weight-sharing branches for learning complementary supervision. An Aug-mentation Branch is utilized for directly regressing the tem-poral boundaries of a complete paragraph within a pseudo video, and an Inference Branch is designed to capture the order-guided feature correspondence for localizing multi-ple sentences in a normal video. We demonstrate by exten-sive experiments that our paradigm has superior practica-bility and flexibility to achieve efficient weakly-supervised or semi-supervised learning, outperforming state-of-the-art methods trained with the same or stronger supervision.

关键词： Location awareness Computer vision Grounding Annotations Semantics Semisupervised learning Transformers

Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tan, Chaolei Lai, Jianhuang Zheng, Wei-Shi Hu, Jian-Fang School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Video Paragraph Grounding (VPG) is an emerging task in video-language understanding, which aims at localizing multiple sentences with semantic relations and temporal order from an untrimmed video. However, existing VPG approaches are heavily reliant on a considerable number of temporal labels that are laborious and time-consuming to acquire. In this work, we introduce and explore Weakly-Supervised Video Paragraph Grounding (WSVPG) to eliminate the need of temporal annotations. Different from previous weakly-supervised grounding frameworks based on multiple instance learning or reconstruction learning for two-stage candidate ranking, we propose a novel siamese learning framework that jointly learns the cross-modal feature alignment and temporal coordinate regression without timestamp labels to achieve concise one-stage localization for WSVPG. Specifically, we devise a Siamese Grounding TRansformer (SiamGTR) consisting of two weight-sharing branches for learning complementary supervision. An Augmentation Branch is utilized for directly regressing the temporal boundaries of a complete paragraph within a pseudo video, and an Inference Branch is designed to capture the order-guided feature correspondence for localizing multiple sentences in a normal video. We demonstrate by extensive experiments that our paradigm has superior practicability and flexibility to achieve efficient weakly-supervised or semi-supervised learning, outperforming state-of-the-art methods trained with the same or stronger supervision. © 2024, CC BY.

关键词： Semantics

Self-Supervised Rotation Learning for 3D Segmentation on Nasopharyngeal Carcinoma MRI Images

学校读者我要写书评

暂无评论

Self-Supervised Rotation Learning for 3D Segmentation on Nas...

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Li, Changtai Jiang, Ruohui Yin, Shihua Yang, Jinzhu Ban, Xiaojuan University of Science and Technology Beijing Beijing Advanced Innovation Center for Materials Genome Engineering School of Intelligence Science and Technology Beijing China University of Science and Technology Beijing School of Intelligence Science and Technology Beijing China Second Affiliated Hospital of Guangxi Medical University Nanning China Northeastern University Key Laboratory of Intelligent Computing in Medical Image Ministry of Education Shenyang China

ISBN: (纸本)9798350337488

Segmenting the tumors of nasopharyngeal carcinoma (NPC) in Magnetic Resonance Imaging (MRI) images is critical for its diagnosis and treatment. In medical image segmentation, it is vital to exploit the rich information contained in 3D sliced images. However, the precise identification and location of lesion regions often require plenty of labels. To alleviate this situation, in this article, we propose 3DRotNPC, a framework based on a tailored Self-Supervised Learning (SSL) strategy, to accurately segment tumor regions for NPC under the circumstance of label limitation. To learn the rich 3D spatial and geometric nature of MRI images in a Self-Supervised way, the proxy task of randomly selecting and rotating images in consecutively sliced data is designed. After the SSL pre-training stage, the learned parameters of the Convolutional Neural Networks (CNNs) based model are transferred to adapt the downstream segmentation task. We verify the capability of 3DRotNPC on the NPC tumor dataset which is collected and curated from the clinical treatment in the representative hospital. Extensive experiments demonstrate that our approach delivers considerable gains in downstream 3D voxel segmentation, especially with a 10% to 50% number of labels. © 2023 IEEE.

关键词： 3D tumor segmentation MRI image nasopharyngeal carcinoma Self-Supervised Learning

Class-specific Prompts in Vision Transformer for Continual Learning of New Diseases

学校读者我要写书评

暂无评论

Class-specific Prompts in Vision Transformer for Continual L...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Defeng Zhao Zejun Ye Wei-Shi Zheng Ruixuan Wang School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Key Laboratory of Machine Intelligence and Advanced Computing MOE China Department of Network Intelligence Peng Cheng Laboratory Shenzhen China

Current intelligent diagnosis systems are often trained to diagnose a small number of diseases and lack the ability of continually learning new disease knowledge. To have such continual learning ability, the deployed intelligent system needs to be continually updated based on training data of only new diseases, without accessing old data of previously learned diseases due to privacy concerns and challenges in data sharing across medical centers. In this study, a novel and effective prompt learning strategy is proposed to help a pretrained and fixed Vision Transformer (ViT) continually learn new diseases. In particular, a set of unique prompts for each disease are effectively learned such that discriminative disease features can be well extracted from the fixed ViT under the instructions of prompts during feature extraction, even though the ViT feature extractor is pretrained in the natural image domain. Extensive empirical evaluations on two medical image datasets and one natural image dataset demonstrate the superior performance of the proposed method. The source code is available at https://***/zhaoedf/CSPrompt.

关键词：

Automatic Detection and Characterization of Human Veins Using Infra-Red Image Processing

学校读者我要写书评

暂无评论

Journal of Computer and Communications 2024年第9期12卷 141-159页

作者： Jean Ndoumbe Brice Ekobo Akoa Gaelle Patricia Talotsing Frederic Franck Kounga Samuel Kaissassou Bertin Chouanmo Njo Laboratory of Computer Engineering Data Science and Artificial Intelligence Department of Computer & Telecommunications Engineering National Higher Polytechnic School Douala Cameroon Laboratory of Electrical Engineering Mechatronics and Signal Processing Department of Electrical & Telecommunications Engineering National Advanced School of Engineering Yaound Cameroon Mobile Computing and Networking Research Laboratory Department of Computer and Software Engineering Polytechnique Montreal Montreal Canada DigiPlus SARL Bonamoussadi Douala Cameroon

The detection and characterization of human veins using infrared (IR) image processing have gained significant attention due to its potential applications in biometric identification, medical diagnostics, and vein-based authentication systems. This paper presents a low-cost approach for automatic detection and characterization of human veins from IR images. The proposed method uses image processing techniques including segmentation, feature extraction, and, pattern recognition algorithms. Initially, the IR images are preprocessed to enhance vein structures and reduce noise. Subsequently, a CLAHE algorithm is employed to extract vein regions based on their unique IR absorption properties. Features such as vein thickness, orientation, and branching patterns are extracted using mathematical morphology and directional filters. Finally, a classification framework is implemented to categorize veins and distinguish them from surrounding tissues or artifacts. A setup based on Raspberry Pi was used. Experimental results of IR images demonstrate the effectiveness and robustness of the proposed approach in accurately detecting and characterizing human. The developed system shows promising for integration into applications requiring reliable and secure identification based on vein patterns. Our work provides an effective and low-cost solution for nursing staff in low and middle-income countries to perform a safe and accurate venipuncture.

关键词： Vein Detection Blood Radiation Infrared Image CLAHE Algorithm Raspberry Pi

FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detector

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Jiankang Deng, Ling Gan, Zhiyong Zheng, Wei-Shi Wang, Ruixuan School of Computer Science and Engineering Sun Yat-sen University Guangdong Guangzhou China Peng Cheng Laboratory Guangdong Shenzhen China Key Laboratory of Machine Intelligence and Advanced Computing Guangdong Guangzhou China China United Network Communications Corporation Limited Guangdong Branch Guangdong Guangzhou China School of Computer Science and Engineering Key Laboratory of Machine Intelligence and Advanced Computing Sun Yat-sen University Guangdong Guangzhou China

Out-of-Distribution (OOD) detection is crucial when deploying machine learning models in open-world applications. The core challenge in OOD detection is mitigating the model’s overconfidence on OOD data. While recent methods using auxiliary outlier datasets or synthesizing outlier features have shown promising OOD detection performance, they are limited due to costly data collection or simplified assumptions. In this paper, we propose a novel OOD detection framework FodFoM that innovatively combines multiple foundation models to generate two types of challenging fake outlier images for classifier training. The first type is based on BLIP-2’s image captioning capability, CLIP’s vision-language knowledge, and Stable Diffusion’s image generation ability. Jointly utilizing these foundation models constructs fake outlier images which are semantically similar to but different from in-distribution (ID) images. For the second type, GroundingDINO’s object detection ability is utilized to help construct pure background images by blurring foreground ID objects in ID images. The proposed framework can be flexibly combined with multiple existing OOD detection methods. Extensive empirical evaluations show that image classifiers with the help of constructed fake images can more accurately differentiate real OOD images from ID ones. New state-of-the-art OOD detection performance is achieved on multiple benchmarks. The code is available at https://***/Cverchen/ACMMM2024-FodFoM. Copyright © 2024, The Authors. All rights reserved.

关键词： Visual languages

Salient Foreground-Aware Network for Person Search 16th

学校读者我要写书评

暂无评论

Salient Foreground-Aware Network for Person Search

16th Chinese Conference on Biometric Recognition, CCBR 2022

作者： Chen, Hongxu Zhang, Quan Lai, Jianhuang The School of Computer Science and Engineering Sun Yat-Sen University Guangzhou510006 China The Guangdong Key Laboratory of Information Security Technology Guangzhou510006 China The Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Guangzhou510006 China

ISBN: (纸本)9783031202322

Person search aims to simultaneously localize and identify a query person from realistic and uncropped images, which consists of person detection and re-identification. In existing methods, the extracted features come from the low-quality proposals generated by the structure like Region Proposal Network (RPN), and convolution is used to learn local features in the process of extracting features while the receptive field cannot grasp the global structural information. We propose an end-to-end network embedded with our Salient Foreground-Aware Module (SFAM). Self-attention mechanism in SFAM allows for better capture of global information. Our network incorporates our embedding method for person detection and person re-identification, which can effectively optimize the process of extracting features and improve feature expression capabilities. We merge the above modules into our Salient Foreground-Aware Network (SFAN). Extensive experiments have shown that our SFAN significantly improves the performance of end-to-end models with acceptable time-consuming and achieves state-of-the-art results. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Embeddings