检索结果-内蒙古大学图书馆

zero-shot object detection with contrastive semantic association network

APPLIED INTELLIGENCE 2023年第24期53卷 30056-30068页

作者： Li, Haohe Wang, Chong Liu, Weijie Gong, Yilin Dai, Xinmiao Ningbo Univ Fac Elect Engn & Comp Sci Ningbo 315000 Zhejiang Peoples R China Shenzhen Anker Innovat Informat & Control Engn Shenzhen 518055 Guangdong Peoples R China

zero-shot object detection (ZSD) is dedicated to the task of precisely localizing and identifying unfamiliar objects that have not been encountered before. In this paper, a contrastive semantic association network is proposed to address the knowledge transfer challenge from seen classes to unseen ones in ZSD. It enables efficient information propagation through similarity-based connections, thereby establishing a clearer link between seen and unseen categories. Moreover, a visual-semantic contrastive learning technique is developed to mitigate the node convergence issue caused by the graph structure of the proposed network. By emphasizing the visual and semantic distinctiveness across different categories, the proposed model leverages semantic information and graph structure knowledge to enhance the generalization capability of seen and unseen feature projection. Extensive experiments demonstrate the superior performance of our model compared to other zero-shot object detection methods, showcasing notable improvement in mean average precision (mAP) on the MS-COCO dataset. The code and models are publicly available at: https://***/lihh1023/CSA-ZSD/tree/master.

关键词： Semantic association Graph propagation Contrastive learning zero-shot object detection

来源：评论

学校读者我要写书评

暂无评论

zero-shot object detection for Infrared Images Using Pre-trained Vision and Language Models 50

Zero-shot Object Detection for Infrared Images Using Pre-tra...

引用

Conference on Infrared Technology and Applications L

作者： Miwa, shotaro Otsubo, Shun Jia, Qu Susumu, Yasuaki Mitsubishi Electr Corp 8-1-1 Tsukaguchi Honmachi Amagasaki Hyogo Japan

ISBN: (纸本)9781510674110;9781510674103

Computer vision systems, such as object detection, traditionally rely on supervised learning and predetermined categories, an approach facing limitations when applied to infrared images due to dataset constraints. Emerging contrastive vision-language models, like (Contrastive Language-Image Pre-Training) CLIP, offer a transformative approach through their pre-training on extensive image-text pairs, providing diverse visual representations integrated with language semantics. Our work proposes a novel zero-shot object detection approach for infrared images by extending the benefits of CLIP into this domain. We have developed a two-stage detection system using CLIP for detecting humans in infrared images. The first stage involves region proposal by a (You Only Look Once) YOLO object detector, followed by CLIP in the second stage. When compared with a YOLO model fine-tuned using infrared images, our proposed system demonstrates comparable performance, illustrating its efficacy as a zero-shot object detection approach. This method opens up new avenues for infrared image processing leveraging the capabilities of foundation models.

关键词： zero-shot object detection open-vocabulary object detection vision and language model cross-domain object detection

来源：评论

学校读者我要写书评

暂无评论

zero-shot object detection WITH PARTITIONED CONTRASTIVE FEATURE ALIGNMENT 49

ZERO-SHOT OBJECT DETECTION WITH PARTITIONED CONTRASTIVE FEAT...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Li, Haohe Wang, Chong Yu, Shenghao Huo, Zheng Zheng, Yujie Qian, Liangbo Ningbo Univ Fac Elect Engn & Comp Sci Ningbo Zhejiang Peoples R China

ISBN: (纸本)9798350344868;9798350344851

How to properly align the extracted visual features with certain semantic embeddings of unseen objects is crucial to the problem of zero-shot object detection (ZSD). To give a better guess of those unseen visual features, a partitioned contrast strategy is proposed in this paper to train the visual and attribute feature alignment networks. To be specific, four types of contrast are considered, including the visual-to-visual, visual-to-attribute, attribute-to-visual and attribute-toattribute contrasts. Combining with two cross-batch memory banks of the visual features and unseen attribute features, it is effective to adjust the alignment rules for unseen visual features. Experimental results on the MS-COCO dataset show the superiority of the proposed model. Our code is available at: https://***/lihh1023/PCFA-ZSD.

关键词： attribute contrastive learning memory bank zero-shot object detection

来源：评论

学校读者我要写书评

暂无评论

Enhancing zero-shot object detection with external knowledge-guided robust contrast learning

引用

PATTERN RECOGNITION LETTERS 2024年 185卷 152-159页

作者： Duan, Lijuan Liu, Guangyuan En, Qing Liu, Zhaoying Gong, Zhi Ma, Bian Beijing Univ Technol Beijing Key Lab Trusted Comp Beijing 100124 Peoples R China Beijing Univ Technol Natl Engn Lab Crit Technol Informat Secur Classifi Beijing 100124 Peoples R China Beijing Univ Technol Fac Informat Technol Beijing 100124 Peoples R China Carleton Univ Sch Comp Sci Ottawa ON K1S 5B6 Canada

zero-shot object detection aims to identify objects from unseen categories not present during training. Existing methods rely on category labels to create pseudo-features for unseen categories, but they face limitations in exploring semantic information and lack robustness. To address these issues, we introduce a novel framework, EKZSD, enhancing zero-shot object detection by incorporating external knowledge and contrastive paradigms. This framework enriches semantic diversity, enhancing discriminative ability and robustness. Specifically, we introduce a novel external knowledge extraction module that leverages attribute and relationship prompts to enrich semantic information. Moreover, a novel external knowledge contrastive learning module is proposed to enhance the model's discriminative and robust capabilities by exploring pseudo- visual features. Additionally, we use cycle consistency learning to align generated visual features with original semantic features and adversarial learning to align visual features with semantic features. Collaboratively trained with contrast learning loss, cycle consistency loss, adversarial learning loss, and classification loss, our framework outperforms superior performance on the MSCOCO and Ship-43 datasets, as demonstrated in experimental results.

关键词： zero-shot object detection External knowledge Supervised contrastive learning

来源：评论

学校读者我要写书评

暂无评论

M-RRFS: A Memory-Based Robust Region Feature Synthesizer for zero-shot object detection

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2024年第10期132卷 4651-4672页

作者： Huang, Peiliang Zhang, Dingwen Cheng, De Han, Longfei Zhu, Pengfei Han, Junwei Northwestern Polytech Univ Sch Automat Xian Peoples R China Hefei Comprehens Natl Sci Ctr Inst Artificial Intelligence Hefei Peoples R China Xidian Univ Sch Telecommun Engn Xian Peoples R China Beijing Technol & Business Univ Sch Comp & Artificial Intelligence Beijing Peoples R China Tianjin Univ Coll Intelligence & Comp Tianjin Peoples R China

With the goal to detect both the object categories appearing in the training phase and those never have been observed before testing, zero-shot object detection (ZSD) becomes a challenging yet anticipated task in the community. Current approaches tackle this problem by drawing on the feature synthesis techniques used in the zero-shot image classification (ZSC) task without delving into the inherent problems of ZSD. In this paper, we analyze the out-standing challenges that ZSD presents compared with ZSC-severe intra-class variation, complex category co-occurrence, open test scenario, and reveal their interference to the region feature synthesis process. In view of this, we propose a novel memory-based robust region feature synthesizer (M-RRFS) for ZSD, which is equipped with the Intra-class Semantic Diverging (IntraSD), the Inter-class Structure Preserving (InterSP), and the Cross-Domain Contrast Enhancing (CrossCE) mechanisms to overcome the inadequate intra-class diversity, insufficient inter-class separability, and weak inter-domain contrast problems. Moreover, when designing the whole learning framework, we develop an asynchronous memory container (AMC) to explore the cross-domain relationship between the seen class domain and unseen class domain to reduce the overlap between the distributions of them. Based on AMC, a memory-assisted ZSD inference process is also proposed to further boost the prediction accuracy. To evaluate the proposed approach, comprehensive experiments on MS-COCO, PASCAL VOC, ILSVRC and DIOR datasets are conducted, and superior performances have been achieved. Notably, we achieve new state-of-the-art performances on MS-COCO dataset, i.e., 64.0%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document}, 60.9%\documentclass[12pt]{minimal} \usepackage{amsmat

关键词： object detection zero-shot learning zero-shot object detection Region feature synthesis

来源：评论

学校读者我要写书评

暂无评论

Semantics-Guided Contrastive Network for zero-shot object detection

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024年第3期46卷 1530-1544页

作者： Yan, Caixia Chang, Xiaojun Luo, Minnan Liu, Huan Zhang, Xiaoqin Zheng, Qinghua Xi An Jiao Tong Univ Sch Elect & Informat Engn Key Lab Intelligent Networks & Network Secur Minist EducNatl Engn Lab Big Data Analyt Xian 710049 Shaanxi Peoples R China Univ Technol Sydney Fac Engn & Informat Technol ReLER Lab AAAI Ultimo NSW 2007 Australia RMIT Univ Sch Comp Technol Melbourne Vic 3000 Australia Wenzhou Univ Coll Comp Sci & Artificial Intelligence Wenzhou 325000 Zhejiang Peoples R China

zero-shot object detection (ZSD), the task that extends conventional detection models to detecting objects from unseen categories, has emerged as a new challenge in computer vision. Most existing approaches tackle the ZSD task with a strict mapping-transfer strategy that may lead to suboptimal ZSD results: 1) the learning process of these models neglects the available semantic information on unseen classes, which can easily bias towards the seen categories;2) the original visual feature space is not well-structured for the ZSD task due to the lack of discriminative information. To address these issues, we develop a novel Semantics-Guided Contrastive Network for ZSD, named ContrastZSD, a detection framework that first brings contrastive learning mechanism into the realm of zero-shot detection. Particularly, ContrastZSD incorporates two semantics-guided contrastive learning subnets that contrast between region-category and region-region pairs respectively. The pairwise contrastive tasks take advantage of supervision signals derived from both the ground truth label and class similarity information. By performing supervised contrastive learning over those explicit semantic supervision, the model can learn more knowledge about unseen categories to avoid the bias problem to seen concepts, while optimizing the visual data structure to be more discriminative for better visual-semantic alignment. Extensive experiments are conducted on two popular benchmarks for ZSD, i.e., PASCAL VOC and MS COCO. Results show that our method outperforms the previous state-of-the-art on both ZSD and generalized ZSD tasks.

关键词： Visualization Semantics object detection Task analysis Proposals Feature extraction Computational modeling zero-shot learning zero-shot object detection supervised contrastive learning

来源：评论

学校读者我要写书评

暂无评论

zero-shot object detection Through Vision-Language Embedding Alignment 22

Zero-shot Object Detection Through Vision-Language Embedding...

引用

22nd IEEE International Conference on Data Mining (ICDM)

作者： Xie, Johnathan Zheng, Shuai Stanford Univ Stanford CA 94305 USA Cruise LLC Sunnyvale CA USA

ISBN: (纸本)9798350346091

Recent approaches have shown that training deep neural networks directly on large-scale image-text pair collections enables zero-shot transfer on various recognition tasks. One central issue is how this can be generalized to object detection, which involves the non-semantic task of localization as well as semantic task of classification. To solve this problem, we introduce a vision-language embedding alignment method that transfers the generalization capabilities of a pretrained model such as CLIP to an object detector like YOLOv5. We formulate a loss function that allows us to align the image and text embeddings from the pretrained model CLIP with the modified semantic prediction head from the detector. With this method, we are able to train an object detector that achieves state-of-the-art performance on the COCO, ILSVRC, and Visual Genome zero-shot detection benchmarks. During inference, our model can be adapted to detect any number of object classes without additional training. We also find that standard object detection scaling can transfer well to our method and find consistent improvements across various scales of YOLOv5 models and the YOLOv3 model. Lastly, we develop a self-labeling method that provides a significant score improvement without needing extra images nor labels. The code is available at https://***/Johnathan-Xie/ZSD-YOLO.

关键词： zero-shot Learning object detection Vision-Language Alignment zero-shot object detection

来源：评论

学校读者我要写书评

暂无评论

Adaptive adjustment with semantic embedding for zero-shot object detection

引用

JOURNAL OF ELECTRONIC IMAGING 2023年第3期32卷 033001-033001页

作者： Lv, Wen Shi, Hongbo Tan, Shuai Song, Bing Tao, Yang East China Univ Sci & Technol Control Sci & Engn Shanghai Peoples R China

Traditional zero-shot object-detection algorithms detect images of untrained classes in the model with the help of semantic embedding. However, these approaches may perform poorly due to the limitations of fixed semantic embedding. Given that fixed semantic attributes lead to a lack of generalization capabilities in the model, a semantic enhancement mechanism is proposed to update the semantic embedding, which is used to serve the needs of the visual space. Specifically, considering that the original semantic space is not enough to construct a visual-semantic mapping relationship, an augmented semantic embedding (ASE) approach is designed to supplement semantic attribute information. Then, a semantic channel attention mechanism is used to adjust the ASE. The adjustment strategy retains adequate attribute information, which is highly relevant to visual features. Finally, to alleviate the domain shift problem, a clustering association strategy is introduced to establish an inferred relationship, which ensures that the predictor is generalized to the unseen domain during training. The superiority of the proposed method is demonstrated by the MS-COCO and PASCAL VOC datasets. (C) 2023 SPIE and IS&T

关键词： zero-shot object detection semantic embedding semantic enhancement mechanism clustering association strategy

来源：评论

学校读者我要写书评

暂无评论

A dynamic semantic knowledge graph for zero-shot object detection

引用

VISUAL COMPUTER 2023年第10期39卷 4513-4527页

作者： Lv, Wen Shi, Hongbo Tan, Shuai Song, Bing Tao, Yang East China Univ Sci & Technol Minist Educ Key Lab Smart Mfg Energy Chem Proc Shanghai 200237 Peoples R China

zero-shot object detection (ZSD) learns a mapping relationship between visual space and semantic space;therefore, ZSD can rely on semantic information to identify and localize novel classes. However, due to the variety of images in the same category, ZSD algorithms usually use a fixed semantic embedding resulting in a visual-semantic significant gap. To bridge the gap, a dynamic semantic knowledge graph (DSKG) is proposed based on the visual-semantic relation. First, in order to capture the similarity between semantic information, a semantic knowledge graph is utilized to establish connections between categories. Then, a dynamic semantic reasoning mechanism is introduced to update semantic embedding based on the self-attention mechanism. Finally, experiments show that the DSKG can achieve significant improvements on MS-COCO and PASCAL VOC datasets.

关键词： zero-shot object detection Semantic embedding Knowledge graph Semantic reasoning

来源：评论

学校读者我要写书评

暂无评论

Synthetic Feature Assessment for zero-shot object detection

Synthetic Feature Assessment for Zero-Shot Object Detection

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Dai, Xinmiao Wang, Chong Li, Haohe Lin, Sunqi Dong, Li Wu, Jiafei Wang, Jun Ningbo Univ Ningbo Peoples R China SenseTime Res Shenzhen Peoples R China China Univ Min & Technol Xuzhou Jiangsu Peoples R China

ISBN: (纸本)9781665468916

zero-shot object detection aims to simultaneously identify and localize classes that were not presented during training. Many generative model-based methods have shown promising performance by synthesizing the visual features of unseen classes from semantic embeddings. However, these synthetic features are inevitably of varied quality, which may be far from the ground truth. It degrades the performance of trained unseen classifier. Instead of tweaking the generative model, a new idea of feature quality assessment is proposed to utilize both the good and bad features to optimize the classifier in the right direction. Moreover, contrastive learning is also introduced to enhance the feature uniqueness between unseen and seen classes, which helps the feature assessment implicitly. To demonstrate the effectiveness of the proposed algorithm, comprehensive experiments are conducted on the MS COCO dataset and PASCAL VOC dataset, the state-of-the-art performance is achieved. Our code is available at: https://***/Dai1029/SFA-ZSD.

关键词： zero-shot object detection generative model contrastive learning feature assessment

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：