Mvp-Hot: A Moderate Visual Prompt for Hyperspectral Object Tracking

作     者：Zhao, Lin Xie, Shaoxiong Li, Jia Tan, Ping Hu, Wenjin 

作者机构：Hunan Institute of Science and Technology Yueyang414006 China Hunan Engineering Technology Research Center for 3D Reconstruction and Intelligent Application Yueyang414006 China 

出 版 物：《SSRN》 

年 卷 期：2024年

核心收录：

主　　题：Large datasets 

摘      要：The growing attention to hyperspectral object tracking (HOT) can be attributed to the extended spectral information available in hyperspectral images (HSIs), especially in complex scenarios. This potential makes it a promising alternative to traditional RGB-based tracking methods. However, the scarcity of large hyperspectral datasets pose a challenge for training robust hyperspectral trackers using deep learning methods. Prompt learning, a new paradigm emerging in large language models, involves adapting or fine-tuning a pre-trained model for downstream task by providing task-specific inputs. We propose a novel prompt learning method for HOT tasks, termed Moderate Visual Prompt for HOT (MVP-HOT). Specifically, MVP-HOT freezes the parameters of the pre-trained model and employs HSIs as visual prompts to leverage the knowledge of the underlying RGB model. Additionally, we develop a moderate and effective strategy to incrementally adapt the HSI prompt information. Our proposed method uses only a few (1.7M) learnable parameters and demonstrates its effectiveness. © 2024, The Authors. All rights reserved.