检索结果-内蒙古大学图书馆

LAMARS: Large language Model-Based Anticipation Mechanism Acceleration in Real-Time Robotic Systems

IEEE ACCESS 2025年 13卷 3864-3880页

作者： Gao, Yifang Luo, Wei Wang, Xuye Zhang, Shunshun Goh, Patrick Univ Sains Malaysia Sch Elect & Elect Engn Nibong Tebal 14300 Penang Malaysia Beijing Jiaotong Univ Sch Elect Engn Beijing 100044 Peoples R China Univ Sains Malaysia Sch Pharmaceut Sci Gelugor 11800 Penang Malaysia Guangxi Univ Sci & Technol Sch Automat Liuzhou 545006 Peoples R China

Large language models (LLMs) have assumed an increasingly crucial role in robotic systems because of their ability to leverage the extensive knowledge they possess in robotic inference and task handling. Although LLMs offer significant potential, their integration into robotic systems poses substantial challenges, particularly with regard to computational efficiency and latency. To address this challenge, this study presents LAMARS, an LLM-based anticipation mechanism designed to accelerate real-time robotic systems. LAMARS leverages the predictive power and zero-shot capabilities of LLMs combined with an anticipation mechanism and vision-language processing to position a robot in advance for upcoming tasks. This reduces latency and optimizes path planning without requiring expensive training data. Our evaluations in a realistic simulation environment and with a variation of the RLBench dataset demonstrated that LAMARS achieved an average success rate of 0.79 and improves efficiency by up to 52.4% compared to existing methods, significantly lowering path planning costs. These results indicate that LAMARS effectively accelerates directive execution, making it a promising solution to minimize delays in real-time robotic systems.

关键词： Robot kinematics Real-time systems Hidden Markov models Visualization Costs Data models Planning Adaptation models Predictive models Vectors Human-robot interaction large language models latency reduction vision-language models path planning

来源：评论

学校读者我要写书评

暂无评论

Global-local prompts guided image-text embedding, alignment and aggregation for multi-label zero-shot learning

引用

JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION 2025年 106卷

作者： Song, Tiecheng Huang, Yu Yang, Feng Qin, Anyong Zhao, Yue Gao, Chenqiang Chongqing Univ Posts & Telecommun Sch Commun & Informat Engn Chongqing 400065 Peoples R China Sun Yat Sen Univ Sch Intelligent Syst Engn Shenzhen Campus Shenzhen 518107 Guangdong Peoples R China

Multi-label zero-shot learning (MLZSL) aims to classify images into multiple unseen label classes, which is a practical yet challenging task. Recent methods have used vision-language models (VLM) for MLZSL, but they do not well consider the global and local semantic relationships to align images and texts, yielding limited classification performance. In this paper, we propose a novel MLZSL approach, named global-local prompts guided image-text embedding, alignment and aggregation (GLP-EAA) to alleviate this problem. Specifically, based on the parameter-frozen VLM, we divide the image into patches and explore a simple adapter to obtain global and local image embeddings. Meanwhile, we design global-local prompts to obtain text embeddings of different granularities. Then, we introduce global-local alignment losses to establish image-text consistencies at different granularity levels. Finally, we aggregate global and local scores to compute the multi- label classification loss. The aggregated scores are also used for inference. As such, our approach integrates prompt learning, image-text alignment and classification score aggregation into a unified learning framework. Experimental results on NUS-WIDE and MS-COCO datasets demonstrate the superiority of our approach over state-of-the-art methods for both ZSL and generalized ZSL tasks.

关键词： multi-label zero-shot learning vision-language models Prompt learning Alignment

来源：评论

学校读者我要写书评

暂无评论

MMTF-DES: A fusion of multimodal transformer models for desire, emotion, and sentiment analysis of social media data

引用

NEUROCOMPUTING 2025年 623卷

作者： Aziz, Abdul Chowdhury, Nihad Karim Kabir, Muhammad Ashad Chy, Abu Nowshed Siddique, Md. Jawad Univ Chittagong Dept Comp Sci & Engn Chattogram 4331 Bangladesh Charles Sturt Univ Sch Comp Math & Engn Bathurst NSW 2795 Australia Southern Illinois Univ Dept Comp Sci Carbondale IL 62901 USA

Desires, emotions, and sentiments are pivotal in understanding and predicting human behavior, influencing various aspects of decision-making, communication, and social interactions. Their analysis, particularly in the context of multimodal data (such as images and texts) from social media, provides profound insights into cultural diversity, psychological well-being, and consumer behavior. Prior studies overlooked the use of image-text pairwise feature representation, which is crucial for the task of human desire understanding. In this research, we have proposed a unified multimodal-based framework with image-text pair settings to identify human desire, sentiment, and emotion. The core of our proposed method lies in the encoder module, which is built using two state-of-the-art multimodal vision-language models (VLMs). To effectively extract visual and contextualized embedding features from social media image and text pairs, we jointly fine-tune two pre-trained multimodal VLMs: vision-and-language Transformer (ViLT) and vision-and-Augmented-language Transformer (VAuLT). Subsequently, we use an early fusion strategy on these embedding features to obtain combined diverse feature representations. Moreover, we leverage a multi-sample dropout mechanism to enhance the generalization ability and expedite the training process of our proposed method. To evaluate our proposed approach, we used the multimodal dataset MSED for the human desire understanding task. Through our experimental evaluation, we demonstrate that our method excels in capturing both visual and contextual information, resulting in superior performance compared to other state-of-the-art techniques. Specifically, our method outperforms existing approaches by 3% for sentiment analysis, 2.2% for emotion analysis, and approximately 1% for desire analysis.

关键词： Human desire understanding Desire analysis Sentiment analysis Emotion analysis Multimodal transformer vision-language models

来源：评论

学校读者我要写书评

暂无评论

A two-step concept-based approach for enhanced interpretability and trust in skin lesion diagnosis

引用

COMPUTATIONAL AND STRUCTURAL BIOTECHNOLOGY JOURNAL 2025年 28卷 71-79页

作者： Patricio, Cristiano Teixeira, Luis F. Neves, Joao C. Univ Beira Interior Covilha Portugal NOVA LINCS Lisbon Portugal Univ Porto Fac Engn Porto Portugal INESC TEC Porto Portugal

The main challenges hindering the adoption of deep learning-based systems in clinical settings are the scarcity of annotated data and the lack of interpretability and trust in these systems. Concept Bottleneck models (CBMs) offer inherent interpretability by constraining the final disease prediction on a set of human-understandable concepts. However, this inherent interpretability comes at the cost of greater annotation burden. Additionally, adding new concepts requires retraining the entire system. In this work, we introduce a novel two-step methodology that addresses both of these challenges. By simulating the two stages of a CBM, we utilize a pretrained vision language Model (VLM) to automatically predict clinical concepts, and an off-the-shelf Large language Model (LLM) to generate disease diagnoses grounded on the predicted concepts. Furthermore, our approach supports test-time human intervention, enabling corrections to predicted concepts, which improves final diagnoses and enhances transparency in decision-making. We validate our approach on three skin lesion datasets, demonstrating that it outperforms traditional CBMs and state-of-the-art explainable methods, all without requiring any training and utilizing only a few annotated examples. The code is available at https://***/CristianoPatricio/2step-concept-based-skin-diagnosis.

关键词： Concept bottleneck models vision-language models Interpretability Skin cancer Dermoscopy

来源：评论

学校读者我要写书评

暂无评论

language-Guided Object Localization via Refined Spotting Enhancement in Remote Sensing Imagery

引用

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2025年 63卷

作者： Zhang, Peirong Zhang, Yidan Wu, Hui Liu, Xiaoxuan Hou, Yingyan Wang, Lei Chinese Acad Sci Aerosp Informat Res Inst Key Lab Network Informat Syst Technol NIST Beijing 100190 Peoples R China Chinese Acad Sci Key Lab Target Cognit & Applicat Technol TCAT Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Elect Elect & Commun Engn Beijing 100190 Peoples R China

language-guided remote sensing image object localization uses intuitive natural language interactions to locate objects of interest within satellite or drone imagery, and has a wide range of practical applications. Early research on this task typically used discriminative models that relied on predefined task heads, limited to locating a single object and lacking flexibility. In recent years, multimodal large language models (MLLMs)-based generative models leverage their language understanding ability to comprehend more complex language references and provide flexible outputs. However, these models tend to be too large and have limited accuracy in spotting dense, small objects in remote sensing scenarios. The reasons are: 1) generative models treat continuous coordinate prediction as a token classification problem, which fails to reflect the actual localization gap and 2) current methods typically use CLIP pre-trained encoders that align text-image pairs at a global level, and may not capture the fine-grained semantic object information necessary for accurate localization. To address these shortcomings, we introduce a lightweight generative model named the localization model with refined spotting enhancement (LM-RSE). Refined spotting enhancement includes the refinement of bounding box (BBox) outputs and the integration of fine-grained semantic features. Specifically, we design a BBox refinement (BBR) approach that includes a special token, a BBox decoder, and a custom regression loss function to refine the spotting precision and optimize the training process. Additionally, we propose a fine-grained semantic integration (FGSI) strategy, integrates a fine-grained image encoder, a vision-language semantic processing (VLSP) layer, and a two-stage, full-parameter training strategy, all working together to effectively enhance the granularity of spotting. Building on this, we further refine the language-guided object localization task into two types: expression-guided and

关键词： Location awareness Semantics Remote sensing Feature extraction Training Decoding Visualization Object detection Numerical models Accuracy Bounding box refinement (BBR) fine-grained semantic features language-guided object localization vision-language models

来源：评论

学校读者我要写书评

暂无评论

LM-CLIP: Adapting Positive Asymmetric Loss for Long-Tailed Multi-Label Classification

引用

IEEE ACCESS 2025年 13卷 71053-71065页

作者： Timmermann, Christoph Jung, Seunghyeon Kim, Miso Lee, Woojin Dongguk Univ Grad Sch Comp Sci & Artificial Intelligence Seoul 04620 South Korea

Accurate multi-label image classification is essential for real-world applications, especially in scenarios with long-tailed class distributions, where some classes appear frequently while others are rare. This imbalance often leads to biased models that struggle to accurately recognize underrepresented classes. Existing methods either trade off performance between head and tail classes or rely on image captions, limiting adaptability. To address these limitations, we propose LM-CLIP, a novel framework built around a unified loss function. Our Balanced Asymmetric Loss (BAL) extends traditional asymmetric loss by emphasizing the gradients of rare positive samples where the model is uncertain, mitigating bias toward dominant classes. This is complemented by a contrastive loss that pushes negative samples further from the decision boundary, creating a more optimal embedding space even in long-tailed scenarios. These loss functions together ensure balanced performance across all classes. Our framework is built on pre-trained models utilizing textual and visual features from millions of image-text pairs. Furthermore, we incorporate a dynamic sampling strategy that prioritizes rare classes based on their occurrence, which ensures effective training without compromising overall performance. Experiments conducted on VOC-MLT and COCO-MLT benchmarks demonstrate the effectiveness of our approach, achieving +4.66% and +8.14% improvements in mean Average Precision (mAP) over state-of-the-art methods. Our code is publicly available at https://***/damilab/lm-clip.

关键词： Heavily-tailed distribution Head Tail Multi label classification Training Visualization Adaptation models Focusing Tuning Optimization Long-tailed learning multi-label classification CLIP vision-language models contrastive learning class imbalance loss functions asymmetric loss balanced asymmetric loss imbalanced sampling

来源：评论

学校读者我要写书评

暂无评论

Enhanced Cleft Lip and Palate Classification Using SigLIP 2: A Comparative Study with vision Transformers and Siamese Networks

引用

APPLIED SCIENCES-BASEL 2025年第9期15卷 4766-4766页

作者： Nantha, Oraphan Sathanarugsawait, Benjaporn Praneetpolgrang, Prasong Sripatum Univ Sch Informat Technol Bangkok 10900 Thailand

This paper extends our previous work on cleft lip and/or palate (CL/P) classification, which employed vision transformers (ViTs) and Siamese neural networks. We now integrate SigLIP 2, a state-of-the-art multilingual vision-language model, for feature extraction, replacing the previously utilized BiomedCLIP. SigLIP 2 offers enhanced semantic understanding, improved localization capabilities, and multilingual support, potentially leading to more robust feature representations for CL/P classification. We hypothesize that SigLIP 2's superior feature extraction will improve the classification accuracy of CL/P types (bilateral, unilateral, and palate-only) from the UltraSuite CLEFT dataset, a collection of ultrasound video sequences capturing tongue movements during speech with synchronized audio recordings. A comparative analysis is conducted, evaluating the performance of our original ViT-Siamese network model (using BiomedCLIP) against a new model leveraging SigLIP 2 for feature extraction. Performance is assessed using accuracy, precision, recall, and F1 score, demonstrating the impact of SigLIP 2 on CL/P classification. The new model achieves statistically significant improvements in overall accuracy (86.6% vs. 82.76%) and F1 scores for all cleft types. We discuss the computational efficiency and practical implications of employing SigLIP 2 in a clinical setting, highlighting its potential for earlier and more accurate diagnosis, personalized treatment planning, and broader applicability across diverse populations. The results demonstrate the significant potential of advanced vision-language models, such as SigLIP 2, to enhance AI-powered medical diagnostics.

关键词： cleft lip and palate vision-language models few-shot learning medical image analysis AI in healthcare

来源：评论

学校读者我要写书评

暂无评论

Towards molecular structure discovery from cryo-ET density volumes via modelling auxiliary semantic prototypes

引用

BRIEFINGS IN BIOINFORMATICS 2025年第1期26卷 bbae570页

作者： Nair, Ashwin Li, Xingjian Solanki, Bhupendra Mukhopadhyay, Souradeep Jha, Ankit Uddin, Mostofa Rafid Singha, Mainak Banerjee, Biplab Xu, Min Indian Inst Sci Educ & Res Dept Data Sci Vithura 695551 Kerela India Carnegie Mellon Univ Computat Biol Dept Pittsburgh PA 15213 USA Indian Inst Technol Machine Learning & Visual Comp Lab Powai 400076 Maharashtra India Indian Inst Sci Comp Sci & Automat CV Raman Rd Bengaluru 560012 Karnataka India LNM Inst Informat Technol Comp Sci & Engn Jaipur 302031 Rajasthan India

Cryo-electron tomography (cryo-ET) is confronted with the intricate task of unveiling novel structures. General class discovery (GCD) seeks to identify new classes by learning a model that can pseudo-label unannotated (novel) instances solely using supervision from labeled (base) classes. While 2D GCD for image data has made strides, its 3D counterpart remains unexplored. Traditional methods encounter challenges due to model bias and limited feature transferability when clustering unlabeled 2D images into known and potentially novel categories based on labeled data. To address this limitation and extend GCD to 3D structures, we propose an innovative approach that harnesses a pretrained 2D transformer, enriched by an effective weight inflation strategy tailored for 3D adaptation, followed by a decoupled prototypical network. Incorporating the power of pretrained weight-inflated Transformers, we further integrate CLIP, a vision-language model to incorporate textual information. Our method synergizes a graph convolutional network with CLIP's frozen text encoder, preserving class neighborhood structure. In order to effectively represent unlabeled samples, we devise semantic distance distributions, by formulating a bipartite matching problem for category prototypes using a decoupled prototypical network. Empirical results unequivocally highlight our method's potential in unveiling hitherto unknown structures in cryo-ET. By bridging the gap between 2D GCD and the distinctive challenges of 3D cryo-ET data, our approach paves novel avenues for exploration and discovery in this domain.

关键词： cryo-electron tomography generalized class discovery vision-language models

来源：评论

学校读者我要写书评

暂无评论

Exploring the Limits of Large language models' Ability to Distinguish Between Objects

引用

APPLIED SCIENCES-BASEL 2025年第9期15卷 4620-4620页

作者： Ju, Hyeongjin Park, Incheol Nalcakan, Yagiz Jin, Youngwan Yeo, Sanghyeop Kim, Shiho Yonsei Univ Sch Integrated Technol Incheon 21983 South Korea Yonsei Univ BK21 Grad Program Intelligent Semicond Technol Incheon 21983 South Korea

This paper explores the capability of large language models (LLMs) to accurately classify objects in challenging visual scenarios, focusing on two main tasks: differentiating real objects from artificial replicas and distinguishing human figures from human-like entities (e.g., mannequins, banners). We evaluate a diverse set of vision-language models (VLMs) ranging from large-scale architectures to parameter-efficient systems across multiple question prompts designed to probe object identification, authenticity verification, and multi-object reasoning. Our experiments reveal that while many models perform reasonably well in identifying single objects, their accuracy declines substantially under more complex conditions, such as multi-object scenes or tasks requiring fine-grained judgments of authenticity. Even top-tier models exhibit noticeable performance drops from around 100% to below 15% accuracy when forced to discern real from fake items among multiple candidates, and from 100% to 83.33% accuracy on providing positional details for human-like figures. We further discuss how these performance limitations indicate gaps in current LLM-based vision systems to highlight the need for more robust spatial reasoning and attribute analysis. Our findings underscore the significance of broadening these models' multimodal understanding and refining prompts, with an eye toward improving real-world applications-from automated quality control to surveillance-where nuanced visual classification is crucial. By comparing a variety of architectures under consistent evaluation settings, this study offers insights into the barriers LLMs face when confronted with increasingly complex visual information.

关键词： largelanguage model prompt engineering vision-language models object classification

来源：评论

学校读者我要写书评

暂无评论

Enhancing open-vocabulary object detection through region-word and region-vision matching

引用

MULTIMEDIA SYSTEMS 2025年第3期31卷 1-15页

作者： Chen, Yi Wang, Chong Li, Zhehao Lin, Sunqi Xiang, Jinhui Li, Yuqi Qian, Jiangbo Ningbo Univ Fac Elect Engn & Comp Sci Ningbo 315000 Zhejiang Peoples R China

Open-vocabulary object detection (OVOD) aims to detect novel object categories beyond the training set. Existing OVOD methods have made encouraging progress by leveraging large-scale image-caption pairs and pre-trained vision-language models (VLMs). However, two main limitations exhibit: (1) The potential category-specific concepts in global captions are not fully utilized, resulting in a lack of fine-grained semantic guidance for the detector. (2) The compositional structure of multiple concepts naturally existing in image-caption pairs as represented by VLMs remains insufficiently explored, limiting the model's ability to generalize to novel category concepts. To address these limitations, we propose a novel framework called Region-Word-vision Matching (RWVM) that integrates two core modules: a Region-Word Matching (RWM) module and a Region-vision Matching (RVM) module. Our key insight is to simultaneously guide textual and visual knowledge alignment with region features to strengthen the model's understanding of complex visual scenes. Specifically, the RWM module guides fine-grained semantic aggregation by fusing local region-word matching with global image-caption matching. The RVM module leverages VLMs to capture the compositional structure of single and multiple object concepts, directly enhancing the detector's ability to learn novel category concepts. Additionally, we demonstrate that the RVM module outperforms embeddings extracted from full language models using only simplified region embeddings. Extensive experiments show that our model achieves superior performance compared to other OVOD methods, improving the average precision (AP) for novel categories on COCO and LVIS datasets.

关键词： Open-vocabulary object detection Region-word matching vision-language models Multi-modal training

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：