检索结果-内蒙古大学图书馆

CPAL: Cross-prompting Adapter with LoRAs for RGB+X Semantic Segmentation

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Liu, Ye Wu, Pengfei Wang, Miaohui Liu, Jun Nanjing University of Posts and Telecommunications School of Automation and Artificial Intelligence Nanjing China Shenzhen University Guangdong Key Laboratory of Intelligent Information Processing Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China Lancaster University School of Computing and Communications Lancaster United Kingdom

As sensor technology evolves, RGB+X systems combine traditional RGB cameras with another type of auxiliary sensor, which enhances perception capabilities and provides richer information for important tasks such as semantic segmentation. However, acquiring massive RGB+X data is difficult due to the need for specific acquisition equipment. Therefore, traditional RGB+X segmentation methods often perform pretraining on relatively abundant RGB data. However, these methods lack corresponding mechanisms to fully exploit the pretrained model, and the scope of the pretraining RGB dataset remains limited. Recent works have employed prompt learning to tap into the potential of pretrained foundation models, but these methods adopt a unidirectional prompting approach i.e., using X or RGB+X modality to prompt pretrained foundation models in RGB modality, neglecting the potential in non-RGB modalities. In this paper, we are dedicated to developing the potential of pretrained foundation models in both RGB and non-RGB modalities simultaneously, which is non-trivial due to the semantic gap between modalities. Specifically, we present the CPAL (Cross-prompting Adapter with LoRAs), a framework that features a novel bi-directional adapter to simultaneously fully exploit the complementarity and bridging the semantic gap between modalities. Additionally, CPAL introduces low-rank adaption (LoRA) to fine-tune the foundation model of each modal. With the support of these elements, we have successfully unleashed the potential of RGB foundation models in both RGB and non-RGB modalities simultaneously. Our method achieves state-of-the-art (SOTA) performance on five multi-modal benchmarks, including RGB+Depth, RGB+Thermal, RGB+Event, and a multi-modal video object segmentation benchmark, as well as four multi-modal salient object detection benchmarks. The code and results are available at: https://***/abelny56/CPAL. © 1991-2012 IEEE.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models

arXiv

引用

arXiv 2025年

作者： Lv, Xingtai Sun, Youbang Zhang, Kaiyan Qu, Shang Zhu, Xuekai Fan, Yuchen Wu, Yi Hua, Ermo Long, Xinwei Ding, Ning Zhou, Bowen Department of Electronic Engineering Tsinghua University China Shanghai Artificial Intelligence Laboratory China Robotics Institute Carnegie Mellon University United States

State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SSMs excel at tasks with sequential data or longer contexts, demonstrating comparable performances with significant efficiency gains. In this survey, we provide a coherent and systematic overview for SSMs, including their theoretical motivations, mathematical formulations, comparison with existing model classes, and various applications. We divide the SSM series into three main sections, providing a detailed introduction to the original SSM, the structured SSM represented by S4, and the selective SSM typified by Mamba. We put an emphasis on technicality, and highlight the various key techniques introduced to address the effectiveness and efficiency of SSMs. We hope this manuscript serves as an introduction for researchers to explore the theoretical foundations of SSMs. Copyright © 2025, The Authors. All rights reserved.

关键词： State space methods

来源：评论

学校读者我要写书评

暂无评论

Refiner: Fine-grained Cross-modal Concepts Refinement for Compositional Zero-Shot Learning

Refiner: Fine-grained Cross-modal Concepts Refinement for Co...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Xiao Zhang Haodong Jing Hui Chen Yongqiang Ma Nanning Zheng National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications and Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Recent Compositional Zero-Shot Learning (CZSL) methods increasingly adopt the pre-trained vision-language models to capture the contextual relations between image and text spaces. However, the single-class-token design from Transformer-based encoder inevitably captures contextual information from unrelated objects and background, thus hindering the modeling of fine-grained class-specific visual features. Suffering from cross-modal gap, prior methods also struggle to improve compositional recognition performance. To address these issues, we propose a fine-grained cross-modal concepts refinement framework, termed as Refiner, which comprises two pivotal components: (i) the fine-grained concepts refinement of image embeddings to capture state-object context within visual scenes, and (ii) the cross-modal information fusion to mitigate the modality gap. By leveraging learnable query vectors to capture region-specific semantic information pertinent to composition labels, our approach refines visual representations with fine-grained state-object context information. As for cross-modal information fusion, we construct a robust image-to-text mapping by aligning visual embeddings with states, objects, and compositions, respectively. Extensive experiments demonstrate that our Refiner achieves new state-of-the-art performance across all popular benchmarks in both closed- and open-world settings.

关键词： Visualization Zero shot learning Semantics Benchmark testing Signal processing Transformers Vectors Acoustics Speech processing Context modeling

来源：评论

学校读者我要写书评

暂无评论

Efficient and Cost-effective Vehicle Recruitment for HD Map Crowdsourcing

引用

IEEE Transactions on Mobile Computing 2025年

作者： Ye, Wentao Luo, Yuan Liu, Bo Huang, Jianwei The Chinese University of Hon g Kong Shenzhen Institute of Artificial Intelligence and Robotics for Society School of Science and Engineering Shenzhen Guangdong518172 China The Chinese University of Hon g Kong School of Science and Engineering Shenzhen Key Laboratory of Crowd Intelligence Empowered Low-Carbon Energy Network Shenzhen Guangdong518172 China Shenzhen Institute of Artificial Intelligence and Robotics for Society Guangdong518172 China The Chinese University of Hon g Kong School of Science and Engineering Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen Key Laboratory of Crowd Intelligence Empowered Low-Carbon Energy Network Csijri Joint Research Centre on Smart Energy Storage Shenzhen Guangdong518172 China

The high-definition (HD) map is a cornerstone of autonomous driving. The crowdsourcing paradigm is a cost-effective way to keep an HD map up-to-date. Current HD map crowdsourcing mechanisms aim to enhance HD map freshness within recruitment budgets. However, many overlook unique and critical traits of crowdsourcing vehicles, such as random arrival and heterogeneity, leading to either compromised map freshness or excessive recruitment costs. Furthermore, these characteristics complicate the characterization of the feasible space of the optimal recruitment policy, necessitating a method to compute it efficiently in dynamic transportation scenarios. To overcome these challenges, we propose an efficient and cost-effective vehicle recruitment (ENTER) mechanism. Specifically, the ENTER mechanism has a threshold structure and balances freshness with recruitment costs while accounting for the vehicles' random arrival and heterogeneity. It also integrates the bound-based relative value iteration (RVI) algorithm, which utilizes the threshold-type structure and upper bounds of thresholds to reduce the feasible space and expedite convergence. Numerical results show that the proposed ENTER mechanism increases the HD map company's payoff by 23.40% and 43.91% compared to state-of-the-art mechanisms that do not account for vehicle heterogeneity and random arrivals, respectively. Furthermore, the bound-based RVI algorithm in the ENTER mechanism reduces computation time by an average of 18.91% compared to the leading RVI-based algorithm. © 2025 IEEE.

关键词： Markov processes

来源：评论

学校读者我要写书评

暂无评论

BOOTSTRAPPED MODEL PREDICTIVE CONTROL

arXiv

引用

arXiv 2025年

作者： Wang, Yuhang Guo, Hanwei Wang, Sizhe Qian, Long Lan, Xuguang National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Application Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an China

Model Predictive Control (MPC) has been demonstrated to be effective in continuous control tasks. When a world model and a value function are available, planning a sequence of actions ahead of time leads to a better policy. Existing methods typically obtain the value function and the corresponding policy in a model-free manner. However, we find that such an approach struggles with complex tasks, resulting in poor policy learning and inaccurate value estimation. To address this problem, we leverage the strengths of MPC itself. In this work, we introduce Bootstrapped Model Predictive Control (BMPC), a novel algorithm that performs policy learning in a bootstrapped manner. BMPC learns a network policy by imitating an MPC expert, and in turn, uses this policy to guide the MPC process. Combined with model-based TD-learning, our policy learning yields better value estimation and further boosts the efficiency of MPC. We also introduce a lazy reanalyze mechanism, which enables computationally efficient imitation learning. Our method achieves superior performance over prior works on diverse continuous control tasks. In particular, on challenging high-dimensional locomotion tasks, BMPC significantly improves data efficiency while also enhancing asymptotic performance and training stability, with comparable training time and smaller network sizes. Code is available at https://***/wertyuilife2/bmpc. Copyright © 2025, The Authors. All rights reserved.

关键词： Model predictive control

来源：评论

学校读者我要写书评

暂无评论

Semantic Graph Embedded Energy Minimization Learning for Scene Graph Generation

Semantic Graph Embedded Energy Minimization Learning for Sce...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Jinghang Chen Chi Zhang Yuehu Liu Le Wang Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an China National Engineering Research Center for Visual Information and Applications National Key Laboratory of Human-Machine Hybrid Augmented Intelligence

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

The performance of current scene graph generation models is affected by training with cross-entropy loss, exacerbating the problem of prediction bias stemming from biased training data. Energy-based model adopts a learning method for joint image and scene graph to alleviate this challenge. However, this method only focuses on the visual features of images, neglecting the rich relation information contained in the semantic space. To address this issue, we innovatively employ powerful pre-trained large models to realize a simple yet effective semantic graph embedded energy minimization framework for the SGG task. Specifically, we use large models to generate image descriptions and extract relation triplets, which are then transformed into semantic graphs with entities as nodes and relations as edges. Moreover, by mapping these graphs into the same space using GNN to learn the minimal energy value, our approach enables SGG model to learn structural information in both semantic and visual spaces. We validate the effectiveness and efficiency of our method on the SGG benchmark Visual Genome dataset. Compared with prevailing models and EBM, we achieve a significant performance improvement of up to 2.99% and 2.24%, respectively.

关键词： Training Learning systems Visualization Image edge detection Semantics Training data Signal processing Predictive models Minimization Speech processing

来源：评论

学校读者我要写书评

暂无评论

BrainCLIP: Brain Representation via CLIP for Generic Natural Visual Stimulus Decoding

引用

IEEE Transactions on Medical Imaging 2025年 PP卷 PP页

作者： Ma, Yongqiang Liu, Yulong Chen, Liangjun Zhu, Guibo Chen, Badong Zheng, Nanning Xi'an Jiaotong University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center of Visual Information and Applications Institute of Artificial Intelligence and Robotics Shaanxi Xi'an710049 China Institute of Automation Beijing China

Functional Magnetic Resonance Imaging (fMRI) presents challenges due to limited paired samples and low signal-to-noise ratios, particularly in tasks involving reconstructing natural images or decoding their semantic content. To address these challenges, we introduce BrainCLIP, an innovative fMRI-based brain decoding model. BrainCLIP leverages Contrastive Language-Image Pre-training's (CLIP) cross-modal generalization abilities to bridge brain activity, images, and text for the first time. Our experiments demonstrate CLIP's effectiveness in diverse brain decoding tasks, including zero-shot visual category decoding, fMRI-image/text alignment, and fMRI-to-image generation. The core objective of BrainCLIP is to train a mapping network that translates fMRI patterns into a unified CLIP embedding space, achieved through visual and textual supervision integration. Our experiments highlight that this approach significantly enhances performance in tasks such as fMRI-text alignment and fMRI-based image generation. Notably, BrainCLIP surpasses BraVL, a recent multi-modal method, in zero-shot visual category decoding. Moreover, BrainCLIP demonstrates strong capability in reconstructing visual stimuli with high semantic fidelity, competing favorably with state-of-the-art methods in capturing high-level semantic features during fMRI-based natural image reconstruction. © 1982-2012 IEEE.

关键词： Signal to noise ratio

来源：评论

学校读者我要写书评

暂无评论

Nearly Lossless Adaptive Bit Switching

arXiv

引用

arXiv 2025年

作者： Huang, Haiduo Liu, Zhenhua Xia, Tian Zhao, Wenzhe Ren, Pengju The National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center of Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Shaanxi Xi’an China

Model quantization is widely applied for compressing and accelerating deep neural networks (DNNs). However, conventional Quantization-Aware Training (QAT) focuses on training DNNs with uniform bit-width. The bit-width settings vary across different hardware and transmission demands, which induces considerable training and storage costs. Hence, the scheme of one-shot joint training multiple precisions is proposed to address this issue. Previous works either store a larger FP32 model to switch between different precision models for higher accuracy or store a smaller INT8 model but compromise accuracy due to using shared quantization parameters. In this paper, we introduce the Double Rounding quantization method, which fully utilizes the quantized representation range to accomplish nearly lossless bit-switching while reducing storage by using the highest integer precision instead of full precision. Furthermore, we observe a competitive interference among different precisions during one-shot joint training, primarily due to inconsistent gradients of quantization scales during backward propagation. To tackle this problem, we propose an Adaptive Learning Rate Scaling (ALRS) technique that dynamically adapts learning rates for various precisions to optimize the training process. Additionally, we extend our Double Rounding to one-shot mixed precision training and develop a Hessian-Aware Stochastic Bit-switching (HASB) strategy. Experimental results on the ImageNet-1K classification demonstrate that our methods have enough advantages to state-of-the-art one-shot joint QAT in both multi-precision and mixed-precision. We also validate the feasibility of our method on detection and segmentation tasks, as well as on LLMs task. Our codes are available at https://***/haiduo/Double-Rounding. Copyright © 2025, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

StructVPR++: Distill Structural and Semantic Knowledge with Weighting Samples for Visual Place Recognition

arXiv

引用

arXiv 2025年

作者： Shen, Yanqing Zhou, Sanping Fu, Jingwen Wang, Ruotong Chen, Shitao Zheng, Nanning National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Shaanxi Xi’an710049 China

Visual place recognition is a challenging task for autonomous driving and robotics, which is usually considered as an image retrieval problem. A commonly used two-stage strategy involves global retrieval followed by re-ranking using patch-level descriptors. Most deep learning-based methods in an end-to-end manner cannot extract global features with sufficient semantic information from RGB images. In contrast, re-ranking can utilize more explicit structural and semantic information in one-to-one matching process, but it is time-consuming. To bridge the gap between global retrieval and re-ranking and achieve a good trade-off between accuracy and efficiency, we propose StructVPR++, a framework that embeds structural and semantic knowledge into RGB global representations via segmentation-guided distillation. Our key innovation lies in decoupling label-specific features from global descriptors, enabling explicit semantic alignment between image pairs without requiring segmentation during deployment. Furthermore, we introduce a sample-wise weighted distillation strategy that prioritizes reliable training pairs while suppressing noisy ones. Experiments on four benchmarks demonstrate that StructVPR++ surpasses state-of-the-art global methods by 5-23% in Recall@1 and even outperforms many two-stage approaches, achieving real-time efficiency with a single RGB input. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

A Novel Dense Object Detector with Scale Balanced Sample Assignment and Refinement

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Dong, Jinpeng Yao, Dingyi Hu, Yufeng Zhou, Sanping Zheng, Nanning Xi’an Jiaotong University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi’an710049 China Xi’an Jiaotong University Shaanxi Xi’an710049 China

Scale variation of objects remains one of the crucial challenges in object detection. Currently, conventional dense detectors with fixed receptive fields and label weights are not conducive to the detection of multi-scale objects. However, the design limitations of unbalanced label weights and fixed refinement for multi-scale objects and multi-tasks in these studies make it difficult to achieve better detection performance. In this paper, we propose a novel dense detector named Balanced FCOS which consists of two components: Balanced Label Assignment (BLA) and Flexible Shape-based Refinement (FSR). The BLA implements scale-balanced sample assignment by introducing reweighting factors consisting of localization and classification scores into the label assignment. Low-quality but high-weight samples can be weakened by the BLA. Furthermore, we design a cross-reweighting mechanism in the BLA to ensure score consistency between classification and localization. The FSR implements scale-balanced sample refinement by learning flexible sample points’ offsets for multi-scale objects and multi-tasks based on objects’ coarse features to get more discriminative features with appropriate receptive field. In addition, better features obtained by FSR are beneficial to get better classification and localization scores, which can be used by BLA to produce accurate label weights. Only equipped with the BLA, we can achieve 41.7/46.6 AP under R50/R101-FCOS without any additional parameters. When combining the BLA with the FSR, our Balanced FCOS achieves SOTA results among dense detectors on the COCO test-dev set. Experiments conducted on other heads (T-Head, DyHead), detectors (DINO), and datasets (AI-TOD) further demonstrate the effectiveness of our method. (Figure presented) © 1991-2012 IEEE.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：