检索结果-内蒙古大学图书馆

visual-Linguistic Feature Alignment with Semantic and Kinematic Guidance for Referring Multi-Object Tracking

IEEE Transactions on Multimedia 2025年 27卷 3034-3044页

作者： Li, Yizhe Zhou, Sanping Qin, Zheng Wang, Le National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University Shaanxi Xi'an710049 China

Referring Multi-Object Tracking (RMOT) aims to dynamically track an arbitrary number of referred targets in a video sequence according to the language expression. Previous methods mainly focus on cross-modal fusion at the feature level with designed structures. However, the insufficient visual-linguistic alignment is prone to causing visual-linguistic mismatches, leading to some targets being tracked but not correctly referred especially when facing the language expression with complex semantics or motion descriptions. To this end, we propose to conduct visual-linguistic alignment with semantic and kinematic guidance to effectively align the visual features with more diverse language expressions. In this paper, we put forward a novel end-to-end RMOT framework SKTrack, which follows the transformer-based architecture with a Language-Guided Decoder (LGD) and a Motion-Aware Aggregator (MAA). In particular, the LGD performs deep semantic interaction layer-by-layer in a single frame to enhance the alignment ability of the model, while the MAA conducts temporal feature fusion and alignment across multiple frames to enable the alignment between visual targets and language expression with motion descriptions. Extensive experiments on the Refer-KITTI and Refer-KITTI-v2 demonstrate that SKTrack achieves state-of-the-art performance and verify the effectiveness of our framework and its components. © 1999-2012 IEEE.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Event-enhanced synthetic aperture imaging

引用

Science China(information Sciences) 2025年第3期68卷 388-390页

作者： Siqi LI Shaoyi DU Jun-Hai YONG Yue GAO Beijing National Research Center for Information Science and Technology School of SoftwareTsinghua University Institute for Brain and Cognitive Sciences Beijing Laboratory of Brain and Cognitive IntelligenceTsinghua University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and ApplicationsInstitute of Artificial Intelligence and Robotics Xi'an Jiaotong University

Synthetic aperture imaging(SAI) methods aim to see through dense occlusions and reconstruct the target scene behind occlusions. Traditional frame-based SAI methods,e.g., DeOccNet [1], take the occluded light field ima...

关键词：

来源：评论

学校读者我要写书评

暂无评论

Refiner: Fine-grained Cross-modal Concepts Refinement for Compositional Zero-Shot Learning

Refiner: Fine-grained Cross-modal Concepts Refinement for Co...

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal processing, ICASSP 2025

作者： Zhang, Xiao Jing, Haodong Chen, Hui Ma, Yongqiang Zheng, Nanning National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University Shaanxi China

ISBN: (纸本)9798350368741

Recent Compositional Zero-Shot Learning (CZSL) methods increasingly adopt the pre-trained vision-language models to capture the contextual relations between image and text spaces. However, the single-class-token design from Transformer-based encoder inevitably captures contextual information from unrelated objects and background, thus hindering the modeling of fine-grained class-specific visual features. Suffering from cross-modal gap, prior methods also struggle to improve compositional recognition performance. To address these issues, we propose a fine-grained cross-modal concepts refinement framework, termed as Refiner, which comprises two pivotal components: (i) the fine-grained concepts refinement of image embeddings to capture state-object context within visual scenes, and (ii) the cross-modal information fusion to mitigate the modality gap. By leveraging learnable query vectors to capture region-specific semantic information pertinent to composition labels, our approach refines visual representations with fine-grained state-object context information. As for cross-modal information fusion, we construct a robust image-to-text mapping by aligning visual embeddings with states, objects, and compositions, respectively. Extensive experiments demonstrate that our Refiner achieves new state-of-the-art performance across all popular benchmarks in both closed- and open-world settings. © 2025 IEEE.

关键词： Compositional Zero-shot Learning Cross-Modal Fusion Fine-Grained Refinement Multimodal Models

来源：评论

学校读者我要写书评

暂无评论

Efficient Matrix-Based Multi-view Projection Features Combined for Multi-modal 3D Semantic Segmentation 21st

Efficient Matrix-Based Multi-view Projection Features Combin...

引用

21st Pacific Rim International Conference on Artificial Intelligence, PRICAI 2024

作者： Zhao, Jiayu Wei, Fei-Fei Liang, Anqi Mei, Kuizhi College of Artificial Intelligence Xi’an Jiaotong University Xi’an China National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center of Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an China

ISBN: (纸本)9789819601219

This paper introduces BFVTModel, an efficient 3D semantic segmentation Model. Multi-modal segmentators, which employ LiDAR and Camera sensors as input, have gained popularity owing to their capacity to leverage the semantic information in image data and complementary details in point cloud data. However, the segmentation model using multi-modal input faces some problems, such as the modal consistency problem and the model complexity problem. In this paper, we propose utilizing the idea of an Occupancy-based multi-view feature combination method to obtain the 3D information of the features of the two modalities independently. We also design a projection structure to obtain the three-axis features in different modalities according to the information, including the main view, side view, and top view features. We construct a module called Feature View Transform (FVT) that combines three-axis plane features with bias. The CNN neural network is used to replace attention mechanisms, reducing the dimensionality of the original three-dimensional space matrix, thereby reducing the parameter count and increasing the model’s speed. We used a bilinear mapping structure to fuse LiDAR and Camera features and complete the 3D semantic segmentation task. We validate the model on the NuScenes dataset and obtain competitive accuracy results while leading the way in efficiency evaluation metrics. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

Spectral Aggregation Cross-Square Transformer for Hyperspectral Image Denoising 27th

Spectral Aggregation Cross-Square Transformer for Hyperspect...

引用

27th International Conference on Pattern Recognition, ICPR 2024

作者： Liu, Yang Ji, Yantao Xiao, Jiahua Guo, Yu Jiang, Peilin Yang, Haiwei Wang, Fei School of Software Engineering Xi’an Jiaotong University Shaanxi Xi’an710049 China National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications and Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Shaanxi Xi’an710049 China

ISBN: (纸本)9783031783531

Hyperspectral image(HSI) denoising addresses noise impact during image acquisition. Transformers have gained notable prominence in the field of denoising, but their quadratic self-attention complexity poses computational challenges, hindering global information processing. Classical window-based self-attention limits non-local information flow, hampering large-scale object capture in HSI. Furthermore, spectral variations among neighboring bands introduce redundancy, which burdens the model and diminishes token variability, resulting in over-smoothing in the attention map. To address these issues, we propose a novel method, named Spectral Aggregation Cross-Square Transformer(SACT). We introduce a cross-square self-attention mechanism to enhance information exchange between windows, capturing long-range dependencies within spatial intra-spectrum from multiple perspectives. Spectrally, it extends the attention region horizontally, surrounding, and vertically, exploring omnidirectional spatial correlations among different receptive windows. Additionally, a spatial-spectral aggregation self-attention module is designed to capture global contextual dependencies across spatial and spectral dimensions, reducing spectral redundancy computation. Our method has evaluated synthetic and real hyperspectral datasets and shows SACT’s effectiveness in enhancing both quantitative and qualitative HSI denoising performance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Image denoising

来源：评论

学校读者我要写书评

暂无评论

VIPNet: Combining Viewpoint information and Shape Priors for Instant Multi-view 3D Reconstruction 17th

VIPNet: Combining Viewpoint Information and Shape Priors fo...

引用

17th Asian Conference on Computer Vision, ACCV 2024

作者： Ye, Weining Li, Zhixuan Jiang, Tingting School of Computer Science Peking University Beijing100871 China College of Computing and Data Science Nanyang Technological University Singapore Singapore National Engineering Research Center of Visual Technology National Key Laboratory for Multimedia Information Processing School of Computer Science National Biomedical Imaging Center Peking University Beijing100871 China

ISBN: (纸本)9789819609680

While the multi-view 3D reconstruction task has made significant progress, existing methods simply fuse multi-view image features without effectively leveraging available auxiliary information, especially the viewpoint information for guiding and associating features of different views. To this end, we propose to enhance multi-view 3D reconstruction with the power of viewpoint information. Specifically, a simple-yet-effective viewpoint estimator is designed to learn and provide comprehensive viewpoint knowledge for locating and associating learned features from different views. Moreover, to improve the 3D reconstruction quality when 2D images of only very few viewpoints are available, we propose to learn the shape prior knowledge to provide sufficient shape information for compensating the limited 2D observations. Overall, we present VIPNet, benefiting from Viewpoint information and Shape Prior learning for high-quality multi-view 3D reconstruction. Extensive experiments validate the effectiveness of the proposed VIPNet, which achieves state-of-the-art performance on challenging datasets and shows well generalization ability in real-world scenarios. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： 3D modeling

来源：评论

学校读者我要写书评

暂无评论

Enhancing NR-IQA Model Robustness through Simple Image Compression Techniques

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Liu, Yujia Yang, Chenxi Yu, Zhaofei Huang, Tiejun Peking University National Engineering Research Center of Visual Technology State Key Laboratory of Multimedia Information Processing School of Computer Science Beijing China Peking University School of Mathematical Sciences Beijing China China Institution for Artificial Intelligence Peking University National Engineering Research Center of Visual Technology State Key Laboratory of Multimedia Information Processing China

No-Reference Image Quality Assessment (NR-IQA) plays a crucial role in various real-world applications by predicting image quality scores without the need for reference images. Despite the impressive performance of deep learning-based NR-IQA models, they remain vulnerable to adversarial attacks, which introduce imperceptible perturbations to input images, causing significant changes in predicted scores. In this study, we explore the use of simple JPEG compression techniques, as well as their combination with norm regularization training, to defend against these adversarial attacks. Our results demonstrate that image compression is an effective method to enhance model robustness, and it can further improve the robustness of NR-IQA models when combined with appropriate training strategies. Since excessive compression may reduce performance on clean images, it is essential to strike a balance. This work provides valuable insights into designing effective image compression methods for NR-IQA models. © 1991-2012 IEEE.

关键词： Image compression

来源：评论

学校读者我要写书评

暂无评论

BrainCLIP: Brain Representation via CLIP for Generic Natural visual Stimulus Decoding

引用

IEEE Transactions on Medical Imaging 2025年 PP卷 PP页

作者： Ma, Yongqiang Liu, Yulong Chen, Liangjun Zhu, Guibo Chen, Badong Zheng, Nanning Xi'an Jiaotong University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center of Visual Information and Applications Institute of Artificial Intelligence and Robotics Shaanxi Xi'an710049 China Institute of Automation Beijing China

Functional Magnetic Resonance Imaging (fMRI) presents challenges due to limited paired samples and low signal-to-noise ratios, particularly in tasks involving reconstructing natural images or decoding their semantic content. To address these challenges, we introduce BrainCLIP, an innovative fMRI-based brain decoding model. BrainCLIP leverages Contrastive Language-Image Pre-training's (CLIP) cross-modal generalization abilities to bridge brain activity, images, and text for the first time. Our experiments demonstrate CLIP's effectiveness in diverse brain decoding tasks, including zero-shot visual category decoding, fMRI-image/text alignment, and fMRI-to-image generation. The core objective of BrainCLIP is to train a mapping network that translates fMRI patterns into a unified CLIP embedding space, achieved through visual and textual supervision integration. Our experiments highlight that this approach significantly enhances performance in tasks such as fMRI-text alignment and fMRI-based image generation. Notably, BrainCLIP surpasses BraVL, a recent multi-modal method, in zero-shot visual category decoding. Moreover, BrainCLIP demonstrates strong capability in reconstructing visual stimuli with high semantic fidelity, competing favorably with state-of-the-art methods in capturing high-level semantic features during fMRI-based natural image reconstruction. © 1982-2012 IEEE.

关键词： Signal to noise ratio

来源：评论

学校读者我要写书评

暂无评论

Refiner: Fine-grained Cross-modal Concepts Refinement for Compositional Zero-Shot Learning

Refiner: Fine-grained Cross-modal Concepts Refinement for Co...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Xiao Zhang Haodong Jing Hui Chen Yongqiang Ma Nanning Zheng National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications and Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

关键词： visualization Zero shot learning Semantics Benchmark testing Signal processing Transformers Vectors Acoustics Speech processing Context modeling

来源：评论

学校读者我要写书评

暂无评论

Semantic Graph Embedded Energy Minimization Learning for Scene Graph Generation

Semantic Graph Embedded Energy Minimization Learning for Sce...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Jinghang Chen Chi Zhang Yuehu Liu Le Wang Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an China National Engineering Research Center for Visual Information and Applications National Key Laboratory of Human-Machine Hybrid Augmented Intelligence

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

The performance of current scene graph generation models is affected by training with cross-entropy loss, exacerbating the problem of prediction bias stemming from biased training data. Energy-based model adopts a learning method for joint image and scene graph to alleviate this challenge. However, this method only focuses on the visual features of images, neglecting the rich relation information contained in the semantic space. To address this issue, we innovatively employ powerful pre-trained large models to realize a simple yet effective semantic graph embedded energy minimization framework for the SGG task. Specifically, we use large models to generate image descriptions and extract relation triplets, which are then transformed into semantic graphs with entities as nodes and relations as edges. Moreover, by mapping these graphs into the same space using GNN to learn the minimal energy value, our approach enables SGG model to learn structural information in both semantic and visual spaces. We validate the effectiveness and efficiency of our method on the SGG benchmark visual Genome dataset. Compared with prevailing models and EBM, we achieve a significant performance improvement of up to 2.99% and 2.24%, respectively.

关键词： Training Learning systems visualization Image edge detection Semantics Training data Signal processing Predictive models Minimization Speech processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：