检索结果-内蒙古大学图书馆

Foundation Models Defining a New Era in Vision: A Survey and Outlook

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2025年第4期47卷 2245-2264页

作者： Awais, Muhammad Naseer, Muzammal Khan, Salman Anwer, Rao Muhammad Cholakkal, Hisham Shah, Mubarak Yang, Ming-Hsuan Khan, Fahad Shahbaz MBZ Univ AI Abu Dhabi U Arab Emirates Georgia Inst Technol Comp Sci Dept Atlanta GA 30332 USA Khalifa Univ Comp Sci Dept Abu Dhabi U Arab Emirates Khalifa Univ Ctr Secure Cyber Phys Secur Syst Abu Dhabi U Arab Emirates Australian Natl Univ CECS Canberra ACT 0200 Australia Univ Cent Florida Ctr Res Comp Vis Orlando FL 32816 USA Univ Calif Merced Merced CA 95344 USA Yonsei Univ Seoul 03722 South Korea Google Res Mountain View CA 94043 USA Linkoping Univ Comp Vis Lab S-58183 Linkoping Sweden

Vision systems that see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The complex relations between objects and their locations, ambiguities, and variations in the real-world environment can be better described in human language, naturally governed by grammatical rules and other modalities such as audio and depth. The models learned to bridge the gap between such modalities and large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. These models are referred to as foundation models. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions. In this survey, we provide a comprehensive review of such emerging foundation models, including typical architecture designs to combine different modalities (vision, text, audio, etc.), training objectives (contrastive, generative), pre-training datasets, fine-tuning mechanisms, and the common prompting patterns;textual, visual, and heterogeneous. We discuss the open challenges and research directions for foundation models in computer vision, including difficulties in their evaluations and benchmarking, gaps in their real-world understanding, limitations of contextual understanding, biases, vulnerability to adversarial attacks, and interpretability issues. We review recent developments in this field, covering a wide range of applications of foundation models systematically and comprehensively.

关键词： Adaptation models Computational modeling Foundation models data models Surveys visualization Reviews Computer vision Computer architecture Context modeling Contrastive learning language and vision large language models masked modeling self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Damage Presence, Localization, and Quantification of Aircraft Structure Based on End-to-End Deep Learning Framework

引用

ieee TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 2025年 74卷

作者： Chen, Gang Song, Wenrui Shao, Weihan Sun, Hu Qing, Xinlin Xiamen Univ Sch Aerosp Engn Xiamen 361005 Peoples R China Fuzhou Univ Maynooth Int Engn Coll Fuzhou 350108 Peoples R China

Structural health monitoring (SHM) plays a vital role in promptly identifying structural damage in aircraft, optimizing maintenance, and reducing costs;however, it faces significant challenges in practical applications, in that it mainly needs to process a large number of continuously collected sensor data, which are inevitably contaminated by random noise. This study, therefore, maps the relationship between Lamb wave signal data with noise and the health condition of aircraft structures using an end-to-end approach to construct a deep learning (DL) framework. The framework integrates deep residual convolutional networks (DRSNs) for feature extraction, efficient channel attention (ECA) for feature enhancement, and long short-term memory (LSTM) in analyzing time series data. Lamb wave signal datasets considering different damage locations and severity are obtained by lead zirconate titanate (PZT) sensors on the aircraft structure, and the datasets are destroyed by using multiple levels of Gaussian random noise to approximate the noise disturbances and unavoidable unpredictability of the industrial environment. The experimental data confirms that the performance metrics of the proposed framework for damage presence, localization, and quantification tasks are all above 97% in a noise-free environment. When dealing with high-noise scenes, the framework provides stronger antinoise robustness and higher accuracy compared to existing state-of-the-art (SOTA) methods. Quantitative analysis of ablation experiments and visualization of the t-distributed stochastic neighbor embedding (t-SNE) algorithm is applied to reveal the contribution of each component of the designed framework toward feature extraction and damage identification of Lamb wave signals in noisy environments.

关键词： Feature extraction Aircraft Location awareness Noise Training Monitoring Noise measurement data mining Atmospheric modeling Vibrations Attention mechanism end-to-end deep learning (DL) Lamb wave noise-robust structural health monitoring (SHM)

来源：评论

学校读者我要写书评

暂无评论

A Masked Reference Token Supervision-Based Iterative Visual-Language Framework for Robust Visual Grounding

引用

ieee TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2025年第1期35卷 75-90页

作者： Wang, Chunlei Feng, Wenquan Lyu, Shuchang Cheng, Guangliang Li, Xiangtai Liu, Binghao Zhao, Qi Beihang Univ Dept Elect & Informat Engn Beijing 100191 Peoples R China Univ Liverpool Dept Comp Sci Liverpool L7 8TX England Nanyang Technol Univ S Lab Singapore 639798 Singapore

Visual Grounding (VG) has become a prominent task in recent years, achieving significant advancements with the development of detection and vision transformers. However, existing VG methods struggle to handle the effects of inaccurate or irrelevant textual descriptions, tending to generate false-alarm objects. Moreover, existing methods fail to capture fine-grained features, accurate localization, and comprehensive context understanding from the whole image and textual descriptions. To address these issues, we propose an Iterative Robust Visual Grounding (IR-VG) framework with Multi-stage False-alarm Sensitive Decoder (MFSD) to prevent the generation of false-alarm objects when presented with inaccurate expressions. The framework introduces Masked Reference based Centerpoint Supervision (MRCS) and Iterative Multi-level Vision-language Fusion (IMVF) for enhancing the accuracy of localization and better visual-language alignment. To investigate the elements that affect VG robustness further, we release a robust VG benchmark with 24,000 instances and we also provide a detailed classification of false-alarm according to different parts of speech. Extensive experiments on existing state-of-the-art (SOTA) VG methods and foundation models have proven that it is difficult to handle the robustness of VG by existing models. Even foundation models, which have been pre-trained with a large amount of data, have difficulty to understand inaccurate language descriptions. Our IR-VG can handle false-alarm issues in robust VG well and achieve new SOTA results on the newly proposed robust VG datasets. Ablation studies and visualization experiments demonstrate the effectiveness of the proposed components. Moreover, the proposed framework is also verified effective on five regular VG datasets. Codes and models will be publicly at https://***/cv516Buaa/IR-VG.

关键词： visualization Grounding Task analysis Transformers Proposals Robustness Iterative methods Deep learning visual grounding robust learning visual language visual-linguistic alignment

来源：评论

学校读者我要写书评

暂无评论

Low-Overlap Point Cloud Registration via Correspondence Augmentation

引用

ieee TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2025年 22卷 9363-9375页

作者： Lin, Zhi-Huang Zhang, Chun-Yang Lin, Xue-Ming Lin, Huibin Zeng, Gui-Huang Chen, C. L. Philip Fuzhou Univ Coll Comp & Data Sci Fuzhou 350025 Peoples R China South China Univ Technol Sch Comp Sci & Engn Guangzhou 510006 Guangdong Peoples R China

Existing works have made some progress in point cloud registration, but most of them measure performance only on point cloud pairs with high overlap. In practical applications, it is often difficult to ensure that the collected point clouds overlap in large regions due to problems such as occlusion and noise. Therefore, a good low-overlap point cloud registration method is of great practical significance. However, extracting reliable correspondences from point clouds has always been a challenging task, particularly when dealing with low-overlap situation. In this paper, we propose a novel method for low-overlap point cloud registration via efficient correspondence augmentation, called AugLPCR, which not only enhances correspondences with high confidence, but also employs confidence weights to mitigate the impact of outliers. After the augmentation, the correspondences used for the transformation have a large amount of inliers, leading to improved registration performance. Extensive experiments on indoor and outdoor datasets demonstrate that the proposed AugLPCR is capable of maintaining consistent performance and achieve results comparable to or better than the state-of-the-art methods. Note to Practitioners-The motivation of this paper is to address the problem of registering two low-overlap point clouds. Mainstream algorithms for point cloud registration typically assume a sufficient overlap between point clouds. However, in practical scenarios, it is common to encounter scans with inadequate overlap. These conditions often hinder the extraction of reliable correspondences. This paper introduces an effective method for augmenting correspondences to address the problem of low inlier rates within predicted correspondences. While augmenting correspondences with high confidence, it also mitigates the influence of outliers and ambiguous points. Additionally, traditional approaches often divide superpoint regions before matching, but this can lead to the elimination of

关键词： Point cloud compression Feature extraction Three-dimensional displays Accuracy Convolution Transforms Reflection Iterative methods Image color analysis Estimation Point cloud registration correspondence augmentation point cloud visualization

来源：评论

学校读者我要写书评

暂无评论

Homeomorphism Prior for False Positive and Negative Problem in Medical Image Dense Contrastive Representation Learning

引用

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2025年第5期47卷 4122-4139页

作者： He, Yuting Wang, Boyu Ge, Rongjun Chen, Yang Yang, Guanyu Li, Shuo Southeast Univ Key Lab New Generat Artificial Intelligence Techno Minist Educ Nanjing Peoples R China Case Western Reserve Univ Dept Biomed Engn Cleveland OH 44106 USA Western Univ Dept Comp Sci London ON N6A 3K7 Canada Southeast Univ Sch Instrument Sci & Engn Nanjing 210018 Peoples R China Ctr Rech en Informat Biomedicale Sino Francais CRI Nanjing 210024 Peoples R China Jiangsu Prov Joint Int Res Lab Med Informat Proc Nanjing 210024 Peoples R China Case Western Reserve Univ Dept Biomed Engn Dept Comp & Data Sci Cleveland OH 44106 USA

Dense contrastive representation learning (DCRL) has greatly improved the learning efficiency for image dense prediction tasks, showing its great potential to reduce the large costs of medical image collection and dense annotation. However, the properties of medical images make unreliable correspondence discovery, bringing an open problem of large-scale false positive and negative (FP&N) pairs in DCRL. In this paper, we propose GEoMetric vIsual deNse sImilarity (GEMINI) learning which embeds the homeomorphism prior to DCRL and enables a reliable correspondence discovery for effective dense contrast. We proposes a deformable homeomorphism learning (DHL) which models the homeomorphism of medical images and learns to estimate a deformable mapping to predict the pixels' correspondence under the condition of topological preservation. It effectively reduces the searching space of pairing and drives an implicit and soft learning of negative pairs via gradient. We also proposes a geometric semantic similarity (GSS) which extracts semantic information in features to measure the alignment degree for the correspondence learning. It will promote the learning efficiency and performance of deformation, constructing positive pairs reliably. We implement two practical variants on two typical representation learning tasks in our experiments. Our promising results on seven datasets which outperform the existing methods show our great superiority. We will release our code at a companion website.

关键词： Biomedical imaging Semantics Reliability Topology Feature extraction Representation learning Deformation Computed tomography data mining visualization Medical image analysis dense contrastive representation learning false positive and negative pairs problem homeomorphism prior correspondence problem

来源：评论

学校读者我要写书评

暂无评论

From Bag-of-Words to Transformers: A Comparative Study for Text Classification in Healthcare Discussions in Social Media

引用

ieee TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE 2025年第1期9卷 1063-1077页

作者： De Santis, Enrico Martino, Alessio Ronci, Francesca Rizzi, Antonello Univ Roma La Sapienza Dept Informat Engn Elect & Telecommun I-00184 Rome Italy LUISS Univ Dept Business & Management Rome Italy

One notable paradigm shift in Natural Language Processing has been the introduction of Transformers, revolutionizing language modeling as Convolutional Neural Networks did for Computer Vision. The power of Transformers, along with many other innovative features, also lies in the integration of word embedding techniques, traditionally used to represent words in a text and to build classification systems directly. This study delves into the comparison of text representation techniques for classifying users who generate medical topic posts on Facebook discussion groups. Short and noisy social media texts in Italian pose challenges for user categorization. The study employs two datasets, one for word embedding model estimation and another comprising discussions from users. The main objective is to achieve optimal user categorization through different pre-processing and embedding techniques, aiming at high generalization performance despite class imbalance. The paper has a dual purpose, i.e., to build an effective classifier, ensuring accurate information dissemination in medical discussions and combating fake news, and to explore also the representational capabilities of various LLMs, especially concerning BERT, Mistral and GPT-4. The latter is investigated using the in-context learning approach. Finally, data visualization tools are used to evaluate the semantic embeddings with respect to the achieved performance. This investigation, focusing on classification performance, compares the classic BERT and several hybrid versions (i.e., employing different training strategies and approximate Support Vector Machines in the classification layer) against LLMs and several Bag-of-Words based embedding (notably, one of the earliest approaches in text classification). This research offers insights into the latest developments in language modeling, advancing in the field of text representation and its practical application for user classification within medical discussions.

关键词： Social networking (online) Transformers Semantics Vectors Training Task analysis Support vector machines Embedding techniques healthcare natural language processing social network analysis text categorization text mining transformers mistral GPT-4 large language models

来源：评论

学校读者我要写书评

暂无评论

MVR: Synergizing large and Vision Transformer for Multimodal Natural Language-Driven Vehicle Retrieval

引用

ieee ACCESS 2025年 13卷 7014-7026页

作者： Alzubi, Tareq Mahmod Mukhtar, Umar Raza Al Balqa Appl Univ Prince Abdullah bin Ghazi Fac Informat & Commun Te Dept Comp Sci As Salt 19117 Jordan Univ Management & Technol Dept Comp Sci Lahore 54770 Pakistan

In recent years, intelligent transportation systems have played a pivotal role in the development of smart cities, with vehicle retrieval becoming a critical component of traffic management and surveillance. Traditional vehicle retrieval systems rely heavily on image-based matching techniques derived from vehicle re-identification (VReID) tasks. However, these approaches are limited by their dependency on image queries, which may not always be available in real-world scenarios. Natural language (NL)-based vehicle retrieval systems offer a more flexible and accessible alternative by enabling users to query vehicles using textual descriptions. Despite progress in NL-based retrieval, existing methods face challenges in fully capturing multi-granularity information and aligning heterogeneous visual and linguistic inputs. This paper addresses these limitations by proposing a robust Multimodal Vehicle Retrieval (MVR) model that integrates both visual and textual data through a dual-stream architecture. Our model captures complementary local features, alongside global information including motion and environmental context. We utilize InfoNCE and instance losses to align the visual and textual modalities within a shared feature space, while post-processing modules, including Granular Vehicle Feature Refinement and Spatial Relationship Modeling, further enhance retrieval performance by refining vehicle attributes and contextual relationships. Our experiments, conducted on the CityFlow-NL dataset, demonstrate that our model achieves a 35.6% improvement in Mean Reciprocal Rank (MRR), a 41.3% increase in recall at 5 (R@5), and a 22.9% improvement in recall at 10 (R@10) compared to the baseline, and overcomes the inherent challenges of cross-modal retrieval in improving real-world VReID.

关键词： visualization Feature extraction Videos Transformers Smart cities Vehicle dynamics Traffic control Streaming media Image color analysis Cameras Traffic surveillance intelligent transportation multimodal smart city vehicle tracking

来源：评论

学校读者我要写书评

暂无评论

Revisiting Viewing Graph Solvability: An Effective Approach Based on Cycle Consistency

引用

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2025年第5期47卷 3271-3284页

作者： Arrigoni, Federica Fusiello, Andrea Rizzi, Romeo Ricci, Elisa Pajdla, Tomas Politecn Milan Dipartimento Elettron Informaz & Bioingn DEIB I-20133 Milan Italy Univ Udine Polytech Dept Engn & Architecture DPIA I-33100 Udine Italy Univ Verona Dept Comp Sci I-37129 Verona Italy Univ Trento Dept Informat Engn & Comp Sci DISI I-38122 Trento Italy Fdn Bruno Kessler FBK Deep Visual Learning Grp I-38121 Trento Italy Czech Tech Univ Czech Inst Informat Robot & Cybernet CIIRC Prague 16000 Czech Republic

In the structure from motion, the viewing graph is a graph where the vertices correspond to cameras (or images) and the edges represent the fundamental matrices. We provide a new formulation and an algorithm for determining whether a viewing graph is solvable, i.e., uniquely determines a set of projective cameras. The known theoretical conditions either do not fully characterize the solvability of all viewing graphs, or are extremely difficult to compute because they involve solving a system of polynomial equations with a large number of unknowns. The main result of this paper is a method to reduce the number of unknowns by exploiting cycle consistency. We advance the understanding of solvability by (i) finishing the classification of all minimal graphs up to 9 nodes, (ii) extending the practical verification of solvability to minimal graphs with up to 90 nodes, (iii) finally answering an open research question by showing that finite solvability is not equivalent to solvability, and (iv) formally drawing the connection with the calibrated case (i.e., parallel rigidity). Finally, we present an experiment on real data that shows that unsolvable graphs may appear in practice.

关键词： Cameras Rigidity Three-dimensional displays Image reconstruction visualization Image edge detection Topology Solvability viewing graph structure from motion fundamental matrix uncalibrated camera

来源：评论

学校读者我要写书评

暂无评论

ProtoCLIP: Prototypical Contrastive Language Image Pretraining

引用

ieee TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2025年第1期36卷 610-624页

作者： Chen, Delong Wu, Zhao Liu, Fan Yang, Zaiquan Zheng, Shaoqiu Tan, Ying Zhou, Erjin Hohai Univ Coll Comp & Informat Nanjing 210000 Peoples R China Hong Kong Univ Sci & Technol Ctr Artificial Intelligence Res CAiRE Hong Kong Peoples R China MEGVII Res Beijing 100000 Peoples R China Sci & Technol Underwater Vehicle Technol Lab Harbin 150001 Peoples R China Nanjing Res Inst Elect Engn Nanjing 210023 Peoples R China Peking Univ Sch EECS Dept Machine Intelligence Beijing 100871 Peoples R China

Contrastive language image pretraining (CLIP) has received widespread attention since its learned representations can be transferred well to various downstream tasks. During the training process of the CLIP model, the InfoNCE objective aligns positive image-text pairs and separates negative ones. We show an underlying representation grouping effect during this process: the InfoNCE objective indirectly groups semantically similar representations together via randomly emerged within-modal anchors. Based on this understanding, in this article, prototypical contrastive language image pretraining (ProtoCLIP) is introduced to enhance such grouping by boosting its efficiency and increasing its robustness against the modality gap. Specifically, ProtoCLIP sets up prototype-level discrimination between image and text spaces, which efficiently transfers higher level structural knowledge. Furthermore, prototypical back translation (PBT) is proposed to decouple representation grouping from representation alignment, resulting in effective learning of meaningful representations under a large modality gap. The PBT also enables us to introduce additional external teachers with richer prior language knowledge. ProtoCLIP is trained with an online episodic training strategy, which means it can be scaled up to unlimited amounts of data. We trained our ProtoCLIP on conceptual captions (CCs) and achieved an +5.81% ImageNet linear probing improvement and an +2.01% ImageNet zero-shot classification improvement. On the larger YFCC-15M dataset, ProtoCLIP matches the performance of CLIP with 33% of training time.

关键词： Training Prototypes visualization Task analysis Semantics Fans Underwater vehicles Contrastive learning K-means clustering multimodal representation learning self-supervised learning (SSL) vision language pretraining

来源：评论

学校读者我要写书评

暂无评论

Revisiting Reliability in large-Scale Machine Learning Research Clusters 31

Revisiting Reliability in Large-Scale Machine Learning Resea...

引用

31st ieee International symposium on High Performance Computer Architecture, HPCA 2025

作者： Kokolis, Apostolos Kuchnik, Michael Hoffman, John Kumar, Adithya Malani, Parth Ma, Faye Devito, Zachary Sengupta, Shubho Saladi, Kalyan Wu, Carole-Jean FAIR at Meta United States

ISBN: (纸本)9798331506476

Reliability is a fundamental challenge in operating large-scale machine learning (ML) infrastructures, particularly as the scale of ML models and training clusters continues to grow. Despite decades of research on infrastructure failures, the impact of job failures across different scales remains unclear. This paper presents a view of managing two large, multi-tenant ML clusters, providing quantitative analysis, operational experience, and our own perspective in understanding and addressing reliability concerns at scale. Our analysis reveals that while large jobs are most vulnerable to failures, smaller jobs make up the majority of jobs in the clusters and should be incorporated into optimization objectives. We identify key workload properties, compare them across clusters, and demonstrate essential reliability requirements for pushing the boundaries of ML training at *** hereby introduce a taxonomy of failures and key reliability metrics, analyze 11 months of data from two state-of-the-art ML environments with 4 million jobs and over 150 million A100 GPU hours. Building on our data, we fit a failure model to project Mean Time to Failure for various GPU scales. We further propose a method to estimate a related metric, Effective Training Time Ratio, as a function of job parameters, and we use this model to gauge the efficacy of potential software mitigations at scale. Our work provides valuable insights and future research directions for improving the reliability of AI supercomputer clusters, emphasizing the need for flexible, workload-agnostic, and reliability-aware infrastructure, system software, and algorithms. © 2025 ieee.

关键词： Program processors

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：