Margin-aware Noise-robust Contrastive Learning for Partially View-aligned Problem

作     者：Qin, Yalan Pu, Nan Wu, Hanzhou Sebe, Nicu 

作者机构：Shanghai Univ Shanghai Peoples R China Univ Trento Trento Italy 

出 版 物：《ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA》 (ACM Trans. Knowl. Discov. Data)

年 卷 期：2025年第19卷第1期

页      面：1-20页

核心收录：

学科分类：08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：European Union China Postdoctoral Science Foundation [2023M742208] National Natural Science Foundation of China [62402303, U23B2023] Science and Technology Commission of Shanghai Municipality (STCSM) [24ZR1424000] The 2024 Xizang Autonomous Region Central Guided Local Science and Technology Development Fund Project [XZ202401YD0015] 

主　　题：Computing methodologies Machine learning 

摘      要：In this article, we study a challenging problem in contrastive learning when just a portion of data is aligned in multi-view dataset due to temporal, spatial, or spatio-temporal asynchronism across views. It is important to study partially view-aligned data since this type of data is common in real-world application and easily leads to data inconsistency among different views. Such a Partially View-aligned Problem (PVP) in contrastive learning has been relatively less touched so far, especially in downstream tasks, i.e., classification and clustering. In order to solve this problem, we introduce a flexible margin and propose margin-aware noise-robust contrastive learning to simultaneously identify the within-category counterparts from the other view of one data point based on the established cross-view correspondence and learn a shared representation. To be specific, the proposed learning framework is built on a novel margin-aware noise-robust contrastive loss. Since data pairs are used as input for the proposed margin-aware noise-robust contrastive learning, we build positive pairs according to the known correspondences and negative pairs in the manner of random sampling. Our margin-aware noise-robust contrastive learning framework is able to effectively reduce or remove the impacts caused by the possible existing noise for the constructed pairs in a margin-aware manner, i.e., false negative pairs led by random sampling in PVP. We relax the proposed margin-aware noise-robust contrastive loss and then give a detailed mathematical analysis for the effectiveness of our loss. As an instantiation, we construct an example under the proposed margin-aware noise-robust contrastive learning framework for validation in this work. To the best of our knowledge, this is the first attempt of extending contrastive learning to a margin-aware noise-robust version for dealing with PVP. We also enrich the learning paradigm when there is noise in the data. Extensive experiments on diff