检索结果-内蒙古大学图书馆

CLIMS++: Cross Language Image Matching with Automatic Context Discovery for Weakly Supervised Semantic Segmentation

International Journal of Computer Vision 2025年 1-20页

作者： Xie, Jinheng Deng, Songhe Hou, Xianxu Luo, Zhaochuan Shen, Linlin Huang, Yawen Zheng, Yefeng Shou, Mike Zheng Computer Vision Institute School of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China Department of Computer Science Wenzhou-Kean University Wenzhou325060 China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen China Show Lab National University of Singapore Singapore639798 Singapore School of AI and Advanced Computing Xi’an Jiaotong-Liverpool University Suzhou215400 China Jarvis Research Center Tencent YouTu Lab Shenzhen518703 China

While promising results have been achieved in weakly-supervised semantic segmentation (WSSS), limited supervision from image-level tags inevitably induces discriminative reliance and spurious relations between target classes and background regions. Thus, Class Activation Map (CAM) usually tends to activate discriminative object regions and falsely includes lots of class-related backgrounds. Without pixel-level supervisions, it could be very difficult to enlarge the foreground activation and suppress those false activation of background regions. In this paper, we propose a novel framework of Cross Language Image Matching with Automatic Context Discovery (CLIMS++), based on the recently introduced Contrastive Language-Image Pre-training (CLIP) model, for WSSS. The core idea of our framework is to introduce natural language supervision to activate more complete object regions and suppress class-related background regions in CAM. In particular, we design object, background region, and text label matching losses to guide the model to excite more reasonable object regions of each category. In addition, we propose to automatically find spurious relations between foreground categories and backgrounds, through which a background suppression loss is designed to suppress the activation of class-related backgrounds. The above designs enable the proposed CLIMS++ to generate a more complete and compact activation map for the target objects. Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 datasets show that our CLIMS++ significantly outperforms the previous state-of-the-art methods. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2025.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow

arXiv

引用

arXiv 2025年

作者： Zhou, Hanyu Wang, Haonan Liu, Haoyue Duan, Yuxing Chang, Yi Yan, Luxin National Key Lab of Multispectral Information Intelligent Processing Technology School of Artificial Intelligence and Automation Huazhong University of Science and Technology China School of Computing National University of Singapore Singapore

High-dynamic scene optical flow is a challenging task, which suffers spatial blur and temporal discontinuous motion due to large displacement in frame imaging, thus deteriorating the spatiotemporal feature of optical flow. Typically, existing methods mainly introduce event camera to directly fuse the spatiotemporal features between the two modalities. However, this direct fusion is ineffective, since there exists a large gap due to the heterogeneous data representation between frame and event modalities. To address this issue, we explore a common-latent space as an intermediate bridge to mitigate the modality gap. In this work, we propose a novel common spatiotemporal fusion between frame and event modalities for high-dynamic scene optical flow, including visual boundary localization and motion correlation fusion. Specifically, in visual boundary localization, we figure out that frame and event share the similar spatiotemporal gradients, whose similarity distribution is consistent with the extracted boundary distribution. This motivates us to design the common spatiotemporal gradient to constrain the reference boundary localization. In motion correlation fusion, we discover that the frame-based motion possesses spatially dense but temporally discontinuous correlation, while the event-based motion has spatially sparse but temporally continuous correlation. This inspires us to use the reference boundary to guide the complementary motion knowledge fusion between the two modalities. Moreover, common spatiotemporal fusion can not only relieve the cross-modal feature discrepancy, but also make the fusion process interpretable for dense and continuous optical flow. Extensive experiments have been performed to verify the superiority of the proposed method. Copyright © 2025, The Authors. All rights reserved.

关键词： Optical flows

来源：评论

学校读者我要写书评

暂无评论

FLIP-80M: 80 Million Visual-Linguistic Pairs for Facial Language-Image Pre-Training 24

FLIP-80M: 80 Million Visual-Linguistic Pairs for Facial Lang...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Li, Yudong Hou, Xianxu Dezhi, Zheng Shen, Linlin Zhao, Zhe School of Computer Science and Software Engineering Shenzhen University Shenzhen China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China School of AI and Advanced Computing Xi'an Jiaotong-Liverpool University Shenzhen China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen China Tencent AI Lab Beijing China

ISBN: (纸本)9798400706868

While significant progress has been made in multi-modal learning driven by large-scale image-text datasets, there is still a noticeable gap in the availability of such datasets within the facial domain. To facilitate and advance the field of facial representation learning, we present FLIP-80M, a large-scale visual-linguistic dataset comprising over 80 million face images paired with text descriptions. FLIP-80M is constructed by leveraging the large openly available image-text-pair dataset LAION-5B and a mixed-method approach to filter face-related pairs from both visual and linguistic perspectives. Our curation process involves face detection, face caption classification, text de-noising, and synthesis-based image augmentation. As a result, FLIP-80M stands as the largest face-text dataset to date. To evaluate the potential of our dataset, we fine-tune the CLIP model using the proposed FLIP-80M, to create FLIP (Facial Language-Image Pretraining) and assess its representation capabilities across various downstream tasks. Our experiments demonstrate that our FLIP model achieves state-of-the-art results in a range of face analysis tasks, including face parsing, face alignment, and face attribute classification. The dataset and models are available at https://***/ydli-ai/FLIP. © 2024 ACM.

关键词： Face recognition

来源：评论

学校读者我要写书评

暂无评论

HD-KT: Advancing Robust Knowledge Tracing via Anomalous Learning Interaction Detection 24

HD-KT: Advancing Robust Knowledge Tracing via Anomalous Lear...

引用

33rd ACM Web Conference, WWW 2024

作者： Ma, Haiping Yang, Yong Qin, Chuan Yu, Xiaoshan Yang, Shangshang Zhang, Xingyi Zhu, Hengshu Information Materials and Intelligent Sensing Laboratory of Anhui Province Institutes of Physical Science and Information Technology Anhui University Anhui Hefei China Institutes of Physical Science and Information Technology Anhui University Anhui Hefei China Career Science Lab BOSS Zhipin PBC School of Finance Tsinghua University Beijing China School of Artificial Intelligence Anhui University Anhui Hefei China The Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education School of Computer Science and Technology Anhui University Anhui Hefei China Career Science Lab BOSS Zhipin Beijing China

ISBN: (纸本)9798400701719

Knowledge tracing (KT) is a crucial task in online learning, aimed at tracing and predicting each student's knowledge states throughout their learning process. Over the past decade, it has garnered widespread attention due to it provides the potential for more tailored and adaptive online learning experiences. Although most current KT methodologies emphasize optimizing network structures to enhance predictive accuracy for future student performance, they often neglect anomalous interactions in students' learning processes, which may arise from low data quality (i.e., inferior question quality) and abnormal student behaviors (i.e., guessing and mistakes). To this end, in this paper, we propose a novel framework, termed HD-KT, designed to enhance the robustness of existing KT methodologies with Hybrid learning interactions Denoising approach. Specifically, we introduce two detectors for anomalous learning interactions, namely knowledge state-guided anomaly detector and student profile-guided anomaly detector. In the first detection module, we design a sequential autoencoder to identify anomalous learning interactions by detecting atypical student knowledge states. In the second module, we incorporate an attention mechanism by modeling a student's long-term profile to capture irregular interactions. Extensive experiments on four real-world benchmark datasets have decisively shown our HD-KT markedly boosts the robustness of numerous prevailing KT models, consequently increasing the accuracy of future student performance predictions. Additionally, our case studies highlight the versatility of HD-KT in addressing diverse downstream tasks, such as exercise quality analysis and learning behavior-based student clustering. © 2024 ACM.

关键词： Students

来源：评论

学校读者我要写书评

暂无评论

Improving Event-Level Financial Sentiment Analysis with Retrieval-Augmented Multipath Chain-of-Thought Prompting 12th

Improving Event-Level Financial Sentiment Analysis with Ret...

引用

12th CCF Conference on BigData, BigData 2024

作者： Zhang, Yiming Ao, Xiang Yu, Guoxin He, Qing Henan Institute of Advanced Technology Zhengzhou University Zhengzhou450002 China Beijing100190 China Key Lab of Intelligent Information Processing Institute of Computing Technology CAS Beijing100190 China

ISBN: (纸本)9789819610235

Event-level Financial Sentiment Analysis (EFSA) aims to extract all the quintuples containing five sentiment elements from a given financial news text, which has gained prominence as an emerging domain recently. The present study utilizes a 4-hop Chain-of-Thought (CoT) prompting based on LLMs to predict sentiment elements in a fixed order, which neglects the interdependencies among the sentiment elements within a quintuple. Inspired by recent multi-view prompting (MvP) and CoT ideas, we propose a novel framework termed Retrieval-Augmented Multipath Chain-of-Thought (RMP-CoT) that aggregates quintuples generated by LLMs through different reasoning paths, leveraging a retrieval-augmented mechanism. Specifically, RMP-CoT integrates different element orders into CoT prompting to guide LLMs in generating multiple sentiment quintuples through the utilization of retrieval-augmented mechanism, and then selects the most plausible quintuples by voting. To investigate the effectiveness of our framework, we conduct extensive experiments on four benchmark tasks of EFSA. RMP-CoT pushes the state-of-the-art by over 6% F1 on the EFSA task and also performs quite effectively on the other sub-tasks of EFSA. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Chain-of-Thought Event-level Sentiment Analysis Financial Sentiment Analysis

来源：评论

学校读者我要写书评

暂无评论

Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning

Masked Video Distillation: Rethinking Masked Feature Modelin...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Mengchen Liu Lu Yuan Yu–Gang Jiang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing Microsoft Cloud + AI

Benefiting from masked visual modeling, self-supervised video representation learning has achieved remarkable progress. However, existing methods focus on learning representations from scratch through reconstructing low-level features like raw pixel values. In this paper, we propose masked video distillation (MVD), a simple yet effective two-stage masked feature modeling framework for video representation learning: firstly we pretrain an image (or video) model by recovering low-level features of masked patches, then we use the resulting features as targets for masked feature modeling. For the choice of teacher models, we observe that students taught by video teachers perform better on temporally-heavy video tasks, while image teachers transfer stronger spatial representations for spatially-heavy video tasks. Visualization analysis also indicates different teachers produce different learned patterns for students. To leverage the advantage of different teachers, we design a spatial-temporal co-teaching method for MVD. Specifically, we distill student models from both video teachers and image teachers by masked feature modeling. Extensive experimental results demonstrate that video transformers pre-trained with spatial-temporal co-teaching outperform models distilled with a single teacher on a multitude of video datasets. Our MVD with vanilla ViT achieves state-of-the-art performance compared with previous methods on several challenging video downstream tasks. For example, with the ViT-Large model, our MVD achieves 86.4% and 76.7% Top-1 accuracy on Kinetics-400 and Something-Something-v2, outperforming VideoMAE by 1.2% and 2.4% respectively. When a larger ViT-Huge model is adopted, MVD achieves the state-of-the-art performance with 77.3% Top-1 accuracy on Something-Something-v2. Code will be available at https://***/ruiwang2021/mvd.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Protein Structure Prediction:Challenges,Advances,and the Shift of Research Paradigms

引用

Genomics, Proteomics & Bioinformatics 2023年第5期21卷 913-925页

作者： Bin Huang Lupeng Kong Chao Wang Fusong Ju Qi Zhang Jianwei Zhu Tiansu Gong Haicang Zhang Chungong Yu Wei-Mou Zheng Dongbo Bu Key Laboratory of Intelligent Information Processing Institute of Computing TechnologyChinese Academy of SciencesBeijing 100190China University of Chinese Academy of Sciences Beijing 100049China Changping Laboratory Beijing 102206China Microsoft Research AI4Science Beijing 100080China Huawei Noah’s Ark Lab Wuhan 430206China Zhongke Big Data Academy Zhengzhou 450046China Institute of Theoretical Physics Chinese Academy of SciencesBeijing 100190China

Protein structure prediction is an interdisciplinary research topic that has attracted researchers from multiple fields,including biochemistry,medicine,physics,mathematics,and computer *** researchers adopt various research paradigms to attack the same structure prediction problem:biochemists and physicists attempt to reveal the principles governing protein folding;mathematicians,especially statisticians,usually start from assuming a probability distribution of protein structures given a target sequence and then find the most likely structure,while computer scientists formulate protein structure prediction as an optimization problem-finding the structural conformation with the lowest energy or minimizing the difference between predicted structure and native *** research paradigms fall into the two statistical modeling cultures proposed by Leo Breiman,namely,data modeling and algorithmic ***,we have also witnessed the great success of deep learning in protein structure *** this review,we present a survey of the efforts for protein structure *** compare the research paradigms adopted by researchers from different fields,with an emphasis on the shift of research paradigms in the era of deep *** short,the algorithmic modeling techniques,especially deep neural networks,have considerably improved the accuracy of protein structure prediction;however,theories interpreting the neural networks and knowledge on protein folding are still highly desired.

关键词： Protein folding Protein structure prediction Deep learning Transformer Language model

来源：评论

学校读者我要写书评

暂无评论

Weighted ROC curve in cost space: extending AUC to cost-sensitive learning 23

Weighted ROC curve in cost space: extending AUC to cost-sens...

引用

Proceedings of the 37th International Conference on Neural information processing Systems

作者： Huiyang Shao Qianqian Xu Zhiyong Yang Peisong Wen Peifeng Gao Qingming Huang Key Lab. of Intelligent Information Processing Institute of Computing Tech. CAS and School of Computer Science and Tech. University of Chinese Academy of Sciences Key Lab. of Intelligent Information Processing Institute of Computing Tech. CAS School of Computer Science and Tech. University of Chinese Academy of Sciences School of Computer Science and Tech. University of Chinese Academy of Sciences and Key Lab. of Intelligent Information Processing Institute of Computing Tech. CAS and BDKM University of Chinese Academy of Sciences

In this paper, we aim to tackle flexible cost requirements for long-tail datasets, where we need to construct a (1) cost-sensitive and (2) class-distribution robust learning framework. The misclassification cost and the area under the ROC curve (AUC) are popular metrics for (1) and (2), respectively. However, limited by their formulations, models trained with AUC are not well-suited for cost-sensitive decision problems, and models trained with fixed costs are sensitive to the class distribution shift. To address this issue, we present a new setting where costs are treated like a dataset to deal with arbitrarily unknown cost distributions. Moreover, we propose a novel weighted version of AUC where the cost distribution can be integrated into its calculation through decision thresholds. To formulate this setting, we propose a novel bilevel paradigm to bridge weighted AUC (WAUC) and cost. The inner-level problem approximates the optimal threshold from sampling costs, and the outer-level problem minimizes the WAUC loss over the optimal threshold distribution. To optimize this bilevel paradigm, we employ a stochastic optimization algorithm (SACCL) which enjoys the same convergence rate (O(ε-4)) with the SGD. Finally, experiment results show that our algorithm performs better than existing cost-sensitive learning methods and two-stage AUC decisions approach.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques 38

Suppress Content Shift: Better Diffusion Features via Off-th...

引用

38th Conference on Neural information processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features 38

Not All Diffusion Model Activations Have Been Evaluated as D...

引用

38th Conference on Neural information processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：