检索结果-内蒙古大学图书馆

2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025

作者： Liu, Guoshan Yin, Hailong Zhu, Bin Chen, Jingjing Ngo, Chong-Wah Jiang, Yu-Gang School of Computer Science Fudan University Shanghai Key Lab of Intelligent Information Processing China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Singapore Management University Singapore

ISBN: (纸本)9798331510831

The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method - first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs), which have achieved notable success across a variety of vision and language tasks, shed light on generating both ingredients and instructions directly from images. Nevertheless, LMMs still face the common issue of hallu-cinations during recipe generation, leading to suboptimal performance. To tackle this issue, we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement, integrating them into the prompt to add diverse and rich context to the input image. Additionally, Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates, which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method, which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset. © 2025 IEEE.

关键词： Food ingredients

来源：评论

学校读者我要写书评

暂无评论

Visual Prompt Flexible-Modal Face Anti-Spoofing

引用

IEEE Transactions on Dependable and Secure computing 2025年第3期22卷 2597-2606页

作者： Yu, Zitong Cai, Rizhao Cui, Yawen Liu, Ajian Chen, Changsheng Great Bay University School of Computing and Information Technology Dongguan523000 China Nanyang Technological University ROSE Lab School of EEE 639798 Singapore Hong Kong Polytechnic University Kowloon Hong Kong Chinese Academy of Sciences University of Chinese Academy of Sciences National Laboratory of Pattern Recognition Institute of Automation Beijing100190 China Shenzhen University Guangdong Key Laboratory of Intelligent Information Processing Shenzhen Key Laboratory of Media Security College of Electronics and Information Engineering Shenzhen518060 China

Recently, vision transformer based multimodal learning methods have been proposed to improve the robustness of face anti-spoofing (FAS) systems. However, multimodal face data collected from the real world is often imperfect due to missing modalities from various imaging sensors. Recently, flexible-modal FAS (Yu et al. 2023) has attracted more attention, which aims to develop a unified multimodal FAS model using complete multimodal face data but is insensitive to test-time missing modalities. In this paper, we tackle one main challenge in flexible-modal FAS, i.e., when missing modality occurs either during training or testing in real-world situations. Inspired by the recent success of the prompt learning in language models, we propose Visual Prompt flexible-modal FAS (VP-FAS), which learns the modal-relevant prompts to adapt the frozen pre-trained foundation model to downstream flexible-modal FAS task. Specifically, both vanilla visual prompts and residual contextual prompts are plugged into multimodal transformers to handle general missing-modality cases, while only requiring less than 4% learnable parameters compared to training the entire model. Furthermore, missing-modality regularization is proposed to force models to learn consistent multimodal feature embeddings when missing partial modalities. Extensive experiments conducted on two multimodal FAS benchmark datasets demonstrate the effectiveness of our VP-FAS framework that improves the performance under various missing-modality cases while alleviating the requirement of heavy model re-training. © 2004-2012 IEEE.

关键词： Alignment

来源：评论

学校读者我要写书评

暂无评论

Retrieval Augmented Recipe Generation

Retrieval Augmented Recipe Generation

引用

IEEE Workshop on Applications of Computer Vision (WACV)

作者： Guoshan Liu Hailong Yin Bin Zhu Jingjing Chen Chong-Wah Ngo Yu-Gang Jiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai Collaborative Innovation Center on Intelligent Visual Computing Singapore Management University

ISBN: (数字)9798331510831

ISBN: (纸本)9798331510848

The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method—first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs), which have achieved notable success across a variety of vision and language tasks, shed light on generating both ingredients and instructions directly from images. Nevertheless, LMMs still face the common issue of hallu-cinations during recipe generation, leading to suboptimal performance. To tackle this issue, we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement, integrating them into the prompt to add diverse and rich context to the input image. Additionally, Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates, which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method, which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset.

关键词： Training Computer vision Accuracy Computational modeling Stochastic processes Predictive models Reliability Faces

来源：评论

学校读者我要写书评

暂无评论

Predicting Calibrated Conversion Rate of Online Advertising Using a Multi-task Mixture-of-Experts Calibration Model 12th

Predicting Calibrated Conversion Rate of Online Advertising...

引用

12th CCF Conference on BigData, BigData 2024

作者： Zhang, Xinyue Guo, Yuyao Ao, Xiang School of Information Engineering China University of Geosciences Beijing China Beijing China Key Lab of Intelligent Information Processing Institute of Computing Technology CAS Beijing China

ISBN: (纸本)9789819610235

Accurately predicting conversion rate (CVR) is paramount in online advertising. However, traditional models may face problems such as delayed feedback, where there is a delay of an indeterminate amount of time between click and conversion. Calibration is an effective way to optimize conversion rate estimates in online advertising. Unlike conversion delays, post-click user behaviors occur rapidly and are informative to conversion rate prediction. Our proposed solution, the Multi-Task Mixture-of-Experts Calibration (MTMEC) framework, integrates multi-task learning and mixture-of-experts models. It modifies CVR prediction using post-click user behavior data, utilizing streaming learning for real-time data access. Each task is weighted by a gating network, enabling adaptive loss functions through multi-task learning. Parametric scaling further minimizes calibration errors, enhancing prediction accuracy without excessive parameters. Experiments on real-world datasets validate the effectiveness of MTMEC. It improves model prediction and reduces calibration errors. This framework offers a robust solution for online advertising, bridging the gap between calibration accuracy and system responsiveness. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Multi-task learning

来源：评论

学校读者我要写书评

暂无评论

Domain-aware Node Representation Learning for Graph Out-of-Distribution Generalization

Domain-aware Node Representation Learning for Graph Out-of-D...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Yi Qiao Yang Liu Qing He Xiang Ao Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciencs Beijing China Institute of Intelligent Computing Technology Suzhou CAS

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Graph Neural Networks (GNNs) have demonstrated impressive success across diverse fields when data satisfies in-distribution (ID) assumption. Nevertheless, GNN performance significantly declines in cases of distribution shifts between training and testing graph data. This degradation primarily stems from spurious correlations between irrelevant domain information and target labels in out-of-distribution (OOD) scenarios. Thus, maximizing the utilization of domain information becomes imperative. In light of this, we propose a novel approach named Domain-aware Node Representation Learning (DNRL), comprehensively incorporates domain information to bolster generalization capability. Specifically, DNRL selectively interpolates nodes with the same label but different domains, extending training data into unseen domains and alleviating the effects caused by domain-related spurious correlations. Futhermore, by introducing a domain-aware contrastive learning strategy, our method implicitly decouples domain information from node information to learn domain-independent node representations. Extensive experiments on graph out-of-distribution benchmarks demonstrate that DNRL can achieve effective OOD generalization performance across diverse domains.

关键词： Representation learning Training Degradation Interpolation Correlation Training data Signal processing Graph neural networks Data models Speech processing

来源：评论

学校读者我要写书评

暂无评论

CLIMS++: Cross Language Image Matching with Automatic Context Discovery for Weakly Supervised Semantic Segmentation

引用

International Journal of Computer Vision 2025年 1-20页

作者： Xie, Jinheng Deng, Songhe Hou, Xianxu Luo, Zhaochuan Shen, Linlin Huang, Yawen Zheng, Yefeng Shou, Mike Zheng Computer Vision Institute School of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China Department of Computer Science Wenzhou-Kean University Wenzhou325060 China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen China Show Lab National University of Singapore Singapore639798 Singapore School of AI and Advanced Computing Xi’an Jiaotong-Liverpool University Suzhou215400 China Jarvis Research Center Tencent YouTu Lab Shenzhen518703 China

While promising results have been achieved in weakly-supervised semantic segmentation (WSSS), limited supervision from image-level tags inevitably induces discriminative reliance and spurious relations between target classes and background regions. Thus, Class Activation Map (CAM) usually tends to activate discriminative object regions and falsely includes lots of class-related backgrounds. Without pixel-level supervisions, it could be very difficult to enlarge the foreground activation and suppress those false activation of background regions. In this paper, we propose a novel framework of Cross Language Image Matching with Automatic Context Discovery (CLIMS++), based on the recently introduced Contrastive Language-Image Pre-training (CLIP) model, for WSSS. The core idea of our framework is to introduce natural language supervision to activate more complete object regions and suppress class-related background regions in CAM. In particular, we design object, background region, and text label matching losses to guide the model to excite more reasonable object regions of each category. In addition, we propose to automatically find spurious relations between foreground categories and backgrounds, through which a background suppression loss is designed to suppress the activation of class-related backgrounds. The above designs enable the proposed CLIMS++ to generate a more complete and compact activation map for the target objects. Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 datasets show that our CLIMS++ significantly outperforms the previous state-of-the-art methods. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2025.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

A Flexible Knowledge Graph Error Detection Framework Combined with Semantic information 12th

A Flexible Knowledge Graph Error Detection Framework Combine...

引用

12th CCF Conference on BigData, BigData 2024

作者： Zhao, Yangwu Liu, Yang Ao, Xiang He, Qing Henan Institute of Advanced Technology Zhengzhou University Zhengzhou China Beijing China Key Lab of Intelligent Information Processing Institute of Computing Technology CAS Beijing China

ISBN: (纸本)9789819610235

Knowledge graphs (KGs) are extensively utilized in numerous applications, including question-answering systems and recommender systems. However, knowledge graphs are often constructed through web crawling or crowdsourcing, leading to errors in the data. The task of knowledge graph error detection aims to identify inaccurate triplets in KGs and has received substantial attention in recent years. However, the majority of current error detection methods overlook the semantic information of the triplets, which can be vital for accurate error detection. In this paper, we introduce a Flexible Knowledge Graph Error Detection Framework that integrates Semantic information (FKED), which combines both structural and semantic information to detect errors within the knowledge graph. FKED first extracts the structural information of KGs using a graph embedding model. Next, FKED employs a pre-trained language model (PLM) to extract semantic information from the triplets. Then the structural and semantic information are combined to detect errors within the knowledge graph. FKED can be flexibly added to other structure-based error detection models, enhancing their capabilities in downstream tasks of knowledge graphs. We assess FKED using two benchmark datasets: FB15k-237 and WN18RR. Experimental results indicate that our method is both superior and effective, substantially improving the performance of the base models. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow

arXiv

引用

arXiv 2025年

作者： Zhou, Hanyu Wang, Haonan Liu, Haoyue Duan, Yuxing Chang, Yi Yan, Luxin National Key Lab of Multispectral Information Intelligent Processing Technology School of Artificial Intelligence and Automation Huazhong University of Science and Technology China School of Computing National University of Singapore Singapore

High-dynamic scene optical flow is a challenging task, which suffers spatial blur and temporal discontinuous motion due to large displacement in frame imaging, thus deteriorating the spatiotemporal feature of optical flow. Typically, existing methods mainly introduce event camera to directly fuse the spatiotemporal features between the two modalities. However, this direct fusion is ineffective, since there exists a large gap due to the heterogeneous data representation between frame and event modalities. To address this issue, we explore a common-latent space as an intermediate bridge to mitigate the modality gap. In this work, we propose a novel common spatiotemporal fusion between frame and event modalities for high-dynamic scene optical flow, including visual boundary localization and motion correlation fusion. Specifically, in visual boundary localization, we figure out that frame and event share the similar spatiotemporal gradients, whose similarity distribution is consistent with the extracted boundary distribution. This motivates us to design the common spatiotemporal gradient to constrain the reference boundary localization. In motion correlation fusion, we discover that the frame-based motion possesses spatially dense but temporally discontinuous correlation, while the event-based motion has spatially sparse but temporally continuous correlation. This inspires us to use the reference boundary to guide the complementary motion knowledge fusion between the two modalities. Moreover, common spatiotemporal fusion can not only relieve the cross-modal feature discrepancy, but also make the fusion process interpretable for dense and continuous optical flow. Extensive experiments have been performed to verify the superiority of the proposed method. Copyright © 2025, The Authors. All rights reserved.

关键词： Optical flows

来源：评论

学校读者我要写书评

暂无评论

High-order diversity feature learning for pedestrian attribute recognition

引用

Neural Networks 2025年 188卷 107463页

作者： Wu, Junyi Huang, Yan Gao, Min Niu, Yuzhen Chen, Yuzhong Wu, Qiang Key Laboratory of Network Computing and Intelligent Information Processing College of Computer and Data Science Fuzhou University Fujian Fuzhou China Australian Artificial Intelligence Institute University of Technology Sydney NSW Australia Fujian Key Lab for Intelligent Processing and Wireless Transmission of Media Information College of Physics and Information Engineering Fuzhou University Fujian Fuzhou China School of Electrical and Data Engineering University of Technology Sydney NSW Australia

Pedestrian attribute recognition (PAR) involves accurately identifying multiple attributes present in pedestrian images. There are two main approaches for PAR: part-based method and attention-based method. The former relies on existing segmentation or region detection methods to localize body parts and learn corresponding attribute-specific feature from the corresponding regions, where the performance heavily depends on the accuracy of body region localization. The latter adopts the embedded attention modules or transformer attention to exploit detailed feature. However, it can focus on certain body regions but often provide coarse attention, failing to capture fine-grained details, the learned feature may also be interfered with by irrelevant information. Meanwhile, these methods overlook the global contextual information. This work argues for replacing coarse attention with detailed attention and integrating it with global contextual feature from ViT to jointly represent attribute-specific regions. To tackle this issue, we propose a High-order Diversity Feature Learning (HDFL) method for PAR based on ViT. We utilize a polynomial predictor to design an Attribute-specific Detailed Feature Exploration (ADFE) module, which can construct the high-order statistics and gain more fine-grained feature. Our ADFE module is a parameter-friendly method that provides flexibility in deciding its utilization during the inference phase. A Soft-redundancy Perception Loss (SPLoss) is proposed to adaptively measure the redundancy between feature of different orders, which can promote diverse characterization of features. Experiments on several PAR datasets show that our method achieves a new state-of-the-art (SOTA) performance. On the most challenging PA100K dataset, our method outperforms previous SOTA by 1.69% and achieves the highest mA of 84.92%. © 2025 The Authors

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

Improving Event-Level Financial Sentiment Analysis with Retrieval-Augmented Multipath Chain-of-Thought Prompting 12th

Improving Event-Level Financial Sentiment Analysis with Ret...

引用

12th CCF Conference on BigData, BigData 2024

作者： Zhang, Yiming Ao, Xiang Yu, Guoxin He, Qing Henan Institute of Advanced Technology Zhengzhou University Zhengzhou450002 China Beijing100190 China Key Lab of Intelligent Information Processing Institute of Computing Technology CAS Beijing100190 China

ISBN: (纸本)9789819610235

Event-level Financial Sentiment Analysis (EFSA) aims to extract all the quintuples containing five sentiment elements from a given financial news text, which has gained prominence as an emerging domain recently. The present study utilizes a 4-hop Chain-of-Thought (CoT) prompting based on LLMs to predict sentiment elements in a fixed order, which neglects the interdependencies among the sentiment elements within a quintuple. Inspired by recent multi-view prompting (MvP) and CoT ideas, we propose a novel framework termed Retrieval-Augmented Multipath Chain-of-Thought (RMP-CoT) that aggregates quintuples generated by LLMs through different reasoning paths, leveraging a retrieval-augmented mechanism. Specifically, RMP-CoT integrates different element orders into CoT prompting to guide LLMs in generating multiple sentiment quintuples through the utilization of retrieval-augmented mechanism, and then selects the most plausible quintuples by voting. To investigate the effectiveness of our framework, we conduct extensive experiments on four benchmark tasks of EFSA. RMP-CoT pushes the state-of-the-art by over 6% F1 on the EFSA task and also performs quite effectively on the other sub-tasks of EFSA. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Chain-of-Thought Event-level Sentiment Analysis Financial Sentiment Analysis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：