检索结果-内蒙古大学图书馆

UCPM: Uncertainty-Guided Cross-Modal Retrieval with Partially Mismatched Pairs

IEEE Transactions on Image Processing 2025年 34卷 3622-3634页

作者： Zha, Quanxing Liu, Xin Cheung, Yiu-Ming Peng, Shu-Juan Xu, Xing Wang, Nannan Huaqiao University Department of Computer Science Xiamen 361021 China Key Laboratory of Pattern Recognition and Computer Vision Xiamen 361021 China Huaqiao University Fujian Key Laboratory of Big Data Intelligence and Security Xiamen 361021 China Hong Kong Baptist University Department of Computer Science Hong Kong Huaqiao University Department of Artificial Intelligence Xiamen 361021 China Fujian Province University Key Laboratory of Computer Vision and Machine Learning (Huaqiao University) Xiamen 361021 China University of Electronic Science and Technology of China Center for Future Multimedia School of Computer Science and Engineering Chengdu 610051 China Xidian University State Key Laboratory of Integrated Services Networks Xi’an 710071 China

The manual annotation of perfectly aligned labels for cross-modal retrieval (CMR) is incredibly labor-intensive. As an alternative, the collection of co-occurring data pairs from the Internet is a remarkably cost-effective way, but which, inevitably induces the Partially Mismatched Pairs (PMPs) and therefore significantly degrades the retrieval performance without particular treatment. Previous efforts often utilize the pair-wise similarity to filter out the mismatched pairs, and such operation is highly sensitive to mismatched or ambiguous data and thus leads to sub-optimal performance. To alleviate these concerns, we propose an efficient approach, termed UCPM, i.e., Uncertainty-guided Cross-modal retrieval with Partially Mismatched pairs, which can significantly reduce the adverse impact of mismatched data pairs. Specifically, a novel Uncertainty Guided Division (UGD) strategy is sophisticatedly designed to divide the corrupted training data into confident matched (clean), easily-identifiable mismatched (noisy) and hardly-determined hard subsets, and the derived uncertainty can simultaneously guide the informative pair learning while reducing the negative impact of potential mismatched pairs. Meanwhile, an effective Uncertainty Self-Correction (USC) mechanism is concurrently presented to accurately identify and rectify the fluctuated uncertainty during the training process, which further improves the stability and reliability of the estimated uncertainty. Besides, a Trusted Margin Loss (TML) is newly designed to enhance the discriminability between those hard pairs, by dynamically adjusting their soft margins to amplify the positive contributions of matched pairs while suppressing the negative impacts of mismatched pairs. Extensive experiments on three widely-used benchmark datasets, verify the effectiveness and reliability of UCPM compared with the existing SOTA approaches, and significantly improve the robustness in both synthetic and real-world PMPs. The code i

关键词： Cross-modal retrieval partially mismatched pairs trusted margin loss uncertainty guided division uncertainty self-correction

来源：评论

学校读者我要写书评

暂无评论

Guest Editorial Introduction to the Special Section on Video and Language

引用

IEEE Transactions on Circuits and Systems for Video technology 2022年第1期32卷 1-4页

作者： Mei, Tao Corso, Jason J. Kim, Gunhee Luo, Jiebo Shen, Chunhua Zhang, Hanwang Computer Vision and Multimedia Laboratory Jd Explore Academy *** Beijing100101 China Department of Electrical Engineering and Computer Science University of Michigan Ann ArborMI48109 United States Department of Computer Science and Engineering Seoul National University Gwanak-gu Seoul08826 Korea Republic of Department of Computer Science University of Rochester RochesterNY14627 United States College of Computer Science and Technology Zhejiang University Hangzhou310058 China School of Computer Science and Engineering Nanyang Technological University 639798 Singapore

computer vision (CV) and Natural Language Processing (NLP) are two most fundamental disciplines under a broad area of artificial intelligence (AI). CV is regarded as a field of research that explores the techniques to teach computers to see and understand digital content such as images and videos. NLP is a branch of linguistics that enables computers to process, interpret, and even generate human language. With the rise and development of deep learning over the past decade, there has been a steady momentum of innovation and breakthroughs that convincingly push the limits and improve the state-of-the-art of both vision and language modeling. An interesting observation is that the research in the two areas starts to interact, with a significant growth in both the volume of publications and extensive applications. Meanwhile, many previous experiences have shown that this can naturally build up the circle of human intelligence.

关键词： Special issues and sections computer vision Question answering (information retrieval) Natural language processing Visual captioning

来源：评论

学校读者我要写书评

暂无评论

Edge Harmony Attention Network for Semi-Supervised Medical Image Segmentation

SSRN

引用

SSRN 2024年

作者： Lu, Manti Liu, Yang Chen, Yunzhu Yang, Feng School of Computer Electronics and Information Guangxi University Guangxi Nanning530004 China Center for Machine Vision and Signal Analysis University of Oulu OuluFI-90014 Finland Guangxi Key Laboratory of Multimedia Communications Network Technology Guangxi University Guangxi Nanning530004 China

Medical image segmentation is crucial for identifying and measuring lesions and tumors, but acquiring high-quality annotated images is costly and challenging. To address this issue, we propose the Edge Harmony Attention Network (EHA-Net). EHA-Net tackles the edge artifacts common in traditional data augmentation methods by introducing the Edge Harmonization module, ensuring smooth transitions at image edges and improving segmentation accuracy. Additionally, we incorporate the Foreground Attention module to enhance feature alignment between the foreground and background, improving image style consistency and segmentation precision. Experimental results show that EHA-Net outperforms five leading semi-supervised methods on two public medical image datasets. The code is available on GitHub at this link. © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Decouple the High-Frequency and Low-Frequency Information of Images for Semantic Segmentation

Decouple the High-Frequency and Low-Frequency Information of...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Lianlei Shan Xiaobin Li Weiqiang Wang Computer Vision and Multimedia Technology Laboratory University of Chinese Academy of Sciences CAS Beijing China

ISBN: (纸本)9781728176055;9781728176062

As a special kind of signal processing technology, image processing has been developed rapidly after the appearance of convolutional neural network (CNN). At present, the semantic segmentation methods are all based on CNN and ignore the advantages of traditional image processing technology. We combine the two and make them promote each other. The high frequency component of the image represents the edge part and the low frequency represents the body part. Based on this assumption, we use Fourier transform to obtain the high and low frequency component from images. Then, a multi-branch parallel network structure is designed, and the high and low frequency components are sent into two branches respectively to obtain the body and edge features. Finally, the two features are fused together through one deep feature fusion to obtain the final output. In this way, the edge information and body information are decoupled on original images and extracted separately, which not only ensures the consistency of the internal information within objects, but also strengthens the supervision of the edge part which is the most error prone area in semantic segmentation. The results on Cityscapes and KITTI fully demonstrate the effectiveness of our work.

关键词： Image segmentation Fourier transforms Fuses Image edge detection Semantics Signal processing Feature extraction

来源：评论

学校读者我要写书评

暂无评论

Regress Before Construct: Regress Autoencoder for Point Cloud Self-supervised Learning

arXiv

引用

arXiv 2023年

作者： Liu, Yang Chen, Chen Wang, Can King, Xulin Liu, Mengyuan College of Computer Science Sichuan University China Center for Research in Computer Vision University of Central Florida United States Laboratory on Multimedia Information Processing The Department of Computer Science Kiel University Hangzhou Linxrobot Company China Hangzhou GOTHEN Technology Co. Ltd China Key Laboratory of Machine Perception Shenzhen Graduate School Peking University China

Masked Autoencoders (MAE) have demonstrated promising performance in self-supervised learning for both 2D and 3D computer vision. Nevertheless, existing MAE-based methods still have certain drawbacks. Firstly, the functional decoupling between the encoder and decoder is incomplete, which limits the encoder’s representation learning ability. Secondly, downstream tasks solely utilize the encoder, failing to fully leverage the knowledge acquired through the encoder-decoder architecture in the pre-text task. In this paper, we propose Point Regress AutoEncoder (Point-RAE), a new scheme for regressive autoencoders for point cloud self-supervised learning. The proposed method decouples functions between the decoder and the encoder by introducing a mask regressor, which predicts the masked patch representation from the visible patch representation encoded by the encoder and the decoder reconstructs the target from the predicted masked patch representation. By doing so, we minimize the impact of decoder updates on the representation space of the encoder. Moreover, we introduce an alignment constraint to ensure that the representations for masked patches, predicted from the encoded representations of visible patches, are aligned with the masked patch presentations computed from the encoder. To make full use of the knowledge learned in the pre-training stage, we design a new finetune mode for the proposed Point-RAE. Extensive experiments demonstrate that our approach is efficient during pre-training and generalizes well on various downstream tasks. Specifically, our pretrained models achieve a high accuracy of 90.28% on the ScanObjectNN hardest split and 94.1% accuracy on ModelNet40, surpassing all the other self-supervised learning methods. Our code and pretrained model are public available at: https://***/liuyyy111/ Point-RAE. Copyright © 2023, The Authors. All rights reserved.

关键词： Supervised learning

来源：评论

学校读者我要写书评

暂无评论

Joint Design of Radar Receive Filter and Unimodular ISAC Waveform with Sidelobe Level Control

arXiv

引用

arXiv 2025年

作者： Zhang, Kecheng Liu, Ya-Feng Wang, Zhongbin Yuan, Weijie Keskin, Musa Furkan Wymeersch, Henk Xia, Shuqiang School of System Design and Intelligent Manufacturing The Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology Shenzhen518055 China State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China Department of Electrical Engineering Chalmers University of Technology Gothenburg41296 Sweden ZTE Corporation The State Key Laboratory of Mobile Network and Mobile Multimedia Technology Shenzhen518055 China

Integrated sensing and communication (ISAC) has been considered a key feature of next-generation wireless networks. This paper investigates the joint design of the radar receive filter and dual-functional transmit waveform for the multiple-input multiple-output (MIMO) ISAC system. While optimizing the mean square error (MSE) of the radar receive spatial response and maximizing the achievable rate at the communication receiver, besides the constraints of full-power radar receiving filter and unimodular transmit sequence, we control the maximum range sidelobe level, which is often overlooked in existing ISAC waveform design literature, for better radar imaging performance. To solve the formulated optimization problem with convex and nonconvex constraints, we propose an inexact augmented Lagrangian method (ALM) algorithm. For each subproblem in the proposed inexact ALM algorithm, we custom-design a block successive upper-bound minimization (BSUM) scheme with closed-form solutions for all blocks of variable to enhance the computational efficiency. Convergence analysis shows that the proposed algorithm is guaranteed to provide a stationary and feasible solution. Extensive simulations are performed to investigate the impact of different system parameters on communication and radar imaging performance. Comparison with the existing works shows the superiority of the proposed algorithm. © 2025, CC BY-NC-SA.

关键词： Mean square error

来源：评论

学校读者我要写书评

暂无评论

ST-KeyS: Self-Supervised Transformer for Keyword Spotting in Historical Handwritten Documents

arXiv

引用

arXiv 2023年

作者： Jemni, Sana Khamekhem Ammar, Sourour Souibgui, Mohamed Ali Kessentini, Yousri Cheddad, Abbas Digital Research Center of Sfax B.P. 275 Sakiet Ezzit Sfax3021 Tunisia Laboratory of Signals systems Artificial Intelligence and networks Sfax Tunisia Multimedia Information systems and Advanced Computing Laboratory Tunisia Computer Vision Center Computer Science Department Universitat Autònoma de Barcelona Spain Department of Computer Science Blekinge Institute of Technology Karlskrona Sweden

Keyword spotting (KWS) in historical documents is an important tool for the initial exploration of digitized collections. Nowadays, the most efficient KWS methods are relying on machine learning techniques that require a large amount of annotated training data. However, in the case of historical manuscripts, there is a lack of annotated corpus for training. To handle the data scarcity issue, we investigate the merits of the self-supervised learning to extract useful representations of the input data without relying on human annotations and then using these representations in the downstream task. We propose ST-KeyS, a masked auto-encoder model based on vision transformers where the pretraining stage is based on the mask-and-predict paradigm, without the need of labeled data. In the fine-tuning stage, the pre-trained encoder is integrated into a siamese neural network model that is fine-tuned to improve feature embedding from the input images. We further improve the image representation using pyramidal histogram of characters (PHOC) embedding to create and exploit an intermediate representation of images based on text attributes. In an exhaustive experimental evaluation on three widely used benchmark datasets (Botany, Alvermann Konzilsprotokolle and George Washington), the proposed approach outperforms state-of-the-art methods trained on the same datasets. © 2023, CC BY.

关键词： Image enhancement

来源：评论

学校读者我要写书评

暂无评论

St-Keys: Self-Supervised Transformer for Keyword Spotting in Historical Handwritten Documents

SSRN

引用

SSRN 2023年

作者： Jemni, Sana Khamekhem Ammar, Sourour Souibgui, Mohamed Ali Kessentini, Yousri Cheddad, Abbas Digital Research Center of Sfax B.P. 275 Sakiet Ezzit Sfax3021 Tunisia SM@RTS: Laboratory of Signals systeMs aRtificial Intelligence and neTworkS MIR@CL: Multimedia InfoRmation systems and Advanced Computing Laboratory Computer Vision Center Computer Science Department Universitat Aut`onoma de Barcelona Spain Department of Computer Science Blekinge Institute of Technology Karlskrona Sweden

Keyword spotting (KWS) in historical documents is an important tool for the initial exploration of digitized collections. Nowadays, the most efficient KWS methods are relying on machine learning techniques that require a large amount of annotated training data. However, in the case of historical manuscripts, there is a lack of annotated corpus for training. To handle the data scarcity issue, we investigate the merits of the self-supervised learning to extract useful representations of the input data without relying on human annotations and then using these representations in the downstream task. We propose ST-KeyS, a masked auto-encoder model based on vision transformers where the pretraining stage is based on the mask-and-predict paradigm, without the need of labeled data. In the fine-tuning stage, the pre-trained encoder is integrated into a siamese neural network model that is fine-tuned to improve feature embedding from the input images. We further improve the image representation using pyramidal histogram of characters (PHOC) embedding to create and exploit an intermediate representation of images based on text attributes. In an exhaustive experimental evaluationon three widely used benchmark datasets (Botany, Alvermann Konzilsprotokolle and George Washington), the proposed approach outperforms state-of-the-art methods trained on the same datasets. © 2023, The Authors. All rights reserved.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

Physical Adversarial Attack Meets computer vision: A Decade Survey

arXiv

引用

arXiv 2022年

作者： Wei, Hui Tang, Hao Jia, Xuemei Wang, Zhixiang Yu, Hanxun Li, Zhubo Satoh, Shin'ichi Van Gool, Luc Wang, Zheng School of Computer Science National Engineering Research Center for Multimedia Software Wuhan University Wuhan China National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University Beijing100871 China Colleage of Software and Technology Zhejiang University Hangzhou China School of Cyber Science and Engineering Wuhan University Wuhan China Digital Content and Media Sciences Research Division National Institute of Informatics Japan Department of Information and Communication Engineering Graduate School of Information Science and Technology The University of Tokyo Japan Computer Vision Lab of ETH Zurich Zürich8092 Switzerland KU Leuven Leuven3000 Belgium INSAIT Sofia Bulgaria

Despite the impressive achievements of Deep Neural Networks (DNNs) in computer vision, their vulnerability to adversarial attacks remains a critical concern. Extensive research has demonstrated that incorporating sophisticated perturbations into input images can lead to a catastrophic degradation in DNNs’ performance. This perplexing phenomenon not only exists in the digital space but also in the physical world. Consequently, it becomes imperative to evaluate the security of DNNs-based systems to ensure their safe deployment in real-world scenarios, particularly in security-sensitive applications. To facilitate a profound understanding of this topic, this paper presents a comprehensive overview of physical adversarial attacks. Firstly, we distill four general steps for launching physical adversarial attacks. Building upon this foundation, we uncover the pervasive role of artifacts carrying adversarial perturbations in the physical world. These artifacts influence each step. To denote them, we introduce a new term: adversarial medium. Then, we take the first step to systematically evaluate the performance of physical adversarial attacks, taking the adversarial medium as a first attempt. Our proposed evaluation metric, hiPAA, comprises six perspectives: Effectiveness, Stealthiness, Robustness, Practicability, Aesthetics, and Economics. We also provide comparative results across task categories, together with insightful observations and suggestions for future research directions. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

MIPI 2024 Challenge on Nighttime Flare Removal: Methods and Results

arXiv

引用

arXiv 2024年

作者： Dai, Yuekun Zhang, Dafeng Li, Xiaoming Yue, Zongsheng Li, Chongyi Zhou, Shangchen Feng, Ruicheng Yang, Peiqing Jin, Zhezhu Liu, Guanqun Loy, Chen Change Zhang, Lize Liu, Shuai Feng, Chaoyu Wang, Luyang Chen, Shuan Shao, Guangqi Wang, Xiaotao Lei, Lei Yang, Qirui Cheng, Qihua Xu, Zhiqiang Liu, Yihao Yue, Huanjing Yang, Jingyu Vasluianu, Florin-Alexandru Wu, Zongwei Ciubotariu, George Timofte, Radu Zhang, Zhao Zhao, Suiyi Wang, Bo Zuo, Zhichao Wei, Yanyan Teja, Kuppa Sai Sri Jayakar Reddy, A. Rongali, Girish Mitra, Kaushik Ma, Zhihao Liu, Yongxu Zhang, Wanying Shang, Wei He, Yuhong Peng, Long Yu, Zhongxin Luo, Shaofei Wang, Jian Miao, Yuqi Li, Baiang Wei, Gang Verma, Rakshank Maheshwari, Ritik Tekchandani, Rahul Hambarde, Praful Tazi, Satya Narayan Vipparthi, Santosh Kumar Murala, Subrahmanyam Zhang, Haopeng Hou, Yingli Yao, Mingde Levin, M.S. Sundararajan, Aniruth Hari Kumar, A. Xiaomi Inc. China School of Electrical and Information Engineering Tianjin University China Shenzhen MicroBT Electronics Technology Co. Ltd China Shanghai Artificial Intelligence Laboratory China Computer Vision Lab CAIDAS IFI University of Würzburg Germany Laboratory for Multimedia Computing Hefei University of Technology China Detect Technologies Pvt Ltd India Indian Institute of Technology Madras India Xidian University China Harbin Institute of Technology China Northeastern University China University of Science and Technology of China China Fujian Normal University China Snap Inc. United States Tongji University China Hefei University of Technology China GEC Ajmer India CVPR Lab IIT Ropar India SCSS Trinity College Dublin Ireland Faculty of Robot Science and Engineering Northeastern University China Software College Northeastern University China The Chinese University of Hong Kong China Shiv Nadar University India

The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems. However, the scarcity of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). Building on the achievements of the previous MIPI Workshops held at ECCV 2022 and CVPR 2023, we introduce our third MIPI challenge including three tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2024. In total, 170 participants were successfully registered, and 14 teams submitted results in the final testing phase. The developed solutions in this challenge achieved state-of-the-art performance on Nighttime Flare Removal. More details of this challenge and the link to the dataset can be found at https://***/MIPI2024. Copyright © 2024, The Authors. All rights reserved.

关键词： Image sensors

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：