检索结果-内蒙古大学图书馆

Robust video question answering via contrastive cross-modality representation learning

Science China(Information Sciences) 2024年第10期67卷 211-226页

作者： Xun YANG Jianming ZENG Dan GUO Shanshan WANG Jianfeng DONG Meng WANG School of Information Science and Technology University of Science and Technology of China Institute of Artificial Intelligence Hefei Comprehensive National Science Center School of Computer Science and Information Engineering Hefei University of Technology Institutes of Physical Science and Information Technology Anhui University School of Computer Science and Technology Zhejiang Gongshang University

Video question answering(VideoQA) is a challenging yet important task that requires a joint understanding of low-level video content and high-level textual semantics. Despite the promising progress of existing efforts, recent studies revealed that current VideoQA models mostly tend to over-rely on the superficial correlations rooted in the dataset bias while overlooking the key video content, thus leading to unreliable results. Effectively understanding and modeling the temporal and semantic characteristics of a given video for robust VideoQA is crucial but, to our knowledge, has not been well investigated. To fill the research gap, we propose a robust VideoQA framework that can effectively model the cross-modality fusion and enforce the model to focus on the temporal and global content of videos when making a QA decision instead of exploiting the shortcuts in datasets. Specifically, we design a self-supervised contrastive learning objective to contrast the positive and negative pairs of multimodal input, where the fused representation of the original multimodal input is enforced to be closer to that of the intervened input based on video perturbation. We expect the fused representation to focus more on the global context of videos rather than some static keyframes. Moreover, we introduce an effective temporal order regularization to enforce the inherent sequential structure of videos for video representation. We also design a Kullback-Leibler divergence-based perturbation invariance regularization of the predicted answer distribution to improve the robustness of the model against temporal content perturbation of videos. Our method is model-agnostic and can be easily compatible with various VideoQA backbones. Extensive experimental results and analyses on several public datasets show the advantage of our method over the state-of-the-art methods in terms of both accuracy and robustness.

关键词： video question answering cross-modality fusion contrastive learning cross-media reasoning

来源：评论

学校读者我要写书评

暂无评论

OCRBench: on the hidden mystery of OCR in large multimodal models

引用

Science China(Information Sciences) 2024年第12期67卷 23-35页

作者： Yuliang LIU Zhang LI Mingxin HUANG Biao YANG Wenwen YU Chunyuan LI Xu-Cheng YIN Cheng-Lin LIU Lianwen JIN Xiang BAI School of Artificial Intelligence and Automation Huazhong University of Science and Technology School of Electronic and Information Engineering South China University of Technology Microsoft Research School of Computer & Communication Engineering University of Science and Technology Beijing Institute of Automation Chinese Academy of Sciences School of Software Engineering Huazhong University of Science and Technology

Large models have recently played a dominant role in natural language processing and multimodal vision-language learning. However, their effectiveness in text-related visual tasks remains relatively unexplored. In this paper, we conducted a comprehensive evaluation of large multimodal models, such as GPT4V and Gemini, in various text-related visual tasks including text recognition, scene text-centric visual question answering(VQA), document-oriented VQA, key information extraction(KIE), and handwritten mathematical expression recognition(HMER). To facilitate the assessment of optical character recognition(OCR) capabilities in large multimodal models, we propose OCRBench, a comprehensive evaluation benchmark. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available. Furthermore, our study reveals both the strengths and weaknesses of these models, particularly in handling multilingual text, handwritten text, non-semantic text, and mathematical expression *** importantly, the baseline results presented in this study could provide a foundational framework for the conception and assessment of innovative strategies targeted at enhancing zero-shot multimodal *** evaluation pipeline and benchmark are available at https://***/Yuliang-Liu/Multimodal OCR.

关键词： large multimodal model OCR text recognition scene text-centric VQA document-oriented VQA key information extraction handwritten mathematical expression recognition

来源：评论

学校读者我要写书评

暂无评论

Bi-directional Projection Framework for Fast Single Image Super Resolution

引用

Recent Patents on engineering 2024年第9期18卷 152-167页

作者： Zhou, Ying Zheng, Zhichao Sun, Quansen School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing China School of Computer and Electronic Information/School of Artificial Intelligence Nanjing Normal University Nanjing China

Background: Collaborative Representation (CR) has been widely used in Single Image Super Resolution (SISR) with the assumption that Low-resolution (LR) and high-resolution (HR) features can be linearly represented by neighborhoods and share consistent CR coefficients. Numerous patents and journal papers have been published. However, this CR consistency does not hold in the reconstruction phase, which leads to degraded performance. Methods: To fulfill this gap, we propose a novel bi-directional projection model (BDPM) to establish a bi-directional mapping between LR and HR features without any consistency constraint. The multiple projection matrices are offline computed to reduce reconstruction time greatly. We further develop several strategies to extract features and group neighborhoods such that local structures can be preserved better. Results: Compared to the learning-based methods, BDPM is about 2 to 10 times faster and compared to the reconstruction-based methods, it is about 500 to 2,000 times faster. Conclusion: The empirical studies verify the effectiveness of BDPM and extensive experimental results demonstrate that BDPM achieves better SISR performance than many state-of-the-arts. © 2024 Bentham Science Publishers.

关键词： Mapping

来源：评论

学校读者我要写书评

暂无评论

Arc Filtering Method Based on Generative Adversarial Networks 5

Arc Filtering Method Based on Generative Adversarial Network...

引用

5th International Symposium on computer engineering and Intelligent Communications, ISCEIC 2024

作者： Zhang, Maihao Chen, Deng School of Computer Science & Engineering Artificial Intelligence Wuhan Institute of Technology Wuhan China

ISBN: (纸本)9798331518677

Robot monitoring has been widely applied to observe the operation of welding robots, but the arc generated during the welding process causes severe overexposure in local areas of robot surveillance images, significantly deteriorating the monitoring effect of the welding robots. Due to the complex background structure of robot welding images, existing methods produce poor quality images after filtering out the arc from robot welding images. In this study, we propose an arc filtering model based on generative adversarial networks (GANs), primarily aimed at filtering out the arc in robot welding images. The well-trained arc filtering model can effectively eliminate the arc in robot welding images, generating high-quality non-arcing images. Experimental studies were conducted using an industrial robot welding image dataset we built autonomously and compared with state-of-the-art methods, demonstrating the superiority of our proposed model. © 2024 IEEE.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

Defect Detection on Wire Based on YOLOv8 3

Defect Detection on Wire Based on YOLOv8

引用

3rd International Conference on artificial intelligence and computer Information technology, AICIT 2024

作者： Guo, Kai Yu, Baocheng School of Computer Science & Engineering Artificial Intelligence Wuhan Institute of Technology Wuhan China

ISBN: (纸本)9798350365443

The false detection rate and missed detection rate of wire defects using manual detection is unstable, when automatic winding machines are applied. Aiming at addressing the problem, a defect detection model based on YOLOv8 named Wire-YOLOv8 was proposed. First of all, SPD-Conv module is used to replace Conv module in the backbone to help extracting multi-scale features and obtain features from small defects, in this way, the ability to extract features is enhanced, detection accuracy is improved and miss detection rate drops. Furthermore, SimSPPF take the place of SPPF to reduce the size of computation and improve detection speed. Finally, WIoU is introduced to make loss function work better, optimizing localization ability of the model, providing high quality anchor and strong ability to detect defects. The experimental results show that the average accuracy (mAP@0.5) of the improved model Wire-YOLOv8 for the grease dirt, outer insulation defect and inner insulation defect of wire reaches 92.7%, 94.6% and 97.3%, which is 1.5%, 2.3% and 1.9% higher than the original model, respectively, and the detection speed reaches 211FPS, which can serve as a compelling solution to detect wire defects in industrial sites. © 2024 IEEE.

关键词： Wire

来源：评论

学校读者我要写书评

暂无评论

Target Detection Algorithm of SAR Ship Image Based on Improved Yolov8n 6

Target Detection Algorithm of SAR Ship Image Based on Improv...

引用

6th International Conference on Robotics and computer Vision, ICRCV 2024

作者： Lu, Qin Zhao, Tongzhou Hu, Wei Chai, Zhengpeng School of Computer Science and Engineering & Artificial Intelligence Wuhan Institute of Technology Wuhan China

ISBN: (纸本)9798331527426

With regard to ship detection in SAR images, there are problems that the ship scale changes greatly and the background environment is complex in SAR images, which leads to the low accuracy of ship detection. In order to solve the above problems, this paper proposes a target detection algorithm for SAR ship images based on the improved Yolov8n. Firstly, by introducing BiFPN (bi-directional feature pyramid network) into Yolov8n, cross-scale feature fusion is realized, and the model pays more attention to information of different scales. Secondly, LSK (Large Separable Kernel Attention) attention mechanism is added after the SPPF network of the model, which can capture more ship information and reduce the computational load in the complex background environment. Finally, Wise- IoU v2 is used to replace the original CIoU function in Yolov8n to improve the detection accuracy. The improved performance of the model is verified by using public data sets SAR-Ship-Dataset dataset and SSDD dataset. The experimental results show that the improved Yolo v8n ship detection model is better than Yolov8n algorithm model to a great extent. On the SAR-Ship-Dataset dataset, mAP0.5 improves by 0.423% and mAP0.5-0.95 improves by 2.362%;On the SSDD dataset, mAP0.5 improves by 0.551 % and mAP0.5-0.95 improves by 3.147%. © 2024 IEEE.

关键词： Radar imaging

来源：评论

学校读者我要写书评

暂无评论

Fake News Detection Based on Attention Mechanism 6

Fake News Detection Based on Attention Mechanism

引用

6th International Conference on Robotics and computer Vision, ICRCV 2024

作者： Wang, Zihao Zhao, Tongzhou Wang, Jiageng Li, Shiya School of Computer Science and Engineering & Artificial Intelligence Wuhan Institute of Technology Wuhan China

ISBN: (纸本)9798331527426

Currently, most multi-modal fake news detection methods rely on fusing multi-modal features using a single criterion, overlooking the potential contribution of original multi-modal information in fake news detection and classification. This paper proposes an attention-based multi-criteria evaluation model that strengthens text feature representation on the basis of using multi-criteria. The model utilizes ResNet for extracting image feature representations and weights the output of BERT hidden layers to obtain richer text features. Subsequently, text and image features are inputted into a multi-modal feature fusion module, and the resulting output is concatenated with pooled text and image features before being passed to the final classification module, which then outputs experimental results. Experimental results demonstrate that, compared to other baseline models, the proposed model achieves better performance on Weibo and Twitter datasets. © 2024 IEEE.

关键词： attention mechanism component fake news detection multi-criteria evaluation

来源：评论

学校读者我要写书评

暂无评论

Residual diverse ensemble for long-tailed multi-label text classification

引用

Science China(Information Sciences) 2024年第11期67卷 92-105页

作者： Jiangxin SHI Tong WEI Yufeng LI National Key Laboratory for Novel Software Technology Nanjing University School of Artificial Intelligence Nanjing University School of Computer Science and Engineering Southeast University Key Laboratory of Computer Network and Information Integration Southeast UniversityMinistry of Education

Long-tailed multi-label text classification aims to identify a subset of relevant labels from a large candidate label set, where the training datasets usually follow long-tailed label distributions. Many of the previous studies have treated head and tail labels equally, resulting in unsatisfactory performance for identifying tail labels. To address this issue, this paper proposes a novel learning method that combines arbitrary models with two steps. The first step is the “diverse ensemble” that encourages diverse predictions among multiple shallow classifiers, particularly on tail labels, and can improve the generalization of tail *** second is the “error correction” that takes advantage of accurate predictions on head labels by the base model and approximates its residual errors for tail labels. Thus, it enables the “diverse ensemble” to focus on optimizing the tail label performance. This overall procedure is called residual diverse ensemble(RDE). RDE is implemented via a single-hidden-layer perceptron and can be used for scaling up to hundreds of thousands of labels. We empirically show that RDE consistently improves many existing models with considerable performance gains on benchmark datasets, especially with respect to the propensity-scored evaluation ***, RDE converges in less than 30 training epochs without increasing the computational overhead.

关键词： multi-label learning extreme multi-label learning long-tailed distribution multi-label text classification ensemble learning

来源：评论

学校读者我要写书评

暂无评论

Deep learning-based method for detection of copy-move forgery in videos

引用

Neural Computing and Applications 2025年第14期37卷 8451-8464页

作者： Elbarougy, Reda Abdelfatah, Osama Behery, G.M. El-Badry, Noha M. Department of Artificial Intelligence and Data Science College of Computer Science and Engineering University of Ha’il Ha’il81481 Saudi Arabia Department of Information Technology Faculty of Computer and Artificial Intelligence Damietta University New Damietta Kafr Saad Egypt Department of Computer Science Faculty of Computer and Artificial Intelligence Damietta University New Damietta Kafr Saad Egypt Department of Mathematics Faculty of Science Damietta University New Damietta Kafr Saad Egypt

Video forgery is one of the most serious problems affecting the credibility and reliability of video content. Therefore, detecting video forgery presents a major challenge for researchers due to the diversity of forgery types, the modernity of the programs used in forgery operations, and the abundance of information and content present in videos. The seriousness of this issue arises from the widespread use of videos in vital fields that require very high accuracy with no room for doubt or error, such as courtrooms, journalism, and others. Copy-move forgery is one of the most common and dangerous types of video forgery because of the difficulty in identifying it with the naked eye and the great diversity of forgery techniques involved in this particular type. One of the biggest challenges researchers have faced in the past is the complexity of the steps required to detect forgery in videos, leading to significant computational complexity and time consumption. The proposed method also aims to achieve better results than previous methods while reducing computational operations. Ultimately, forgery is detected with great efficiency. Compared to previous methods used for detecting copy-move forgery in the Rewind dataset, the proposed method achieves the highest F1, reaching 0.86, with a significant difference of 0.13 compared to the best result of the previous methods. © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2025.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Image enhancement with intensity transformation on embedding space

引用

CAAI Transactions on intelligence technology 2024年第1期9卷 101-115页

作者： Hanul Kim Yeji Jeon Yeong Jun Koh Department of Applied Artificial Intelligence Seoul National University of Science and TechnologySeoulSouth Korea Department of Computer Science and Engineering Chungnam National UniversityDaejeonSouth Korea

In recent times,an image enhancement approach,which learns the global transformation function using deep neural networks,has gained ***,many existing methods based on this approach have a limitation:their transformation functions are too simple to imitate complex colour transformations between low-quality images and manually retouched high-quality *** order to address this limitation,a simple yet effective approach for image enhancement is *** proposed algorithm based on the channel-wise intensity transformation is ***,this transformation is applied to the learnt embedding space instead of specific colour spaces and then return enhanced features to *** this end,the authors define the continuous intensity transformation(CIT)to describe the mapping between input and output intensities on the embedding ***,the enhancement network is developed,which produces multi-scale feature maps from input images,derives the set of transformation functions,and performs the CIT to obtain enhanced *** experiments on the MIT-Adobe 5K dataset demonstrate that the authors’approach improves the performance of conventional intensity transforms on colour space ***,the authors achieved a 3.8%improvement in peak signal-to-noise ratio,a 1.8%improvement in structual similarity index measure,and a 27.5%improvement in learned perceptual image patch ***,the authors’algorithm outperforms state-of-the-art alternatives on three image enhancement datasets:MIT-Adobe 5K,Low-Light,and Google HDRþ.

关键词： computer vision deep learning image enhancement image processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：