检索结果-内蒙古大学图书馆

Geometry-sensitive semantic modeling in visual and visual-language domains for image captioning

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2025年 147卷

作者： Zhu, Wencai Jiang, Zetao He, Yuting Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China

Transformer-based models with grid features as visual representations perform well in image captioning. However, the division and flattening operations increase the difficulty of capturing objects and their relationships via pure semantic modeling. Furthermore, the natural language generated by the current Transformer model still suffers from semantic overconcentration. In this paper, we aim to improve the attention modules in two ways to solve the above issues. We first propose a Geometry-Sensitive Self-Attention (GSSA) module, subdivide geometric signals in the visual domain into relative position and distance, and assist the semantic modeling process according to their unique characteristics. It compensates for the lack of objects and their relationships in the grid features. Then, we propose a Geometry-Sensitive Cross-Attention (GSCA) module, which perceives the source neighboring relationships between images and text in the visual-language domain from a geometric perspective and uses these relationships to adjust the semantic correspondences between the two dynamically. It spreads overly focused attention to surrounding grids to improve understanding of full image content during captioning. To prove our designs, we apply GSSA and GSCA to a standard Transformer to construct a novel Geometry-Sensitive Transformer Network (GSTNet), which conducts geometry-sensitive semantic modeling in visual and visual-language domains. Extensive experiments are conducted to verify the effectiveness of our proposal. The results show that our GSTNet achieves superior performance compared to many state-of-theart image captioning models on the Microsoft Common Objects in Context (MSCOCO) dataset. Besides, the generalization of GSTNet is also verified on the Flickr30k dataset.

关键词： image captioning Grid features Transformer mechanism Geometry-sensitive semantic modeling

来源：评论

学校读者我要写书评

暂无评论

DMFormer: Dense Memory Linformer for image Captioning

引用

ELECTRONICS 2025年第9期14卷 1716-1716页

作者： He, Yuting Jiang, Zetao Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China

image captioning is a cross-task of computer vision and natural language processing, aiming to describe image content in natural language. Existing methods still have deficiencies in modeling the spatial location and semantic correlation between image regions. Furthermore, these methods often exhibit insufficient interaction between image features and text features. To address these issues, we propose a Linformer-based image captioning method, the Dense Memory Linformer for image Captioning (DMFormer), which has lower time and space complexity than the traditional Transformer architecture. The DMFormer contains two core modules: the Relation Memory Augmented Encoder (RMAE) and the Dense Memory Augmented Decoder (DMAD). In the RMAE, we propose Relation Memory Augmented Attention (RMAA), which combines explicit spatial perception and implicit spatial perception. It explicitly uses geometric information to model the geometric correlation between image regions and implicitly constructs memory unit matrices to learn the contextual information of image region features. In the DMAD, we introduce Dense Memory Augmented Cross Attention (DMACA). This module fully utilizes the low-level and high-level features generated by the RMAE through dense connections, and constructs memory units to store prior knowledge of image and text. It learns the cross-modal associations between visual and linguistic features through an adaptive gating mechanism. Experimental results on the MS-COCO dataset show that the descriptions generated by the DMFormer are richer and more accurate, with significant improvements in various evaluation metrics compared to mainstream methods.

关键词： image captioning Linformer relation memory augmented attention dense memory augmented cross attention

来源：评论

学校读者我要写书评

暂无评论

Channel-level Matching Knowledge Distillation for object detectors via MSE

引用

PATTERN RECOGNITION LETTERS 2024年 179卷 52-57页

作者： Jiang, Zetao Huang, Qinyang Zhang, Huijuan Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China

Knowledge distillation (KD) has been widely used in different tasks as a practical model compression technique. Due to the poor performance of directly using Mean Square Error (MSE) between the intermediate features of the teacher and student, most feature -based detector distillation methods are primarily concerned with proposing diverse attention mechanisms and employing MSE to guide the student in learning critical information. However, the significance of MSE in detector distillation is often overlooked. To enhance the distillation performance of MSE and provide a novel perspective, we propose channel -level Matching Knowledge Distillation via MSE (MKD). The idea of MKD is simple but effective: based on the match or mismatch of student and teacher channel features in Feature Pyramid Networks (FPN), we divide channels at the same location in the FPN layers of the student and teacher into match channels and mismatch channels and then assign different weights to them to calculate the MSE-based distillation loss, which forces the student to distinguish the teacher's channels and imitate the features of the corresponding channels to achieve better performance. We conduct extensive experiments on MS COCO and PASCAL VOC benchmarks to prove that MKD is superior to state-of-the-art distillation methods. For example, with a powerful RetinaNet-ResNeXt101 detector as the teacher, ResNet-50 based RetinaNet and RepPoints with our distillation method achieve 40.9% and 42.2% mAP on COCO2017, which are 3.5% and 3.6% higher than the baseline, respectively.

关键词： Knowledge distillation Object detection Channel matching Mean squared error

来源：评论

学校读者我要写书评

暂无评论

Semantic-alignment transformer and adversary hashing for cross-modal retrieval

引用

APPLIED INTELLIGENCE 2024年第17-18期54卷 7581-7602页

作者： Sun, Yajun Wang, Meng Ma, Ying Guangxi Univ Sci & Technol Key Lab Intelligent Informat Proc & Graph Comp Liuzhou 545006 Guangxi Peoples R China

Deep Cross-Modal Hashing (DCMH) has garnered significant attention in the field of cross-modal retrieval due to its advantages such as high computational efficiency and small storage space. However, existing DCMH methods still face certain limitations: (1) they neglect the correlation between labels, while label features exhibit high sparsity;(2) they lack fine-grained semantic alignment;(3) they fail to effectively address data imbalance. In order to tackle these issues, this paper introduces a framework named Semantic-Alignment Transformer and Adversary Hashing for Cross-modal Retrieval (SATAH). To the best of our knowledge, this is the first attempt at the Semantic-Alignment Transformer algorithm. Specifically, this paper first designs a label learning network that utilizes a crafted transformer module to extract label information, guiding adversarial learning and hash function learning accordingly. Subsequently, a Balanced Conditional Generative Adversarial Network (BCGAN) is constructed, marking the first instance of adversarial training guided by label information. Furthermore, a Weighted Semi-Hard Cosine Triplet Constraint is proposed to better ensure high-ranking similarity relationships among all items. Lastly, considering the correlation between labels, a semantic-alignment constraint is introduced to handle label correlation from a fine-grained perspective, capturing similarity on a global scale more effectively. Extensive experiments are conducted on multiple representative cross-modal datasets. In experiments with 64-bit hash code length, SATAH achieves average mAP values of 84.75%, 68.87%, and 68.73% on MIR Flickr, NUS-WIDE, and MS COCO datasets, respectively, outperforming state-of-the-art methods. The code is available at https://***/Daydaylight/SATAH.

关键词： Adversarial learning Cross-modal hashing Representation learning Semantic-alignment Transformer

来源：评论

学校读者我要写书评

暂无评论

Linear self-attention with multi-relational graph for knowledge graph completion

引用

APPLIED INTELLIGENCE 2025年第8期55卷 1-15页

作者： Liu, Weida Qiang, Baohua Chen, Ruidong Xie, Yuan Chen, Lirui Chen, Zhiqin Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China

Knowledge graph completion (KGC) aims to infer missing facts based on the existing knowledge. graph Convolutional Networks (GCNs) have gained significant traction due to their proficiency in effectively modeling graph structures, especially within the realm of Knowledge graph Completion (KGC). In GCN-based KGC methodologies, GCNs are initially employed to generate comprehensive representations of entities, followed by the application of Knowledge graph Embedding (KGE) models to elucidate the interactions among entities and relations. However, most GCN-based KGC models ignore the long-range pairwise relationships in the graph. To address these limitations and enhance KGC, we propose a model called Linear Self-Attention with Multi-Relational graph Network (LTRGN). Specifically, this model merges GCN and linear self-attention to serve as the encoder. This model introduces a linear self-attention that can capture long-range node dependencies without introducing excessive computational overhead. Furthermore, we implement an attention mechanism designed to better assess the significance of various neighboring nodes relative to the source node. We demonstrate the effectiveness of the proposed LTRGN on the standard FB15k-237, WN18RR, Kinship, and UMLS datasets. On the dense graphs Kinship and UMLS, the MRR of our model improves by 1.3% and 4.1%, respectively, while Hits@1 increases by 1.7% and 6.4% compared to the best-performing model. The results show the efficacy of the model for the KGC task. The code is released at https://***/lixianqingliuyan/LTRGN.

关键词： Knowledge graph completion Knowledge graph embedding graph convolutional networks Linear self-attention

来源：评论

学校读者我要写书评

暂无评论

FRSE-Net: low-illumination object detection network based on feature representation refinement and semantic-aware enhancement

引用

VISUAL COMPUTER 2024年第5期40卷 3233-3247页

作者： Jiang, Zetao Shi, Daoquan Zhang, Shaoqin Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China Nanchang Hangkong Univ Nanchang 330063 Peoples R China

Deep learning-based object detection methods have achieved great performance improvement. However, the current mainstream object detectors focus on normal illumination images, while low-illumination object detection is often ignored. It is still a challenging task to detect objects in low-illumination scenes due to insufficient illumination and low visibility. To address this issue, we propose a low-illumination object detection network based on feature representation refinement and semantic-aware enhancement, called FRSE-Net. There are two key components in the proposed network, including a feature capture module (FCM) and a semantic aggregation module (SAM). First, the FCM is designed to enhance the feature representation of the feature map, thus making the object features more discriminative. This is beneficial to capture more effective feature information for subsequent prediction tasks. Furthermore, the SAM is introduced to enhance the semantic-aware ability of the model in low-light images, which makes the detection network focus on the objects of interest to learn rich semantic information. Finally, the experimental results on two low-light image datasets demonstrate the effectiveness and superiority of the proposed network when compared with other advanced low-illumination detection methods.

关键词： Object detection Low-illumination Feature representation Semantic-aware Deep learning

来源：评论

学校读者我要写书评

暂无评论

Adaptive tree-like neural network: Overcoming catastrophic forgetting to classify streaming data with concept drifts

引用

KNOWLEDGE-BASED SYSTEMS 2024年 293卷

作者： Wen, YiMin Liu, Xiang Yu, Hang Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Guangxi Peoples R China Shanghai Univ Sch Comp Engn & Sci Shanghai 200444 Peoples R China

With the development of deep neural networks (DNNs), classifying streaming data with concept drifts based on DNNs is becoming more and more effective. However, the continuous and infinite characteristic of streaming data makes it difficult to set an appropriate depth for a DNN-based model in advance. Moreover, how to improve the model's adaptability to concept drifts while overcoming catastrophic forgetting still remains a difficult issue. To address these issues, an Adaptive Tree -like Neural Network (ATNN) is proposed in this paper. ATNN adaptively increases the depth of its active branch according to the weight of the deepest node in the active branch. Once a new concept is detected, it chooses a suitable position on its trunk to grow a branch for the new concept according to the relation between the Fisher information and the gradient of parameters. Experiments demonstrate the rationality of ATNN to adaptively increase the depth of its branch or to choose a suitable position to grow a new branch, and illustrate that ATNN can more quickly adapt to concept drifts, and continuously improve classification performance for recurring concepts. The code of the proposed algorithm is available at https://***/mlmmwym/ATNN.

关键词： Deep neural networks Streaming data Concept drift

来源：评论

学校读者我要写书评

暂无评论

EOFD-NET: EDGE OPTIMIZATION AND FEATURE DENOISING FOR WEAKLY SUPERVISED DEEP NUCLEI SEGMENTATION WITH POINT ANNOTATIONS 49

EOFD-NET: EDGE OPTIMIZATION AND FEATURE DENOISING FOR WEAKLY...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Pan, Xipeng Hou, Feihu Liu, Zhenbing Feng, Siyang Lan, Rushi Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Peoples R China

ISBN: (纸本)9798350344868;9798350344851

Nuclei segmentation is a fundamental and critical step in digital pathological image analysis. Fully supervised nuclei segmentation requires a lot of pixel-by-pixel manual annotation by pathologists, which is very time-consuming and laborious. To minimize the labeling burden of pathologists, this paper uses only point annotations of nuclei data for weakly supervised learning. Specifically, a two-stage model named EOFD-Net with feature denoising and edge optimization is proposed. In the first stage, three weak labels (K-means cluster labels, Voronoi labels, and superpixel labels) with complementary information are used to train the encoder-decoder network to achieve coarse segmentation of nuclei. A feature denoising module(FDM) is designed in the encoder part, which can effectively reduce noise interference. In the second stage, we designed an edge optimization strategy using the prior knowledge of the trained model in the first stage. Confident learning is employed to denoise pseudo-label and rectify the mislabel. These optimized labels are input into the second stage to obtain the final segmentation results. The performance of our method outperforms current state-of-the-art methods on two publicly nuclei segmentation datasets, MoNuSeg and TNBC.

关键词： Nuclei segmentation weakly supervised feature denoising edge optimization point annotations

来源：评论

学校读者我要写书评

暂无评论

YOLO-TC: An Optimized Detection Model for Monitoring Safety-Critical Small Objects in Tower Crane Operations

引用

ALGORITHMS 2025年第1期18卷 27-27页

作者： Ding, Dong Deng, Zhengrong Yang, Rui Guilin Univ Elect Technol Guangxi Key Lab Images & Graph Intelligent Proc Guilin 541004 Peoples R China

Ensuring operational safety within high-risk environments, such as construction sites, is paramount, especially for tower crane operations where distractions can lead to severe accidents. Despite existing behavioral monitoring approaches, the task of identifying small yet hazardous objects like mobile phones and cigarettes in real time remains a significant challenge in ensuring operator compliance and site safety. Traditional object detection models often fall short in crane operator cabins due to complex lighting conditions, cluttered backgrounds, and the small physical scale of target objects. To address these challenges, we introduce YOLO-TC, a refined object detection model tailored specifically for tower crane monitoring applications. Built upon the robust YOLOv7 architecture, our model integrates a novel channel-spatial attention mechanism, ECA-CBAM, into the backbone network, enhancing feature extraction without an increase in parameter count. Additionally, we propose the HA-PANet architecture to achieve progressive feature fusion, addressing scale disparities and prioritizing small object detection while reducing noise from unrelated objects. To improve bounding box regression, the MPDIoU Loss function is employed, resulting in superior accuracy for small, critical objects in dense environments. The experimental results on both the PASCAL VOC benchmark and a custom dataset demonstrate that YOLO-TC outperforms baseline models, showcasing its robustness in identifying high-risk objects under challenging conditions. This model holds significant promise for enhancing automated safety monitoring, potentially reducing occupational hazards by providing a proactive, resilient solution for real-time risk detection in tower crane operations.

关键词： computer vision small object detection YOLOv7 attention mechanism feature fusion

来源：评论

学校读者我要写书评

暂无评论

KinD-LCE: curve estimation and Retinex Fusion on low-light image

引用

SIGNAL image AND VIDEO procESSING 2024年第2期18卷 1733-1746页

作者： Lei, Xiaochun Mai, Weiliang Xie, Junlin Liu, He Jiang, Zetao Gong, Zhaoting Lu, Chang Lu, Linjun Guilin Univ Elect Technol Sch Comp Sci & Informat Secur Guilin 541004 Guangxi Peoples R China Guilin Univ Elect Technol Guangxi Key Lab Image & Graph Intelligent Proc Guilin 541004 Guangxi Peoples R China

Low-light images often suffer from noise and color distortion. Object detection, semantic segmentation, instance segmentation, and other tasks are challenging when working with low-light images because of image noise and chromatic aberration. We also found that the conventional Retinex theory loses information in adjusting the image for low-light tasks. In response to the aforementioned problem, this paper proposes an algorithm for low illumination enhancement. The proposed method, KinD-LCE, uses a light curve estimation module to enhance the illumination map in the Retinex decomposed image, improving the overall image brightness. An illumination map and reflection map fusion module were also proposed to restore the image details and reduce detail loss. Additionally, a TV(total variation) loss function was applied to eliminate noise. Our method was trained on the GladNet dataset, known for its diverse collection of low-light images, tested against the Low-Light dataset, and evaluated using the ExDark dataset for downstream tasks, demonstrating competitive performance with a PSNR of 19.7216 and SSIM of 0.8213.

关键词： Low-light image Retinex Fusion Curve estimation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：