检索结果-内蒙古大学图书馆

Multimodal image feature fusion for improving medical ultrasound image segmentation

BIOMEDICAL SIGNAL PROCESSING AND CONTROL 2024年 89卷

作者： Chen, Jiashuo Chen, Junying South China Univ Technol Sch Software Engn Guangzhou Peoples R China Minist Educ Key Lab Big Data & Intelligent Robot SCUT Guangzhou Peoples R China

Ultrasound imaging is the most commonly used technique to detect abnormalities of the thyroid and diagnose abnormal breasts. However, due to the inherent characteristics of ultrasound imaging, the contrast of ultrasound images is relatively low, and the tissue boundaries in the image are blurry. This makes it a challenging task to segment thyroid or breast lesions from ultrasound images. In this work, we explore the role of multimodal features at different levels of deep neural network models and their fusion mechanisms for multimodal images generated by ultrasound radio frequency (RF) data. The ultrasound RF data contain information related to the arrangement of scatterers in tissues and organs, which is only partially available in B-mode images. To solve this problem, we use ultrasound RF data to generate parametric images, including Nakagami and entropy images, to illustrate the abundant backscattered statistics contained in RF data. A channel-aware fusion module is proposed to adaptively fuse the features from images of different modalities. Experimental results show that the abundant information contained in RF data can be helpful to identify thyroid glands or breast lesions with low contrast and fuzzy boundaries in B-mode ultrasound images and improve the segmentation performance.

关键词： Medical ultrasound image segmentation Ultrasound parametric imaging Multimodal fusion Deep learning

来源：评论

学校读者我要写书评

暂无评论

A knowledge-enhanced network for joint multimodal entity-relation extraction

引用

INFORMATION PROCESSING & MANAGEMENT 2025年第3期62卷

作者： Huang, Shubin Cai, Yi Yuan, Li Wang, Jiexin South China Univ Technol Sch Software Engn Guangzhou Peoples R China South China Univ Technol Minist Educ Key Lab Big Data & Intelligent Robot Guangzhou Peoples R China

In the domain of multimodal analysis, Multimodal Named Entity Recognition (MNER) and Multimodal Relation Extraction (MRE) are two pivotal tasks aiming at identifying named entities and their relations by leveraging integrated information from text-image pairs. Recently, Joint Multimodal Entity-Relation Extraction (JMERE) has emerged as a unified task to combine MNER and MRE, exploiting the bidirectional interactions between the two tasks for enhanced performance. However, existing JMERE studies primarily focus on improving visual information utilization through feature alignment, often falling short when social media texts and accompanying images lack sufficient information, leading to inaccuracies in entity type and relation recognition. To tackle this challenge, we propose KEJME, a knowledge- enhanced network tailored for the JMERE task. KEJME utilizes GPT-3.5 and ConceptNet as knowledge sources to supplement the missing context information of text-image pairs by infusing external knowledge. Moreover, to ensure the relevance of imported knowledge, we introduce a knowledge feature selection module, which performs a fine-grained selection of knowledge features according to different visual objects and textual entities. Numerous experiments have demonstrated that KEJME significantly surpasses state-of-the-art methods, achieving substantial improvements in both recall and F1 score. These findings highlight the critical role of external knowledge integration and fine-grained knowledge selection in advancing multimodal entity-relation extraction.

关键词： Joint multimodal entity-relation extraction Knowledge graphs Large language models

来源：评论

学校读者我要写书评

暂无评论

Semantic-Constraint Matching for transformer-based weakly supervised object localization

引用

PATTERN RECOGNITION 2025年 158卷

作者： Cao, Yiwen Su, Yukun Wang, Wenjun Liu, Yanxia Wu, Qingyao South China Univ Technol Sch Software Engn Guangzhou Peoples R China Peng Cheng Lab Shenzhen Peoples R China Minist Educ Key Lab Big Data & Intelligent Robot Guangzhou Peoples R China

Weakly supervised object localization (WSOL) strives to localize objects with only image-level supervision. WSOL often faces challenges such as incomplete localization due to classifier bias and over-localization in real scenes where objects and backgrounds are strongly associated or structurally similar. While the latest Transformer-based methods effectively enhance localization performance by leveraging long-range feature dependencies, they may inadvertently amplify divergent background activation and remain susceptible to classification bias. To this end, we proposed a novel Se mantic-Constraint C onstraint M atching (SeCM) plug-in module tailored for transformer-based approaches. In detail, a local patch shuffle strategy is first introduced to disentangle partial contextual linkages, thereby creating image pairs. Then a semantic matching module extracts co-object knowledge from the primal-shuffled image pairs, drives the network to identify the association of foreground with semantic label to suppress divergent activation. Moreover, to alleviate incomplete localization and prevent excessive suppression of activation, we propose leveraging multi-modal class-specific textual representations to guide object localization by complementing intra-class priori diverse knowledge. Extensive experimental results conducted on CUB-200-2011 and ILSVRC datasets show that our method can achieve the new state-of-the-art performance.

关键词： Weakly-supervised object localization Vision transformer Image matching Vision language model

来源：评论

学校读者我要写书评

暂无评论

Improve Cross-Domain Mixed Sampling With Guidance Training for Adaptive Segmentation

引用

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 2025年 74卷

作者： Zhou, Wenlve Zhou, Zhiheng Wang, Tianlei Zeng, Delu South China Univ Technol Sch Elect & Informat Engn Key Lab Big Data & Intelligent Robot Minist Educ Guangzhou 510641 Guangdong Peoples R China Wuyi Univ Dept Intelligent Mfg Jiangmen 529020 Peoples R China

Unsupervised domain adaptation (UDA) endeavors to adjust models trained on a source domain to perform well on a target domain without requiring additional annotations. In the context of domain adaptive semantic segmentation, which tackles UDA for dense prediction, the goal is to circumvent the need for costly pixel-level annotations. Typically, various prevailing methods baseline rely on constructing intermediate domains via cross-domain mixed sampling techniques to mitigate the performance decline caused by domain gaps. However, such approaches generate synthetic data that diverge from real-world distributions, potentially leading the model astray from the true target distribution. To address this challenge, we propose a novel auxiliary task called Guidance Training. This task facilitates the effective utilization of cross-domain mixed sampling techniques while mitigating distribution shifts from the real world. Specifically, Guidance Training guides the model to extract and reconstruct the target-domain feature distribution from mixed data, followed by decoding the reconstructed target-domain features to make pseudo-label predictions. Importantly, integrating Guidance Training incurs minimal training overhead and imposes no additional inference burden. We demonstrate the efficacy of our approach by integrating it with existing methods, consistently improving performance. The implementation will be available at https://***/Wenlve-Zhou/Guidance-Training.

关键词： Training Semantic segmentation Computational modeling Adaptation models Predictive models Decoding Feature extraction Computer vision Annotations Semantics Distribution shift Guidance Training pseudo-label semantic segmentation unsupervised domain adaptation (UDA)

来源：评论

学校读者我要写书评

暂无评论

A benchmark dataset and semantics-guided detection network for spatial-temporal human actions in urban driving scenes

引用

PATTERN RECOGNITION 2025年 158卷

作者： Zhong, Fujin Wu, Yini Yu, Hong Wang, Guoyin Lu, Zhantao Chongqing Univ Posts & Telecommun Sch Comp Sci & Technol Chongqing Peoples R China Chongqing Univ Posts & Telecommun Key Lab Big Data Intelligent Comp Chongqing Peoples R China Minist Educ Key Lab Cyberspace Big Data Intelligent Secur Chongqing Peoples R China

In real urban driving scenes, human actions are very complex and have the characteristic of multiple concurrent actions. It has a great significance to detect human actions in urban traffic scenes for auxiliary or autonomous driving systems. In this view, we introduce the TITAN-Human Action dataset for the task of multi-person spatial-temporal action detection in urban driving scenes. TITAN-Human Action provides the fine-grained action labels and location coordinates for 17,574 persons in the processed frames from the TITAN dataset. Furthermore, we propose a semantics-guided detection network (SGDNet) based on a semantic inference module (SIM) for spatial-temporal human action detection in urban driving scenes. SIM encodes the category labels into sentence vectors at the semantic level with prompting and embedding, utilizes graphs to represent the directed co-occurrence relations between categories, and adopts the graph convolutional network for semantic inference. SGDNet exploits the inference results of SIM to guide the visual branch in better performing human action detection, thereby achieving the integration of visual and linguistic information. We conducted experiments to evaluate SGDNet and several baseline methods on the TITAN-Human Action dataset, and reveal the generalizability of SIM in spatial-temporal human action detection. The source code and annotation files will be available at https://***/yyhbswyn/SGDNet.

关键词： Spatial-temporal action detection Urban driving scenes Benchmark dataset Semantic inference

来源：评论

学校读者我要写书评

暂无评论

Graph-Based Multimodal Topic Modeling With Word Relations and Object Relations

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2024年 26卷 8210-8225页

作者： Zhu, Bingshan Cai, Yi Wang, Jiexin South China Univ Technol Sch Engn Guangzhou 510006 Peoples R China South China Univ Technol Key Lab Big Data & Intelligent Robot Minist Educ Guangzhou 510006 Peoples R China

In recent years, multimodal topic models have gained significant attention in various tasks involving short texts. Despite their impressive results, most models rely on bag-of-words assumptions for each modality, neglecting the intrinsic word relations in the textual modality and the underlying object relations in the visual modality. To address this limitation, we propose a novel approach that represents each document modality as a graph, harnessing the word relations and the visual object relations to guide the topic extraction process. Our approach is grounded in the insight that, in the textual modality, words with specific relations, such as co-occurrence relations, semantic relations and syntactic relations, are more likely to be assigned to the same topic. Similarly, in the visual modality, the relations between objects, such as spatial relations and contextual relations, can also provide valuable information for topic extraction. By leveraging graph-based representations, our model captures the inherent associations between words and visual objects, resulting in the generation of more coherent and interpretable topics. To infer the model's parameters, we develop an effective algorithm that integrates neural variational inference and contrastive learning. The experimental results on three datasets verify the effectiveness of our proposed model in terms of topic coherence, topic diversity and mean average precision, confirming that incorporating word relations and object relations through graph-based representations significantly enhances the quality of the extracted topics.

关键词： Visualization data mining Social networking (online) Feature extraction data models Context modeling Blogs Graph convolutional network multimodal topic model neural variational inference

来源：评论

学校读者我要写书评

暂无评论

Towards Automated Infographic Authoring From Natural Language Statement With Multiple Proportional Facts

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2024年 26卷 7101-7113页

作者： Lu, Jianwei Wang, Guohua Cai, Yi Wu, Xin South China Univ Technol Sch Software Engn Guangzhou 510642 Peoples R China South China Univ Technol Key Lab Big Data & Intelligent Robot Minist Educ Guangzhou Peoples R China

Infographics, which usually contain many well-designed visual elements, have significant advantages in delivering information efficiently and accurately. Previous research shows that proportion-related infographics make up the majority of all infographics. However, the creation of proportion-related infographics is difficult for general users. Recently, many researchers focus on generating infographics from the text with a single proportional fact. Our further research found that users tend to create infographics with multiple proportional facts. Existing research lacks modeling of relations between different facts, resulting in poor performance when generating infographics with multiple facts. In this paper, we model the relationship of different proportional facts based on the results of our investigation and design a deep learning-based model to classify them. At the same time, we also optimize the ability to extract multiple proportional facts from text. The experiments show that our model outperforms existing models when visualizing text with multiple proportional facts.

关键词： data visualization Task analysis data models Surveys Semantics Deep learning Costs Infographic natural language processing deep learning visualization automated infographic authoring

来源：评论

学校读者我要写书评

暂无评论

Synthetic oversampling with Mahalanobis distance and local information for highly imbalanced class-overlapped data

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 260卷

作者： Yan, Yuanting Zheng, Lei Han, Shuangyue Yu, Chengjin Zhou, Peng Anhui Univ Sch Comp Sci & Technol Key Lab Intelligent Comp & Signal Proc Minist Educ Anhui 230601 Peoples R China Anhui Univ Sch Big Data & Stat Key Lab Intelligent Comp & Signal Proc Minist Educ Anhui 230601 Peoples R China

Minority oversampling is currently one of the most popular and effective methods for handling imbalanced data. However, oversampling that relies on the observations of the minority class to generate new samples is not applicable in the scenario of imbalanced data with extremely scarce minority samples, because the strongly underrepresented minority class does not contain enough information to support the oversampling process. Since some recent studies have exhibited the effectiveness of using majority information to bootstrap oversampling, the neglect of class overlap in the sampling process would increase the overlapping degree and complicate the decision boundary. To this end, this paper proposes a Mahalanobis distance and Local information based OverSampling (MLOS) for highly imbalanced class-overlapped data. MLOS first employs the majority density to guide the sample synthesis, with Mahalanobis distance to extract the majority probability contour. Then for each minority seed sample, to avoid the generation of overlapping samples, MLOS constrain the synthetic process by finding the auxiliary sample (in its 5 nearest neighbors) with similar probability density value to the seed. Finally, MLOS uses a pair-wise data cleaning process to improve the visibility of the decision boundary according to the probability density of synthetic samples. Comparative experiments conducted on 16 highly imbalanced class-overlapped datasets, using 17 different methods, demonstrates the superiority of our proposed method in terms of three popular evaluation metrics AUC, , G- mean and Recall for imbalance classification. The source code of MLOS is available at https://***/ytyancp/MLOS.

关键词： Imbalanced data Highly imbalanced Mahalanobis distance Class overlap

来源：评论

学校读者我要写书评

暂无评论

Learning defense transformations for counterattacking adversarial examples

引用

NEURAL NETWORKS 2023年第1期164卷 177-185页

作者： Li, Jincheng Zhang, Shuhai Cao, Jiezhang Tan, Mingkui South China Univ Technol Guangzhou Peoples R China PengCheng Lab Shenzhen Peoples R China Minist Educ Key Lab Big Data & Intelligent Robot Beijing Peoples R China

Deep neural networks (DNNs) are vulnerable to adversarial examples with small perturbations. Adversarial defense thus has been an important means which improves the robustness of DNNs by defending against adversarial examples. Existing defense methods focus on some specific types of adversarial examples and may fail to defend well in real-world applications. In practice, we may face many types of attacks where the exact type of adversarial examples in real-world applications can be even unknown. In this paper, motivated by that adversarial examples are more likely to appear near the classification boundary and are vulnerable to some transformations, we study adversarial examples from a new perspective that whether we can defend against adversarial examples by pulling them back to the original clean distribution. We empirically verify the existence of defense affine transformations that restore adversarial examples. Relying on this, we learn defense transformations to counterattack the adversarial examples by parameterizing the affine transformations and exploiting the boundary information of DNNs. Extensive experiments on both toy and real-world data sets demonstrate the effectiveness and generalization of our defense method. The code is avaliable at https://***/SCUTjinchengli/DefenseTransformer.

关键词： Adversarial examples Classification boundary Defense transformations Affine transformations

来源：评论

学校读者我要写书评

暂无评论

Attributes-Assisted Joint Contrastive Learning for Person Re-Identification

引用

IEEE INTERNET OF THINGS JOURNAL 2024年第14期11卷 24672-24684页

作者： Wu, Qingru Zhou, Zhiheng Niu, Chang Liu, Xiaosheng Li, Bo South China Univ Technol Sch Elect & Informat Engn Guangzhou Guangdong Peoples R China South China Univ Technol Key Lab Big Data & Intelligent Robot Minist Educ Guangzhou Guangdong Peoples R China

Person reidentification (Re-ID) is a crucial technology for intelligent security in Internet of Things (IoT) systems. Recently, unsupervised learning has been widely used for person Re-ID due to its generalization property. However, the effectiveness of commonly used unsupervised clustering methods heavily relies on the quality of the clustered pseudo-labels. Moreover, pedestrian shots in real scenes are prone to factors, such as occlusion. In this article, we propose a novel global and local joint contrastive learning (GLCL) framework based on the memory bank. Specifically, we establish separate memory banks for global and local features, which are updated using global simple samples and local hard samples. The GLCL module helps excavate information from simple and hard samples, aiming to overcome the effects of poor retrieval scenarios, such as background clutter and occlusion. Additionally, we design an attributes-assisted clustering (AAC) module that utilizes pedestrian attributes to refine the clustering results. The AAC module can effectively reduce the impact of pseudo-label noise owing to the supplementary information offered by attributes. Our approach shows improved performance in person Re-ID tasks in complex scenarios, providing a promising solution for intelligent security systems in the IoT. Experimental results demonstrate the superiority of our proposed method.

关键词： Attributes-assisted contrastive learning Internet of Things (IoT) person reidentification (Re-ID) pseudo-label Attributes-assisted contrastive learning Internet of Things (IoT) person reidentification (Re-ID) pseudo-label

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：