检索结果-内蒙古大学图书馆

Radiology report generation with medical knowledge and multilevel image-report alignment: A new method and its verification

引用

ARTIFICIAL INTELLIGENCE IN MEDICINE 2023年 146卷 102714-102714页

作者： Zhao, Guosheng Zhao, Zijian Gong, Wuxian Li, Feng Shandong Univ Sch Control Sci & Engn Jinan 250061 Peoples R China Shandong First Med Univ Shandong Prov Hosp Dept Radiol Jinan 250021 Peoples R China Shandong Univ Qilu Hosp Dept Gen Surg Jinan 250012 Peoples R China Shandong Univ Sch Control Sci & Engn 17923 Jing Shi Rd Jinan 250061 Peoples R China

Medical report generation is an integral part of computer-aided diagnosis aimed at reducing the workload of radiologists and physicians and alerting them of misdiagnosis risks. In general, medical report generation is an image captioning task. Since medical reports have long sequences with data bias, the existing medical report generation models lack medical knowledge and ignore the interaction alignment between the two modalities of reports and images. The current paper attempts to mitigate these deficiencies by proposing an approach based on knowledge enhancement with multilevel alignment (MKMIA). To this end, it includes a knowledge enhancement (MKE) module and a multilevel alignment module (MIRA). Specifically, the MKE deals with general medical knowledge (MK) and historical knowledge (HK) obtained via data training. The general knowledge is embedded in the form of a dictionary with characteristic organs (referred to as Key) and organ aliases, disease symptoms, etc. (referred to as Value). It provides explicit exception candidates to mitigate data bias. Historical knowledge ensures the comparison of similar cases to provide a better diagnosis. MIRA furnishes coarse-to-fine multilevel alignment, reducing the gap between image and text features, improving the knowledge enhancement module's performance, and facilitating the generation of lengthy reports. Experimental results on two radiology report datasets (i.e., IU X-ray and MIMIC-CXR) proved the effectiveness of the proposed approach, achieving state-of-the-art performance.

关键词： Medical report generation Knowledge enhancement Multilevel alignment Self-attention encoder-decoder

来源：评论

学校读者我要写书评

暂无评论

Text Summarization Method Based on Gated Attention Graph Neural Network

引用

SENSORS 2023年第3期23卷 1654-1654页

作者： Huang, Jingui Wu, Wenya Li, Jingyi Wang, Shengchun Hunan Normal Univ Coll Informat Sci & Engn Changsha 410081 Peoples R China

Text summarization is an information compression technology to extract important information from long text, which has become a challenging research direction in the field of natural language processing. At present, the text summary model based on deep learning has shown good results, but how to more effectively model the relationship between words, more accurately extract feature information and eliminate redundant information is still a problem of concern. This paper proposes a graph neural network model GA-GNN based on gated attention, which effectively improves the accuracy and readability of text summarization. First, the words are encoded using a concatenated sentence encoder to generate a deeper vector containing local and global semantic information. Secondly, the ability to extract key information features is improved by using gated attention units to eliminate local irrelevant information. Finally, the loss function is optimized from the three aspects of contrastive learning, confidence calculation of important sentences, and graph feature extraction to improve the robustness of the model. Experimental validation was conducted on a CNN/Daily Mail dataset and MR dataset, and the results showed that the model in this paper outperformed existing methods.

关键词： encoder-decoder GNN contrastive learning confidence calculation of important sentences attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Boundary refinement network with semantic embedding connections for UAV aerial image semantic segmentation

引用

JOURNAL OF ELECTRONIC IMAGING 2023年第6期32卷 063003-063003页

作者： Li, Runzeng Shi, Zaifeng Kong, Fanning Zhao, Xiangyang Luo, Tao Tianjin Univ Sch Microelect Tianjin Peoples R China Tianjin Key Lab Imaging & Sensing Microelect Techn Tianjin Peoples R China Tianjin Univ Coll Intelligence & Comp Tianjin Peoples R China

Deep-learning-based semantic segmentation is the research focus for unmanned aerospace vehicle (UAV) aerial images analysis. However, there are problems in segmenting small and narrow objects and boundary regions, due to the large size differences between objects and the unbalanced class data in aerial images. A network named SEC-BRNet is proposed for the boundary refinement problem. First, the semantic embedding connections and progressive upsampling decoder are used to obtain spatial details for generating fused feature maps, which are then concatenated in decoding process level by level for recovering the boundary details. Second, a multiloss training strategy is developed for data imbalance and boundary roughness problems, including cross-entropy loss, Dice loss, and active boundary loss. After extensive experiments, our network could achieve 84.8% mIoU and 89.04% Boundary IoU on the AeroScapes dataset and achieve 62.81% mIoU and 90.78% Boundary IoU on the Semantic Drone Dataset. The experimental results indicate that the proposed SEC-BRNet performs well in semantic segmentation task for UAV aerial images.

关键词： semantic segmentation UAV aerial image boundary refinement encoder-decoder multiloss training

来源：评论

学校读者我要写书评

暂无评论

A sequence-to-sequence model for joint bridge response forecasting

引用

MECHANICAL SYSTEMS AND SIGNAL PROCESSING 2023年第1期203卷

作者： Bahrami, Omid Wang, Wentao Hou, Rui Lynch, Jerome P. Univ Michigan Depart Civil & Env Engn Ann Arbor MI 48109 USA Duke Univ Depart Civil & Env Engn Durham NC 27780 USA Univ Michigan Dept Civil & Environm Engn 2160 G G Brown Bldg Ann Arbor MI 48109 USA

Knowledge of the structural response of bridges is extremely important for highway asset management and bridge structural health monitoring. Instrumenting every bridge in a road network and maintaining the monitoring instrumentation over decades of service can be financially infeasible. Mechanical intuition suggests a significant relationship exists between responses of two sets of bridges reasonably similar in design exposed to an identical load. This study explores the use of data-driven models to forecast the response of one bridge to a given truck load using the response of another bridge to the same loading profile. By deploying a modern monitoring system in multiple bridges in the same highway corridor integrated in a cyber-physical systems (CPS) framework, and utilizing advanced computer vision algorithms, the authors have gathered a unique dataset consisting of pairs of bridge responses to the same truck load from live traffic moving across a 32.2 km (20 miles) stretch of the I-275 highway in southeast Michigan. Signal processing techniques have been employed to isolate the response of the bridges to a single truck load in a time series of recorded responses. Then, a deep-learning-based time series forecasting framework using the encoder-decoder architecture with gated recurrent unit (GRU) and long short-term memory (LSTM) cells has been used for bridge response forecasting. Baseline models based on linear time series models are also developed to which the deep-learning forecasting models can be compared. After training the models, it is observed that deep-learning-based models can accurately forecast the response of one bridge using the response of another and reduce the forecasting root-mean-squared error (RMSE) by at least 20% relative to baseline linear models. The forecasting capabilities of the encoder-decoder architecture proposed herein outperform traditional approaches to response forecasting. Trained versions of the encoderdecoder forecasting model

关键词： Time Series Forecasting Bridge Monitoring encoder-decoder Recurrent Neural Networks Deep Learning Sequence-to-Sequence

来源：评论

学校读者我要写书评

暂无评论

ICEAP: An advanced fine-grained image captioning network with enhanced attribute predictor

引用

DISPLAYS 2024年 84卷

作者： Hossen, Md. Bipul Ye, Zhongfu Abdussalam, Amr Hossain, Mohammad Alamgir Univ Sci & Technol China Sch Informat Sci & Technol Hefei 230027 Anhui Peoples R China

Fine-grained image captioning is a focal point in the vision-to-language task and has attracted considerable attention for generating accurate and contextually relevant image captions. Effective attribute prediction and their utilization play a crucial role in enhancing image captioning performance. Despite progress in prior attribute-related methods, they either focus on predicting attributes related to the input image or concentrate on predicting linguistic context-related attributes at each time step in the language model. However, these approaches often overlook the importance of balancing visual and linguistic contexts, leading to ineffective exploitation of semantic information and a subsequent decline in performance. To address these issues, an Independent Attribute Predictor (IAP) is introduced to precisely predict attributes related to the input image by leveraging relationships between visual objects and attribute embeddings. Following this, an Enhanced Attribute Predictor (EAP) is proposed, initially predicting linguistic context-related attributes and then using prior probabilities from the IAP module to rebalance image and linguistic context-related attributes, thereby generating more robust and enhanced attribute probabilities. These refined attributes are then integrated into the language LSTM layer to ensure accurate word prediction at each time step. The integration of the IAP and EAP modules in our proposed image captioning with the enhanced attribute predictor (ICEAP) model effectively incorporates high-level semantic details, enhancing overall model performance. The ICEAP outperforms contemporary models, yielding significant average improvements of 10.62% in CIDEr-D scores for MS-COCO, 9.63% for Flickr30K and 7.74% for Flickr8K datasets using cross-entropy optimization, with qualitative analysis confirming its ability to generate fine-grained captions.

关键词： Fine-grained image caption Attention mechanism encoder-decoder Independent attribute predictor Enhanced attribute predictor

来源：评论

学校读者我要写书评

暂无评论

LA-ResUNet: An Efficient Linear Attention Mechanism in ResUNet for the Semantic Segmentation of Pulmonary Nodules

引用

IEEE ACCESS 2024年 12卷 182894-182907页

作者： Sarah Prithvika, P. C. Jani Anbarasi, L. Vellore Inst Technol Sch Comp Sci & Engn Chennai 600127 India

Numerous people die from lung cancer every year, making it a serious public health issue. Oftentimes, the symptoms of lung cancer manifest only at a later stage, when it is difficult to treat. Pulmonary nodules are commonly found while screening the lungs using a Computed Tomography (CT) scan, and some of the nodules may be cancerous. So, an efficient automated pulmonary nodule segmentation system is needed to isolate the pulmonary nodules from the scan images. The doctors can track the nodules that are likely to be malignant and provide early treatment if they become cancerous, thereby improving the patient's chance of survival. The attention mechanism is a technique that is often used in computer vision to enhance the neural network's performance. LA-ResUNet, a pulmonary nodule segmentation model, built using ResUNet with a linear attention mechanism and the Leaky ReLU activation function is proposed. LA-ResUNet efficiently segments pulmonary nodules, while achieving a linear time and space complexity. By employing residual blocks, it is possible to construct a deep network without facing the vanishing gradient problem. Additionally, it makes deep network training simpler. Skip connections allow for better gradient flow during training and better information flow between layers. Leaky ReLU addresses the dying ReLU scenario, a situation where some neurons cease to learn when the network is being trained. LA-ResUNet was used on the dataset LIDC-IDRI (The Lung Image Database Consortium and Image Database Resource Initiative) and it produced a dice score coefficient (DSC) of 73.11% and Intersection over Union score (IoU) of 60.62%.

关键词： Lungs Lung cancer Image segmentation Tumors Solids Glass Feature extraction Computed tomography Biomedical imaging Blood vessels Convolutional neural network encoder-decoder linear attention pulmonary nodule segmentation ResUNet

来源：评论

学校读者我要写书评

暂无评论

Enhancing building segmentation by deep multiview classification for advancing sustainable urban development

引用

JOURNAL OF BUILDING ENGINEERING 2024年 83卷

作者： El Hajjar, Sally Kassem, Hassan Abdallah, Fahed Omrani, Hichem Luxembourg Inst Socio Econ Res LISER Urban Dev & Mobil Dept 11 Porte Sci L-4366 Esch Sur Alzette Luxembourg Lebanese Univ Beirut Lebanon Univ Lorraine Lab LCOMS Metz France

Accurate building segmentation plays a crucial role in a wide range of applications such as urban planning, monitoring, and mapping. Different deep learning models were employed for building segmentation. However, these models analyze images from a single view. Given the limitations of single-view building segmentation models, our research aims to enhance accuracy by proposing a novel multi-view U-Net deep model for accurate building segmentation that incorporates multiple views of the images. We employ two pre-trained convolutional neural network architectures, MobileNetV2 and ResNet50, to extract features representing two different views of our images. By fusing these features, our proposed method effectively captures complementary information, leading to enhanced segmentation accuracy. To further improve the model's performance, we incorporate skip connections and up-convolutional layers to ensure fine-grained feature propagation. Our experimental results on a large building dataset demonstrate a significant improvement in segmentation accuracy 91% compared to state-of-theart methods, highlighting the effectiveness of our multiview fusion approach. The experimental results enhance the benefits of creating different views by adopting the novel concept proposed in this paper. This research has the potential to redefine the landscape of building segmentation in applications such as urban planning and mapping. We also conducted a test on a large study area (city scale of Belval-Luxembourg). This demonstrates the capabilities of our method and its efficiency in segmenting satellite images from a large extent area and reinforces its potential for real-world applications.

关键词： Deep multiview building segmentation U-Net model Pix2pix Skip connection encoder-decoder

来源：评论

学校读者我要写书评

暂无评论

Enhanced transformer model for video caption generation

引用

EXPERT SYSTEMS 2023年第0期

作者： Varma, Soumya Peter, J. Dinesh Karunya Inst Technol & Sci Dept CSE Coimbatore Tamil Nadu India

Automatic Video captioning system is a method of describing the content in a video by analysing its visual aspects with regard to space and time and producing a meaningful caption that explains the video. A decade of research in this area has resulted in a steep growth in the quality and appropriateness of the generated caption compared with the expected result. The research has been driven from the very basic method to most advanced transformer method. Machine generated caption of a video must be adhering to many expected standards. For humans, this task may be a trivial one, however its not the same for a machine to analyse the content and generate a semantically coherent description for it. The caption which is generated in a natural language must also adhere to its lexical and syntactical structure. The video captioning process is a culmination of computer vision and natural language processing tasks. Commencing with template based conventional approach, it has surpassed statistical method, traditional deep learning approaches and is now in the trend of using transformers. This work made an extensive study of the literature and has proposed an improved transformer-based architecture for video captioning process. The transformer architecture made use of an encoder and decoder model that has two and three sublayers respectively. Multi-head self attention and cross attention are part of the model which bring about very beneficial results. The decoder is auto-regressive and uses a masked layer to prevent the model from foreseeing future words in the caption. An enhanced encoder-decoder Transformer model with CNN for feature extraction has been used in our work. This model captures the long-range dependencies and temporal relationships more effectively. The model has been evaluated with benchmark datasets and compared with state-of-the-art methods and found to be slightly better in the performance. The performance scores are slightly varying for BLEU, METEOR, ROUGE a

关键词： curriculum learning deep learning encoder-decoder engineering applications neural networks transformer video caption

来源：评论

学校读者我要写书评

暂无评论

Multi-Source Interactive Stair Attention for Remote Sensing Image Captioning

引用

REMOTE SENSING 2023年第3期15卷 579-579页

作者： Zhang, Xiangrong Li, Yunpeng Wang, Xin Liu, Feixiang Wu, Zhaoji Cheng, Xina Jiao, Licheng Xidian Univ Sch Artificial Intelligence Key Lab Intelligent Percept & Image Understanding Minist Educ Xian 710071 Peoples R China

The aim of remote sensing image captioning (RSIC) is to describe a given remote sensing image (RSI) using coherent sentences. Most existing attention-based methods model the coherence through an LSTM-based decoder, which dynamically infers a word vector from preceding sentences. However, these methods are indirectly guided through the confusion of attentive regions, as (1) the weighted average in the attention mechanism distracts the word vector from capturing pertinent visual regions and (2) there are few constraints or rewards for learning long-range transitions. In this paper, we propose a multi-source interactive stair attention mechanism that separately models the semantics of preceding sentences and visual regions of interest. Specifically, the multi-source interaction takes previous semantic vectors as queries and applies an attention mechanism on regional features to acquire the next word vector, which reduces immediate hesitation by considering linguistics. The stair attention divides the attentive weights into three levels-that is, the core region, the surrounding region, and other regions-and all regions in the search scope are focused on differently. Then, a CIDEr-based reward reinforcement learning is devised, in order to enhance the quality of the generated sentences. Comprehensive experiments on widely used benchmarks (i.e., the Sydney-Captions, UCM-Captions, and RSICD data sets) demonstrate the superiority of the proposed model over state-of-the-art models, in terms of its coherence, while maintaining high accuracy.

关键词： remote sensing image captioning cross-modal interaction attention mechanism semantic information encoder-decoder

来源：评论

学校读者我要写书评

暂无评论

Polarization image fusion with self-learned fusion strategy

引用

PATTERN RECOGNITION 2021年 118卷 108045-108045页

作者： Zhang, Junchao Shao, Jianbo Chen, Jianlai Yang, Degui Liang, Buge Cent South Univ Sch Aeronaut & Astronaut Changsha 410083 Peoples R China

Polarization image fusion aims to integrate intensity and degree of linear polarization images into one with more details, which is beneficial to improve the ability of targets detection under complex background. The fusion strategies in conventional methods are designed in a hand-crafted way and not robust to different fusion tasks. In this paper, we propose a novel and deep network to address the polarization image fusion issue with self-learned strategy. The network consists of encoder, Fusion, and decoder layers. Feature maps extracted by encoder are fused, then fed into decoder to generate fused images. Besides, a novel loss function is adopted to train the network in an unsupervised way, without ground truth of fused images. To verify the advantage, the network trained on polarization images is also used to infrared and visible images fusion, and multi-focus image fusion. Experimental results showed that our method outperforms several state-of-the-art methods in terms of visual quality and quantitative measurement. The proposed fused method can be applied in the military and civilian fields such as camouflage and hidden targets detection, medical diagnosis, and environmental monitoring. (c) 2021 Elsevier Ltd. All rights reserved.

关键词： Image fusion encoder-decoder Polarization image Unsupervised learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：