检索结果-内蒙古大学图书馆

Radiology report generation with medical knowledge and multilevel image-report alignment: A new method and its verification

引用

ARTIFICIAL INTELLIGENCE IN MEDICINE 2023年 146卷 102714-102714页

作者： Zhao, Guosheng Zhao, Zijian Gong, Wuxian Li, Feng Shandong Univ Sch Control Sci & Engn Jinan 250061 Peoples R China Shandong First Med Univ Shandong Prov Hosp Dept Radiol Jinan 250021 Peoples R China Shandong Univ Qilu Hosp Dept Gen Surg Jinan 250012 Peoples R China Shandong Univ Sch Control Sci & Engn 17923 Jing Shi Rd Jinan 250061 Peoples R China

Medical report generation is an integral part of computer-aided diagnosis aimed at reducing the workload of radiologists and physicians and alerting them of misdiagnosis risks. In general, medical report generation is an image captioning task. Since medical reports have long sequences with data bias, the existing medical report generation models lack medical knowledge and ignore the interaction alignment between the two modalities of reports and images. The current paper attempts to mitigate these deficiencies by proposing an approach based on knowledge enhancement with multilevel alignment (MKMIA). To this end, it includes a knowledge enhancement (MKE) module and a multilevel alignment module (MIRA). Specifically, the MKE deals with general medical knowledge (MK) and historical knowledge (HK) obtained via data training. The general knowledge is embedded in the form of a dictionary with characteristic organs (referred to as Key) and organ aliases, disease symptoms, etc. (referred to as Value). It provides explicit exception candidates to mitigate data bias. Historical knowledge ensures the comparison of similar cases to provide a better diagnosis. MIRA furnishes coarse-to-fine multilevel alignment, reducing the gap between image and text features, improving the knowledge enhancement module's performance, and facilitating the generation of lengthy reports. Experimental results on two radiology report datasets (i.e., IU X-ray and MIMIC-CXR) proved the effectiveness of the proposed approach, achieving state-of-the-art performance.

关键词： Medical report generation Knowledge enhancement Multilevel alignment Self-attention encoder-decoder

来源：评论

学校读者我要写书评

暂无评论

Boundary refinement network with semantic embedding connections for UAV aerial image semantic segmentation

引用

JOURNAL OF ELECTRONIC IMAGING 2023年第6期32卷 063003-063003页

作者： Li, Runzeng Shi, Zaifeng Kong, Fanning Zhao, Xiangyang Luo, Tao Tianjin Univ Sch Microelect Tianjin Peoples R China Tianjin Key Lab Imaging & Sensing Microelect Techn Tianjin Peoples R China Tianjin Univ Coll Intelligence & Comp Tianjin Peoples R China

Deep-learning-based semantic segmentation is the research focus for unmanned aerospace vehicle (UAV) aerial images analysis. However, there are problems in segmenting small and narrow objects and boundary regions, due to the large size differences between objects and the unbalanced class data in aerial images. A network named SEC-BRNet is proposed for the boundary refinement problem. First, the semantic embedding connections and progressive upsampling decoder are used to obtain spatial details for generating fused feature maps, which are then concatenated in decoding process level by level for recovering the boundary details. Second, a multiloss training strategy is developed for data imbalance and boundary roughness problems, including cross-entropy loss, Dice loss, and active boundary loss. After extensive experiments, our network could achieve 84.8% mIoU and 89.04% Boundary IoU on the AeroScapes dataset and achieve 62.81% mIoU and 90.78% Boundary IoU on the Semantic Drone Dataset. The experimental results indicate that the proposed SEC-BRNet performs well in semantic segmentation task for UAV aerial images.

关键词： semantic segmentation UAV aerial image boundary refinement encoder-decoder multiloss training

来源：评论

学校读者我要写书评

暂无评论

A sequence-to-sequence model for joint bridge response forecasting

引用

MECHANICAL SYSTEMS AND SIGNAL PROCESSING 2023年第1期203卷

作者： Bahrami, Omid Wang, Wentao Hou, Rui Lynch, Jerome P. Univ Michigan Depart Civil & Env Engn Ann Arbor MI 48109 USA Duke Univ Depart Civil & Env Engn Durham NC 27780 USA Univ Michigan Dept Civil & Environm Engn 2160 G G Brown Bldg Ann Arbor MI 48109 USA

Knowledge of the structural response of bridges is extremely important for highway asset management and bridge structural health monitoring. Instrumenting every bridge in a road network and maintaining the monitoring instrumentation over decades of service can be financially infeasible. Mechanical intuition suggests a significant relationship exists between responses of two sets of bridges reasonably similar in design exposed to an identical load. This study explores the use of data-driven models to forecast the response of one bridge to a given truck load using the response of another bridge to the same loading profile. By deploying a modern monitoring system in multiple bridges in the same highway corridor integrated in a cyber-physical systems (CPS) framework, and utilizing advanced computer vision algorithms, the authors have gathered a unique dataset consisting of pairs of bridge responses to the same truck load from live traffic moving across a 32.2 km (20 miles) stretch of the I-275 highway in southeast Michigan. Signal processing techniques have been employed to isolate the response of the bridges to a single truck load in a time series of recorded responses. Then, a deep-learning-based time series forecasting framework using the encoder-decoder architecture with gated recurrent unit (GRU) and long short-term memory (LSTM) cells has been used for bridge response forecasting. Baseline models based on linear time series models are also developed to which the deep-learning forecasting models can be compared. After training the models, it is observed that deep-learning-based models can accurately forecast the response of one bridge using the response of another and reduce the forecasting root-mean-squared error (RMSE) by at least 20% relative to baseline linear models. The forecasting capabilities of the encoder-decoder architecture proposed herein outperform traditional approaches to response forecasting. Trained versions of the encoderdecoder forecasting model

关键词： Time Series Forecasting Bridge Monitoring encoder-decoder Recurrent Neural Networks Deep Learning Sequence-to-Sequence

来源：评论

学校读者我要写书评

暂无评论

Non-intrusive residential electricity load decomposition via low-resource model transferring

引用

JOURNAL OF BUILDING ENGINEERING 2023年 73卷

作者： Lin, Lin Shi, Jiancheng Ma, Changwen Zuo, Shi Zhang, Jie Chen, Cheng Huang, Nantian Jilin Inst Chem Technol Coll Informat & Control Engn Jilin 132022 Peoples R China Shanghai Second Polytech Univ Sch Environm & Mat Engn Shanghai 201209 Peoples R China State Grid Jilin Elect Power Co Ltd Econ & Tech Res Inst Changchun 130022 Peoples R China Northeast Dianli Univ Sch Elect Engn Jilin 132012 Peoples R China

Non-intrusive load decomposition (NILD) technology has a broad application prospect because it can deeply excavate the internal electricity consumption data of customers and obtain electricity consumption information. A non-intrusive load decomposition method based on model transfer is proposed to address the problem of insufficient accumulation of historical electricity consumption data of some household customers. Firstly, Time2vec, an embedding layer with time vectors, is introduced as an encoder to convert the time series into a continuous vector space, which improves the computational efficiency of the model for power time series information. Then, by introducing the Channel-wise attention mechanism, favorable residential electricity load characteristics can be effectively obtained. The model transfer method is used to fine-tune the pretrained model based on complete data to overcome the problem of insufficient accumulation of historical electricity consumption data of some households. The experimental results show that the proposed method is a significant improvement over the existing state-of-the-art methods. The proposed method significantly improves the accuracy of load decomposition while reducing the model parameters. The model parameters are reduced by 62.4%, and the average absolute error is reduced by 13.8%-59.3%. Even with a small number of data training source domains, the proposed method is still able to accurately obtain the power consumption information of each appliance, proving that the proposed method overcomes the problem of insufficient accumulation of electricity consumption data of household users.

关键词： Residential electricity load Non-intrusive load decomposition encoder-decoder Channel-wise attention mechanism Model transferring network

来源：评论

学校读者我要写书评

暂无评论

Enhanced transformer model for video caption generation

引用

EXPERT SYSTEMS 2023年第0期

作者： Varma, Soumya Peter, J. Dinesh Karunya Inst Technol & Sci Dept CSE Coimbatore Tamil Nadu India

Automatic Video captioning system is a method of describing the content in a video by analysing its visual aspects with regard to space and time and producing a meaningful caption that explains the video. A decade of research in this area has resulted in a steep growth in the quality and appropriateness of the generated caption compared with the expected result. The research has been driven from the very basic method to most advanced transformer method. Machine generated caption of a video must be adhering to many expected standards. For humans, this task may be a trivial one, however its not the same for a machine to analyse the content and generate a semantically coherent description for it. The caption which is generated in a natural language must also adhere to its lexical and syntactical structure. The video captioning process is a culmination of computer vision and natural language processing tasks. Commencing with template based conventional approach, it has surpassed statistical method, traditional deep learning approaches and is now in the trend of using transformers. This work made an extensive study of the literature and has proposed an improved transformer-based architecture for video captioning process. The transformer architecture made use of an encoder and decoder model that has two and three sublayers respectively. Multi-head self attention and cross attention are part of the model which bring about very beneficial results. The decoder is auto-regressive and uses a masked layer to prevent the model from foreseeing future words in the caption. An enhanced encoder-decoder Transformer model with CNN for feature extraction has been used in our work. This model captures the long-range dependencies and temporal relationships more effectively. The model has been evaluated with benchmark datasets and compared with state-of-the-art methods and found to be slightly better in the performance. The performance scores are slightly varying for BLEU, METEOR, ROUGE a

关键词： curriculum learning deep learning encoder-decoder engineering applications neural networks transformer video caption

来源：评论

学校读者我要写书评

暂无评论

Multi-Source Interactive Stair Attention for Remote Sensing Image Captioning

引用

REMOTE SENSING 2023年第3期15卷 579-579页

作者： Zhang, Xiangrong Li, Yunpeng Wang, Xin Liu, Feixiang Wu, Zhaoji Cheng, Xina Jiao, Licheng Xidian Univ Sch Artificial Intelligence Key Lab Intelligent Percept & Image Understanding Minist Educ Xian 710071 Peoples R China

The aim of remote sensing image captioning (RSIC) is to describe a given remote sensing image (RSI) using coherent sentences. Most existing attention-based methods model the coherence through an LSTM-based decoder, which dynamically infers a word vector from preceding sentences. However, these methods are indirectly guided through the confusion of attentive regions, as (1) the weighted average in the attention mechanism distracts the word vector from capturing pertinent visual regions and (2) there are few constraints or rewards for learning long-range transitions. In this paper, we propose a multi-source interactive stair attention mechanism that separately models the semantics of preceding sentences and visual regions of interest. Specifically, the multi-source interaction takes previous semantic vectors as queries and applies an attention mechanism on regional features to acquire the next word vector, which reduces immediate hesitation by considering linguistics. The stair attention divides the attentive weights into three levels-that is, the core region, the surrounding region, and other regions-and all regions in the search scope are focused on differently. Then, a CIDEr-based reward reinforcement learning is devised, in order to enhance the quality of the generated sentences. Comprehensive experiments on widely used benchmarks (i.e., the Sydney-Captions, UCM-Captions, and RSICD data sets) demonstrate the superiority of the proposed model over state-of-the-art models, in terms of its coherence, while maintaining high accuracy.

关键词： remote sensing image captioning cross-modal interaction attention mechanism semantic information encoder-decoder

来源：评论

学校读者我要写书评

暂无评论

Using various pre-trained models for audio feature extraction in automated audio captioning

引用

EXPERT SYSTEMS WITH APPLICATIONS 2023年第1期231卷

作者： Won, Hyejin Kim, Baekseung Kwak, Il-Youp Lim, Changwon Chung Ang Univ Dept Appl Stat Seoul 06974 South Korea Chung Ang Univ Inst Community Care & Hlth Equ Seoul 06974 South Korea

The DCASE automated audio captioning challenge aimed to construct a model that generates captions describing given audio. Our team developed a CNN14 encoder (pre-trained on AudioSet data) along with a Transformer decoder model that ranked sixth place in the competition. Many teams utilized pre-trained networks, and it was evident that more research into their utilization was required. This paper presented comprehensive experiments conducted with various encoder networks for the proposed system, including CNN10, CNN14 ResNet54, AST, VGGNet, and EfficientNet. The pre-trained networks of CNN10, CNN14, ResNet54, and AST were trained on AudioSet data, while the pre-trained networks of AST, VGGNet, and EfficientNet were trained on ImageNet data. The best outcomes were achieved when the pre-trained CNN10, trained on AudioSet data, was utilized as an encoder with the Transformer serving as a decoder, and fine-tuning applied. Moreover, a qualitative study confirmed that our model generates plausible captions for different types of audio.

关键词： Audio captioning Acoustic scene detection Transfer learning encoder-decoder Convolutional neural network Transformer

来源：评论

学校读者我要写书评

暂无评论

FBC-ANet: A Semantic Segmentation Model for UAV Forest Fire Images Combining Boundary Enhancement and Context Awareness

引用

DRONES 2023年第7期7卷

作者： Zhang, Lin Wang, Mingyang Ding, Yunhong Wan, Tingting Qi, Bo Pang, Yutian Northeast Forestry Univ Coll Informat & Comp Engn Harbin 150040 Peoples R China Heilongjiang Inst Construction Technol Dept Informat Engn Harbin 150025 Peoples R China Harbin Inst Technol Sch Astronaut Harbin 150010 Peoples R China Arizona State Univ Sch Engn Matter Transport & Energy Tempe AZ 85287 USA

Forest fires are one of the most serious natural disasters that threaten forest resources. The early and accurate identification of forest fires is crucial for reducing losses. Compared with satellites and sensors, unmanned aerial vehicles (UAVs) are widely used in forest fire monitoring tasks due to their flexibility and wide coverage. The key to fire monitoring is to accurately segment the area where the fire is located in the image. However, for early forest fire monitoring, fires captured remotely by UAVs have the characteristics of a small area, irregular contour, and susceptibility to forest cover, making the accurate segmentation of fire areas from images a challenge. This article proposes an FBC-ANet network architecture that integrates boundary enhancement modules and context-aware modules into a lightweight encoder-decoder network. FBC-Anet can extract deep semantic features from images and enhance shallow edge features, thereby achieving an effective segmentation of forest fire areas in the image. The FBC-ANet model uses an Xception network as the backbone of an encoder to extract features of different scales from images. By transforming the extracted deep semantic features through the CIA module, the model's feature learning ability for fire pixels is enhanced, making feature extraction more robust. FBC-ANet integrates the decoder into the BEM module to enhance the extraction of shallow edge features in images. The experimental results indicate that the FBC-ANet model has a better segmentation performance for small target forest fires compared to the baseline model. The segmentation accuracy on the dataset FLAME is 92.19%, the F1 score is 90.76%, and the IoU reaches 83.08%. This indicates that the FBC-ANet model can indeed extract more valuable features related to fire in the image, thereby better segmenting the fire area from the image.

关键词： forest fires semantic segmentation boundary enhancement contextual information awareness encoder-decoder

来源：评论

学校读者我要写书评

暂无评论

Efficient hand segmentation for rehabilitation tasks using a convolution neural network with attention

引用

EXPERT SYSTEMS WITH APPLICATIONS 2023年第1期234卷

作者： Dutta, H. Pallab Jyoti Bhuyan, M. K. Neog, Debanga Raj Macdorman, Karl Fredric Laskar, Rabul Hussain Indian Inst Technol Dept Elect & Elect Engn Gauhati 781039 Assam India Indian Inst Technol Mehta Family Sch Data Sci & Artificial Intelligenc Gauhati 781039 Assam India Indiana Univ Luddy Sch Informat Comp & Engn Indianapolis IN 46202 USA Natl Inst Technol Silchar Dept Elect & Commun Engn Silchar 788010 Assam India

We designed an interface to support hand rehabilitation tasks to restore hand function and relieve discomfort. The interface requires accurate hand segmentation, which is impeded by background clutter, occlusion, and variations in illumination. To overcome these challenges, we propose a novel encoder-decoder that segments the hand by encoding spatial and channel correlations using two attention blocks. This approach requires much less computation than benchmark self-attention mechanisms. Moreover, a novel loss function optimizes the model to resolve class imbalance, ensure boundary smoothness, and retain the hand's shape. The quantitative and qualitative results show the model's ability to segment the hands. It performed exceptionally well for images with different hand poses and orientations, the presence of a human face, background clutter, specularity, and variations in illumination. The model attained an F1-score of 97.3% for the Ouhands and 99.3% for the HGR dataset, higher than baseline models, with faster inference times. Furthermore, the model could generalize hand segmentation to multiple hands and unseen environments. Its segmentation precision enabled the development of the hand rehabilitation interface, which guided users to perform hand exercises. For five weeks, patients steadily improved hand function while using the interface.

关键词： Channel attention Efficient attention mechanism encoder-decoder Hand rehabilitation Hand segmentation Spatial attention

来源：评论

学校读者我要写书评

暂无评论

Scale-aware spatial pyramid pooling with both encoder-mask and scale-attention for semantic segmentation

引用

NEUROCOMPUTING 2020年第0期383卷 174-182页

作者： Zhou, Feng Hu, Yong Shen, Xukun Beihang Univ State Key Lab Virtual Real Technol & Syst Beijing Peoples R China Beihang Univ Sch New Media Art & Design Beijing Peoples R China

This paper focuses on semantic segmentation of scenes by capturing the appropriate scale, rich detail and contextual dependencies in a feature representation. Semantic segmentation is a pixel-level classification task and has made steady progress on the basis of fully convolutional networks (FCNs). However, we find there is still room for improvement in the following aspects. The first is that the pixel itself has not enough information for semantic prediction, it needs to look around to determine which category it belongs to. However, the fixed size of the receptive field defined by the network is not suitable for all pixels when an image contains objects with various scales. The second is that the extracted scale-aware features do not handle sharper object boundaries due to low-resolution. The final aspect is regarding the ability to model long-range dependencies. In order to solve the above challenges, in this paper we propose three modules: Scale-aware spatial pyramid pool module, encoder mask module and Scale-attention module (SSPP-ES). Extensive experiments on the Cityscapes and ADE20K benchmarks demonstrate the effectiveness of our approach for semantic segmentation. (C) 2019 Elsevier B.V. All rights reserved.

关键词： Scene understanding Semantic segmentation encoder-decoder Scale selection Attention

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：