检索结果-内蒙古大学图书馆

CFN: A coarse-to-fine network for eye fixation prediction

IET IMAGE PROCESSING 2022年第9期16卷 2373-2383页

作者： Xu, Binwei Liang, Haoran Liang, Ronghua Chen, Peng Zhejiang Univ Technol Coll Comp Sci & Technol Hangzhou 31023 Peoples R China

Many image-to-image computer vision approaches have made great progress by an end-to-end framework with the encoder-decoder architecture. However, the same image-to-image eye fixation prediction task is not the same as those computer vision tasks in that it focuses more on salient regions rather than precise predictions for every pixel. Thus, it is not appropriate to directly apply the end-to-end encoder-decoder to the eye fixation prediction task. In addition, although high-level feature is important, the contribution of low-level feature should also be kept and balanced in computational model. Nevertheless, some low-level features that attract attention are easily neglected while transiting through the deep network. Therefore, the effective way to integrate low-level and high-level features for improving eye fixation prediction performance is still a challenging task. In this paper, a coarse-to-fine network (CFN) that encompasses two pathways with different training strategies are proposed: coarse perceiving network (CFN-Coarse) can be a simple encoder network or any of the existing pretrained network to capture the distribution of salient regions and generate high-quality feature maps;fine integrating network (CFN-Fine) uses fixed parameters from the CFN-Coarse and combines features from deep to shallow in the deconvolution process by adding skip connections between down-sampling and up-sampling paths to efficiently integrate deep and shallow features. The saliency map obtained by the method is evaluated over 6 standard benchmark datasets, namely SALICON, MIT1003, MIT300, Toronto, OSIE, and SUN500. The results demonstrate that the method can surpass the state-of-the-art accuracy of eye fixation prediction and achieves the competitive performance to date under most evaluation metrics on SALICON Saliency Prediction Challenge (LSUN2017).

关键词： OSIE dataset saliency map SALICON Saliency Prediction Challenge low-level feature CFN-Fine image-to-image eye fixation prediction task SALICON dataset deconvolution image-to-image computer vision convolutional neural nets MIT300 dataset high-quality feature map generation MIT1003 dataset salient region distribution image sampling feature extraction CFN-Coarse deep learning (artificial intelligence) image segmentation SUN500 dataset computer vision coarse perceiving network Toronto dataset high-level feature down-sampling path up-sampling path deconvolution process coarse-to-fine network encoder-decoder architecture deep network

来源：评论

学校读者我要写书评

暂无评论

GNet: An integrated context-aware neural framework for transcription factor binding signal at single nucleotide resolution prediction

引用

MATHEMATICAL BIOSCIENCES AND ENGINEERING 2023年第9期20卷 15809-15829页

作者： Zhuang, Jujuan Feng, Kexin Teng, Xinyang Jia, Cangzhi Dalian Maritime Univ Sch Sci Dalian 116026 Liaoning Peoples R China

Transcription factors (TFs) are important factors that regulate gene expression. Revealing the mechanism affecting the binding specificity of TFs is the key to understanding gene regulation. Most of the previous studies focus on TF-DNA binding sites at the sequence level, and they seldom utilize the contextual features of DNA sequences. In this paper, we develop an integrated spatiotemporal context-aware neural network framework, named GNet, for predicting TF-DNA binding signal at single nucleotide resolution by achieving three tasks: single nucleotide resolution signal prediction, identification of binding regions at the sequence level, and TF-DNA binding motif prediction. GNet extracts implicit spatial contextual information with a gated highway neural mechanism, which captures large context multi-level patterns using linear shortcut connections, and the idea of it permeates the encoder and decoder parts of GNet. The improved dual external attention mechanism, which learns implicit relationships both within and among samples, and improves the performance of the model. Experimental results on 53 human TF ChIP-seq datasets and 6 chromatin accessibility ATAC-seq datasets shows that GNet outperforms the state-of-the-art methods in the three tasks, and the results of cross-species studies on 15 human and 18 mouse TF datasets of the corresponding TF families indicate that GNet also shows the best performance in cross-species prediction over the competitive methods.

关键词： transcription factor binding site gated highway neural network encoder-decoder architecture external attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Learning and Adapting Robust Features for Satellite Image Segmentation on Heterogeneous Data Sets

引用

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2019年第9期57卷 6517-6529页

作者： Ghassemi, Sina Fiandrotti, Attilio Francini, Gianluca Magli, Enrico Polytech Univ Turin Elect & Telecommun Dept I-10129 Turin Italy Univ Paris Saclay Telecom Paristech Image Data Signals Dept Multimedia Grp St Aubin France Politecn Torino Telecom Italia I-10148 Turin Italy

This paper addresses the problem of training a deep neural network for satellite image segmentation so that it can be deployed over images whose statistics differ from those used for training. For example, in postdisaster damage assessment, the tight time constraints make it impractical to train a network from scratch for each image to be segmented. We propose a convolutional encoder- decoder network able to learn visual representations of increasing semantic level as its depth increases, allowing it to generalize over a wider range of satellite images. Then, we propose two additional methods to improve the network performance over each specific image to be segmented. First, we observe that updating the batch normalization layers' statistics over the target image improves the network performance without human intervention. Second, we show that refining a trained network over a few samples of the image boosts the network performance with minimal human intervention. We evaluate our architecture over three data sets of satellite images, showing the state-of-the-art performance in binary segmentation of previously unseen images and competitive performance with respect to more complex techniques in a multiclass segmentation task.

关键词： Convolutional neural network (CNN) deep learning domain adaptation encoder-decoder architecture satellite image segmentation

来源：评论

学校读者我要写书评

暂无评论

Attribute-Driven Filtering: A new attributes predicting approach for fine-grained image captioning

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2024年第PartA期137卷

作者： Hossen, Md. Bipul Ye, Zhongfu Abdussalam, Amr Ul Hassan, Shabih Univ Sci & Technol China Sch Informat Sci & Technol Hefei 230027 Anhui Peoples R China

Fine-grained image captioning with attribute information has garnered significant attention in the realms of computer vision and natural language processing, demanding precise and contextually relevant descriptions of visual content. While previous attribute-driven image captioning models have shown improvements, challenges remain, such as the independence of attribute predictors and caption generators and the semantic gap between images and attributes. Another common issue is the inclusion of all attributes at every time step, despite most attributes being irrelevant to the word currently being generated. This can divert the model's attention toward erroneous semantic details, resulting in a performance decline. To address these issues, we propose a novel Attribute-Driven Filtering (ADF) captioning network designed to provide rich and nuanced descriptions. This model incorporates a unique Attribute Predictor Module (APM) that dynamically predicts the most pertinent attributes in accordance with the textual context, utilizing different attributes at various time steps. The novelty of this approach lies in recognizing that not all attributes hold equal relevance at each time step, and the APM filters out irrelevant attributes to generate precise and contextually relevant captions. Furthermore, this model features a fusion mechanism that integrates visual information from a conventional attention module with attribute information predicted by the APM, aiming to reduce the visual semantic gap between images and attributes. Extensive experimentation demonstrates that the ADF model outperforms advanced models, achieving impressive CIDEr-D scores of 72.0 (Flickr30K) and 123.3 (MS-COCO) through reinforcement learning optimization. It consistently surpasses baseline models across diverse evaluation metrics, highlighting its effectiveness and robustness.

关键词： Fine-grained captioning Fusion mechanism encoder-decoder architecture Attribute predictor module

来源：评论

学校读者我要写书评

暂无评论

A Mono SLAM Method Based on Depth Estimation by DenseNet-CNN

引用

IEEE SENSORS JOURNAL 2022年第3期22卷 2447-2455页

作者： Jin, Yifan Yu, Lei Chen, Zhong Fei, Shumin Huaiyin Inst Technol Jiangsu Key Lab Adv Mfg Technol Huaian 223003 Peoples R China Soochow Univ Sch Mech & Elect Engn Suzhou 215000 Peoples R China Southeast Univ Sch Automat Nanjing 210000 Peoples R China

Currently, SLAM (simultaneous localization and mapping) systems based on monocular cameras cannot directly obtain depth information, and most of them have problems with scale uncertainty and need to be initialized. In some application scenarios that require navigation and obstacle avoidance, the inability to achieve dense mapping is also a defect of monocular SLAM. In response to the above problems, this paper proposes a method which learns depth estimation by DenseNet and CNN for a monocular SLAM system. We use an encoder-decoder architecture based on transfer learning and convolutional neural networks to estimate the depth information of monocular RGB images. At the same time, through the front-end ORB feature extraction and the back-end direct RGB-D Bundle Adjustment optimization method, it is possible to obtain accurate camera poses and achieve dense indoor mapping when using estimated depth information. The experimental results show that the monocular depth estimation model used in this paper can achieve good results, and it is also competitive in comparison with the current popular methods. On this basis, the error of camera pose estimation is also smaller than traditional monocular SLAM solutions and can complete the dense indoor reconstruction task. It is a complete SLAM system based on monocular camera.

关键词： Simultaneous localization and mapping Estimation Cameras Convolutional neural networks Sensors Image reconstruction Feature extraction Monocular depth estimation encoder-decoder architecture transfer learning camera pose estimation dense mapping

来源：评论

学校读者我要写书评

暂无评论

LiCENt: Low-Light Image Enhancement Using the Light Channel of HSL

引用

IEEE ACCESS 2022年 10卷 33547-33560页

作者： Garg, Atik Pan, Xin-Wen Dung, Lan-Rong Natl Yang Ming Chiao Tung Univ EECS Int Grad Program Hsinchu 30010 Taiwan Natl Yang Ming Chiao Tung Univ Dept Elect & Comp Engn Hsinchu 30010 Taiwan

Images captured in low-brightness environments often lead to poor visibility and exhibit artifacts such as low brightness, low contrast, and color distortion. These artifacts not only affect the visual perception of the human eye but also decrease the performance of computer vision algorithms. Existing deep learning-based image enhancements studies are quite slow and usually require extensive hardware specifications. Conversely, lightweight enhancement approaches do not provide satisfactory performance as compared to state-of-the-art methods. Therefore, we proposed a fast and lightweight deep learning-based algorithm for performing low-light image enhancement using the light channel of Hue Saturation Lightness (HSL). LiCENt stands for Light Channel Enhancement Network that uses a combination of an autoencoder and convolutional neural network (CNN) to train a low-light enhancer to first improve the illumination and later improve the details of the low-light image in a unified framework. This method used a single channel lightness 'L' of HSL color space instead of traditional RGB color channels which helps in reducing the number of learnable parameters by a factor of 8.92, at the most. LiCENt also has significant advantages for the Brilliance Perception Adjustment, which enables the model to avoid issues including over-enhancement and color distortion. The experimental results demonstrate that our approach generalizes well in synthetic and natural low-light images and outperforms other methods in terms of qualitative and quantitative metrics.

关键词： Image color analysis Lighting Image enhancement Visualization Training Generative adversarial networks Convolutional neural networks Deep learning image processing low-light enhancement encoder-decoder architecture image enhancement computer vision convolutional neural network

来源：评论

学校读者我要写书评

暂无评论

C-LIENet: A Multi-Context Low-Light Image Enhancement Network

引用

IEEE ACCESS 2021年 9卷 31053-31064页

作者： Ravirathinam, Praveen Goel, Divyam Ranjani, J. Jennifer Birla Inst Technol & Sci Dept Comp Sci & Informat Syst Pilani 333031 Rajasthan India

Enhancement of low-light images is a challenging task due to the impact of low brightness, low contrast, and high noise. The inability to collect natural labeled data intensifies this problem further. Many researchers have attempted to solve this problem using learning-based approaches;however, most models ignore the impact of noise in low-lit images. In this paper, an encoder-decoder architecture, made up of separable convolution layers that solve the issues encountered in low-light image enhancement, is proposed. The architecture is trained end-to-end on a custom low-light image dataset (LID), comprising both clean and noisy images. We introduce a unique multi-context feature extraction module (MC-FEM) where the input first passes through a feature pyramid of dilated separable convolutions for hierarchical-context feature extraction followed by separable convolutions for feature compression. The model is optimized using a novel three-part loss function that focuses on high-level contextual features, structural similarity, and patch-wise local information. We conducted several ablation studies to determine the optimal model for low-light image enhancement under noisy and noiseless conditions. We have used performance metrics like peak-signal-to-noise ratio, structural similarity index matrix, visual information fidelity, and average brightness to demonstrate the superiority of the proposed work against the state-of-the-art algorithms. Qualitative results presented in this paper prove the strength and suitability of our model for real-time applications.

关键词： Feature extraction Convolution Image enhancement Noise measurement Lighting Image coding Computer architecture encoder-decoder architecture separable convolution dilated convolution ASPP perceptual loss low-light image enhancement

来源：评论

学校读者我要写书评

暂无评论

Beyond the Status Quo: A Contemporary Survey of Advances and Challenges in Audio Captioning

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2024年 32卷 95-112页

作者： Xu, Xuenan Xie, Zeyu Wu, Mengyue Yu, Kai Shanghai Jiao Tong Univ AI Inst MoE Key Lab Artificial Intelligence X LANCE LabDept Comp Sci & Engn Shanghai 200240 Peoples R China

Automated audio captioning (AAC), a task that mimics human perception as well as innovatively links audio processing and natural language processing, has overseen much progress over the last few years. AAC requires recognizing contents such as the environment, sound events and the temporal relationships between sound events and describing these elements with a fluent sentence. Currently, an encoder-decoder-based deep learning framework is the standard approach to tackle this problem. Plenty of works have proposed novel network architectures and training schemes, including extra guidance, reinforcement learning, audio-text self-supervised learning and diverse or controllable captioning. Effective data augmentation techniques, especially based on large language models are explored. Benchmark datasets and AAC-oriented evaluation metrics also accelerate the improvement of this field. This article situates itself as a comprehensive survey covering the comparison between AAC and its related tasks, the existing deep learning techniques, datasets, and the evaluation metrics in AAC, with insights provided to guide potential future research directions.

关键词： Automated audio captioning audio recognition encoder-decoder architecture evaluation metrics natural language generation training schemes

来源：评论

学校读者我要写书评

暂无评论

LARFNet: Lightweight asymmetric refining fusion network for real-time semantic segmentation

引用

COMPUTERS & GRAPHICS-UK 2022年 109卷 55-64页

作者： Hu, Xuegang Gong, Juelin Chongqing Univ Posts & Telecommun Sch Commun & Informat Engn Chongqing 400065 Peoples R China Chongqing Univ Posts & Telecommun Chongqing Key Lab Signal & Informat Proc Chongqing 400065 Peoples R China

In this paper, we propose a lightweight asymmetric refining fusion network (LARFNet) for real-time semantic segmentation to solve the problem that some existing models cannot achieve good segmentation accuracy with real-time inference speed in mobile devices due to the huge computational overhead. Specifically, LARFNet adopts an asymmetric encoder-decoder structure. The depth-wise separable asymmetric interaction module (DSAI module) is designed in the encoding process, which effectively extracted local and surrounding information under different receptive fields with optimized convolution in the condition of ensuring communication between channels. In the decoder, we design the bilateral pyramid pooling attention module (BPPA module) and the multi-stage refinement fusion module (MRF Module). The BPPA module is used to integrate the high-level output multi-scale context information. Based on spatial and channel attention mechanisms, the MRF module is proposed to refine the feature maps of different resolutions and guide the feature fusion. Experimental results show that LARFNet achieves 69.2% mIoU and 65.6% mIoU on Cityscapes and Camvid datasets at 127 FPS and 222 FPS respectively, only using a single NVIDIA GeForce GTX2080Ti GPU and 0.72M parameters without any pre-training or pre-processing. Compared with most of the existing state-of-the-art models, the proposed method realizes the efficient use of network parameters at a faster speed, reduces the number of network parameters, and still achieves the accuracy of good segmentation.(c) 2022 Elsevier Ltd. All rights reserved.

关键词： Attention mechanism Lightweight network encoder-decoder architecture Real-time semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

Semantic Scene Object-Camera Motion Recognition for Scene Transition Detection Using Dense Spatial Frame Segments and Temporal Trajectory Analysis

引用

IEEE ACCESS 2024年 12卷 21673-21697页

作者： Chakraborty, Dipanita Chiracharit, Werapon Chamnongthai, Kosin King Mongkuts Univ Technol Thonburi Dept Elect & Telecommun Engn Bangkok 10140 Thailand

Semantic video scene-understanding applications rely on object-camera motion recognition techniques for scene contextual movement representation. While existing machine learning-based methods perform efficiently, their primary limitation is to analyze motion patterns from normal frames only, neglecting the scene transition frames. This causes significant false alarms due to the undetected object-camera motion patterns during scene transitions. In this paper, we propose a novel method for object and camera motion recognition of two consecutive scenes from their transition frames. First, our method detects cut transitions using principal component analysis (PCA) to segment the video into shots. Additionally, it eliminates large text transitions that are often falsely detected as cut transitions using structural similarity index measurement (SSIM) properties. Second, it selects candidate segments to localize normal and wipe transition frames using slope angle characteristics obtained from linear regression. Third, it extracts dense semantic spatial features at multi-scale using the modified DeepLabv3+ network to segment selected candidate frames into foreground, background, and wipe pixels. Finally, an optical flow algorithm-based temporal trajectory tracking model is applied on each segmented pixel to recognize the object, camera pan, zoom-in, and zoom-out motion patterns. We further remove falsely detected non-transition motion frames to improve wipe transition detection. The experimental results are obtained using the benchmark TRECVID and the multimedia datasets. The proposed method using pixel-level classification and temporal trajectory analysis achieved an average accuracy improvement of 9.28% for object-camera motion recognition, 3.75% for cut transition detection, and 3.01% for wipe transition detection.

关键词： encoder-decoder architecture frame segmentation object and camera motion recognition motion pattern scene change detection shot boundary detection transition pattern video processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：