检索结果-内蒙古大学图书馆

Transformer-Based Reinforcement Learning for Pickup and Delivery Problems With Late Penalties

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2022年第12期23卷 24649-24661页

作者： Zhang, Ke Lin, Xi Li, Meng Tsinghua Univ Dept Civil Engn Beijing 100084 Peoples R China

Pickup and delivery problems with late penalties can be adopted to model a wide range of practical situations in the field of transportation and logistics. However, the restrictions on the multiple vehicles' service sequences and non-linearity caused by the late penalties make it time-consuming to solve this problem. To overcome this difficulty, we propose a novel reinforcement learning framework inspired by transformer architecture to generate tours instantly after offline training. This framework, as trained through the policy gradient method, consists of the information encoder process which can extract the coupling relationships among the pickup and delivery customers, and the decoder process with multi-vehicle attention network to allocate reasonable orders to each vehicle. Validated on Sioux Falls network, the proposed method yields the improvement of 2.4%-8.0% on the solution quality compared with Google OR-Tools and several heuristic algorithms. Notably, the baselines require dozens of minutes to achieve a lesser result on the case with 100 customers while the well-trained model based on our method can be deployed to provide a high-quality solution within seconds. Furthermore, the proposed model also shows good generalization ability in different scenarios with various scale problems, and the obtained results are shown to be quite robust to counter the fluctuation of travel time.

关键词： Transformers Routing Reinforcement learning Costs Logistics Transportation Heuristic algorithms Pickup and delivery problem deep reinforcement learning multiple agents encoder-decoder framework

来源：评论

学校读者我要写书评

暂无评论

Video captioning: a review of theory, techniques and practices

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2022年第25期81卷 35619-35653页

作者： Jain, Vanita Al-Turjman, Fadi Chaudhary, Gopal Nayar, Devang Gupta, Varun Kumar, Aayush Bharati Vidyapeeths Coll Engn New Delhi India Near East Univ Nicosia Cyprus

In today's world, video captioning is extensively used in various applications for specially-abled and, more specifically, visually abled persons. With advancements in technology for object detection and natural processing, there has been an instant surge infusing the above mainstream tasks. One such example of this fusion resulted in the generation of Image captions when an input image is fed to the system, and it gives a short description of what is present in the image. This fusion pertained to images and was further moved to be implemented on the Videos, with some tweaking in the current methods. This paper presents the survey of the state of art techniques of various video captioning methods. There have been many inputs provided by people worldwide in this domain;thus, there was a need to compile, study and analyze all the results and present that in a comprehensive study, which we have done in this paper. The comparison of various video captioning methods on the distinct dataset was evaluated on different parameters, which were most common and mainly used for image and video analysis. This review was done for methods used from the year 2015-2019 (year by year). The most commonly used dataset and evaluation method are also pictorially represented in a bar graph and scatter plot for each year for the respective evaluation parameter. Though a lot of analysis and research has been done on video captioning, our survey shows many problems.

关键词： Video captioning Natural language processing CNN RNN encoder-decoder framework

来源：评论

学校读者我要写书评

暂无评论

Convolutional Reconstruction-to-Sequence for Video Captioning

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2020年第11期30卷 4299-4308页

作者： Wu, Aming Han, Yahong Yang, Yi Hu, Qinghua Wu, Fei Tianjin Univ Coll Intelligence & Comp Tianjin 300072 Peoples R China Univ Technol Sydney Sch Comp Sci Sydney NSW 2007 Australia Zhejiang Univ Coll Comp Sci Hangzhou 310027 Peoples R China

Recent advances towards video captioning mainly follow an encoder-decoder (sequence-to-sequence) framework and generate captions via a recurrent neural network (RNN). However, employing RNN as the decoder (generator) is prone to diluting long-term information, which weakens its ability to capture long-term dependencies. Recently, some work has demonstrated that the convolutional neural network (CNN) could be used to model sequential information. Though strengths in representation ability and computation efficiency, CNN has not been well exploited in video captioning. The reason partially comes from the difficulty of modeling multi-modal sequence with CNN. In this paper, we devise a novel CNN-based encoder-decoder framework for video captioning. Particularly, we first append inter-frame differences to each CNN-extracted frame feature to get a more discriminative representation;then with that as the input, we encode each frame to be a more compact feature by a one-layer convolutional mapping, which could be taken as a reconstruction network. In the decoding stage, we first fuse visual and lexical feature;then we stack multiple dilated convolutional layers to form a hierarchical decoder. As long-term dependencies could be captured by a shorter path along the hierarchical structure, the decoder could alleviate the loss of long-term information. Experiments on two benchmark datasets show that our method could obtain state-of-the-art performance.

关键词： Video captioning encoder-decoder framework reconstruction dilated convolution hierarchical decoder

来源：评论

学校读者我要写书评

暂无评论

Integrating Scene Semantic Knowledge into Image Captioning

引用

ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 2021年第2期17卷 1–22页

作者： Wei, Haiyang Li, Zhixin Huang, Feicheng Zhang, Canlong Ma, Huifang Shi, Zhongzhi Guangxi Normal Univ Guangxi Key Lab Multisource Informat Min & Secur 15 Yucai Rd Guilin 541004 Guangxi Peoples R China Northwest Normal Univ Coll Comp Sci & Engn 967 Anning East Rd Lanzhou 730070 Gansu Peoples R China Chinese Acad Sci Inst Comp Technol Key Lab Intelligent Informat Proc 6 Kexueyuan South Rd Beijing 100190 Peoples R China

Most existing image captioning methods use only the visual information of the image to guide the generation of captions, lack the guidance of effective scene semantic information, and the current visual attention mechanism cannot adjust the focus intensity on the image. In this article, we first propose an improved visual attention model. At each timestep, we calculated the focus intensity coefficient of the attention mechanism through the context information of themodel, then automatically adjusted the focus intensity of the attention mechanism through the coefficient to extract more accurate visual information. In addition, we represented the scene semantic knowledge of the image through topic words related to the image scene, then added them to the language model. We used the attention mechanism to determine the visual information and scene semantic information that the model pays attention to at each timestep and combined them to enable the model to generate more accurate and scene-specific captions. Finally, we evaluated our model on Microsoft COCO (MSCOCO) and Flickr30k standard datasets. The experimental results show that our approach generates more accurate captions and outperforms many recent advanced models in various evaluation metrics.

关键词： Image captioning attention mechanism scene semantics encoder-decoder framework

来源：评论

学校读者我要写书评

暂无评论

Joint Embedding of Deep Visual and Semantic Features for Medical Image Report Generation

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2023年 25卷 167-178页

作者： Yang, Yan Yu, Jun Zhang, Jian Han, Weidong Jiang, Hanliang Huang, Qingming Hangzhou Dianzi Univ Sch Comp Sci & Technol Key Lab Complex Syst Modeling & Simulat Hangzhou 310018 Peoples R China Zhejiang Int Studies Univ Hangzhou 310018 Peoples R China Zhejiang Univ Sir Run Run Shaw Hosp Coll Med Dept Med Oncol Hangzhou 310016 Peoples R China Zhejiang Univ Sir Run Run Shaw Hosp Reg Med Ctr Natl Inst Resp DisColl Med Hangzhou 310016 Peoples R China Univ Chinese Acad Sci Sch Comp & Control Engn Beijing 101408 Peoples R China

Medical image report generation (MeIRG) aims at generating associated diagnosis descriptions with natural language sentences from medical images, which is essential in the computer-aided diagnosis system. Nevertheless, this task remains challenging in that medical images and linguistic expressions should be understood jointly which however show great discrepancies in the modality. To fill this visual-to-semantic gap, we propose a novel framework that follows the encoder-decoder pipeline. Our framework is characterized by encoding both deep visual and semantic embeddings through a triple-branch network (TriNet) during the encoding phase. The visual attention branch captures attended visual embeddings from medical images with the soft-attention mechanism. The medical report (MeRP) embedding branch predicts semantic report embeddings. The embedding branch of medical subject headings (MeSH) obtains semantic embeddings of related medical tags as complementary information. Then, outputs of these branches are fused and fed into a decoder for the report generation. Experimental results on two benchmark datasets have demonstrated the excellent performance of our method. Related codes are available at https://***/yangyan22/Medical-Report-Generation-TriNet.

关键词： Medical image report generation visual-semantic joint embedding encoder-decoder framework deep neural network

来源：评论

学校读者我要写书评

暂无评论

Enhancing Soil Moisture Forecasting Accuracy with REDF-LSTM: Integrating Residual En-Decoding and Feature Attention Mechanisms

引用

WATER 2024年第10期16卷

作者： Li, Xiaoning Zhang, Ziyin Li, Qingliang Zhu, Jinlong Changchun Normal Univ Coll Comp Sci & Technol Changchun 130032 Peoples R China

This study introduces an innovative deep learning model, Residual-EnDecode-Feedforward Attention Mechanism-Long Short-Term Memory (REDF-LSTM), designed to overcome the high uncertainty challenges faced by traditional soil moisture prediction methods. The REDF-LSTM model, by integrating a residual learning encoder-decoder LSTM layer, enhanced LSTM layers, and feedforward attention, not only captures the deep features of time series data but also optimizes the model's ability to identify key influencing factors, including land surface features, atmospheric conditions, and other static environmental variables. Unlike existing methods, the innovation of this model lies in its first-time combination of the residual learning encoder-decoder and feedforward attention mechanisms in the soil moisture prediction field. It delves into the complex patterns of time series through the encoder-decoder structure and accurately locates key influencing factors through the feedforward attention mechanism, significantly improving predictive performance. The choice to combine the feedforward attention mechanism and encoder-decoder with the LSTM model is to fully leverage their advantages in processing complex data sequences and enhancing the model's focus on important features, aiming for more accurate soil moisture prediction. After comparison with current advanced models such as EDLSTM, FAMLSTM, and GANBiLSTM, our REDF-LSTM demonstrated the best performance. Compared to traditional LSTM models, it achieved an average improvement of 13.07% in R2, 20.98% in RMSE, 24.86% in BIAS, and 11.1% in KGE key performance indicators, fully proving its superior predictive capability and potential application value in precision agriculture and ecosystem management.

关键词： soil moisture prediction long short-term memory networks encoder-decoder framework feedforward attention mechanism residual learning

来源：评论

学校读者我要写书评

暂无评论

HSGNet: hierarchically stacked graph network with attention mechanism for 3D human pose estimation

引用

MULTIMEDIA SYSTEMS 2023年第4期29卷 2085-2097页

作者： Yang, Honghong Liu, Hongxi Zhang, Yumei Wu, Xiaojun Shaanxi Normal Univ Key Lab Modern Teaching Technol Minist Educ Xian 710062 Peoples R China Shaanxi Normal Univ Key Lab Intelligent Comp & Serv Technol Folk Song Minist Culture & Tourism Xian Peoples R China Shaanxi Normal Univ Sch Comp Sci Xian 710062 Peoples R China

With the powerful representative ability of learning human skeleton, the graph convolutional network (GCN) is a popular baseline for 3D human pose estimation (HPE). However, current GCN-based 3D HPE methods primarily use "message-passing" architectures to aggregate the node information through the edges of the graph at "one scale". In such architectures, the learnt node features are uniform and cannot learn hierarchical representation of the graph-structured data. In this study, a hierarchically stacked graph network (HSGNet) with attention constraint for 3D HPE was proposed. An attention-constrained GCN layer (AGCN) was designed as the basic unit for constructing the HSGNet. With the specially designed AGCN layer, we computed the attention coefficients for each node to pick the most important node and suppressed the redundant information from the neighbors in feature aggregation. Then, a coarse graph layer with pooling map was devised for stacking the multiple GCN layers in a hierarchical manner, where a pooling map matrix was used to cluster the nodes for graph representation according to the human skeleton structure. Finally, an HSGNet was constructed in an encoder-decoder framework to further embed the global and local information of the full skeleton to achieve the final embedding feature for 3D pose regression. Our method was validated on two benchmark datasets: Human3.6M and MPI-INF-3DHP. Experimental results showed that the proposed method yielded good performance for 3D HPE.

关键词： Pose estimation Hierarchical stacked graph network Attention mechanism encoder-decoder framework

来源：评论

学校读者我要写书评

暂无评论

Automated audio captioning: an overview of recent progress and new challenges

引用

EURASIP JOURNAL ON AUDIO SPEECH AND MUSIC PROCESSING 2022年第1期2022卷 1-18页

作者： Mei, Xinhao Liu, Xubo Plumbley, Mark D. Wang, Wenwu Univ Surrey Ctr Vision Speech & Signal Proc CVSSP Dept Elect & Elect Engn Fac Engn & Phys Sci Guildford GU2 7XH Surrey England

Automated audio captioning is a cross-modal translation task that aims to generate natural language descriptions for given audio clips. This task has received increasing attention with the release of freely available datasets in recent years. The problem has been addressed predominantly with deep learning techniques. Numerous approaches have been proposed, such as investigating different neural network architectures, exploiting auxiliary information such as keywords or sentence information to guide caption generation, and employing different training strategies, which have greatly facilitated the development of this field. In this paper, we present a comprehensive review of the published contributions in automated audio captioning, from a variety of existing approaches to evaluation metrics and datasets. We also discuss open challenges and envisage possible future research directions.

关键词： Audio captioning Deep learning Audio processing Natural language processing encoder-decoder framework

来源：评论

学校读者我要写书评

暂无评论

NumCap: A Number-controlled Multi-caption Image Captioning Network

引用

ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 2023年第4期19卷 1-24页

作者： Abdussalam, Amr Ye, Zhongfu Hawbani, Ammar Al-Qatf, Majjed Khan, Rashid Univ Sci & Technol China Sch Informat Sci & Technol Hefei 230027 Peoples R China Univ Sci & Technol China Sch Comp Sci & Technol Hefei 230027 Peoples R China

Image captioning is a promising task that attracted researchers in the last few years. Existing image captioning models are primarily trained to generate one caption per image. However, an image may contain rich contents, and one caption cannot express its full details. A better solution is to describe an image with multiple captions, with each caption focusing on a specific aspect of the image. In this regard, we introduce a new number-based image captioning model that describes an image with multiple sentences. An image is annotated with multiple ground-truth captions;thus, we assign an external number to each caption to distinguish its order. Given an image-number pair as input, we could achieve different captions for the same image under different numbers. First, a number is attached to the image features to form an image-number vector (INV). Then, this vector and the corresponding caption are embedded using the order-embedding approach. Afterward, the INV's embedding is fed to a language model to generate the caption. To show the efficiency of the numbers incorporation strategy, we conduct extensive experiments using MS-COCO, Flickr30K, and Flickr8K datasets. The proposed model attains 24.1 in METEOR on MS-COCO. The achieved results demonstrate that our method is competitive with a range of state-of-the-art models and validate its ability to produce different descriptions under different given numbers.

关键词： Numbers incorporation strategy encoder-decoder framework image captioning order-embedding

来源：评论

学校读者我要写书评

暂无评论

Zero-Shot Learning via Latent Space Encoding

引用

IEEE TRANSACTIONS ON CYBERNETICS 2019年第10期49卷 3755-3766页

作者： Yu, Yunlong Ji, Zhong Guo, Jichang Zhang, Zhongfei Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China SUNY Binghamton Watson Sch Dept Comp Sci Binghamton NY 13902 USA

Zero-shot learning (ZSL) is typically achieved by resorting to a class semantic embedding space to transfer the knowledge from the seen classes to unseen ones. Capturing the common semantic characteristics between the visual modality and the class semantic modality (e.g., attributes or word vector) is a key to the success of ZSL. In this paper, we propose a novel encoder-decoder approach, namely latent space encoding (LSE), to connect the semantic relations of different modalities. Instead of requiring a projection function to transfer information across different modalities like most previous work, LSE performs the interactions of different modalities via a feature aware latent space, which is learned in an implicit way. Specifically, different modalities are modeled separately but optimized jointly. For each modality, an encoder-decoder framework is performed to learn a feature aware latent space via jointly maximizing the recoverability of the original space from the latent space and the predictability of the latent space from the original space. To relate different modalities together, their features referring to the same concept are enforced to share the same latent codings. In this way, the common semantic characteristics of different modalities are generalized with the latent representations. Another property of the proposed approach is that it is easily extended to more modalities. Extensive experimental results on four benchmark datasets [animal with attribute, Caltech UCSID birds, aPY, and ImageNet] clearly demonstrate the superiority of the proposed approach on several ZSL tasks, including traditional ZSL, generalized ZSL, and zero-shot retrieval.

关键词： encoder-decoder framework latent space encoding (LSE) zero-shot learning (ZSL)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：