Optimizing quay crane scheduling using deep reinforcement learning with hybrid metaheuristic algorithm

作     者：Long, Le Ngoc Bao You, Sam-Sang Cuong, Truong Ngoc Kim, Hwan-Seong 

作者机构：Korea Maritime & Ocean Univ Dept Logist 727 Taejong Ro Busan 49112 South Korea Korea Maritime & Ocean Univ Northeast Asia Shipping & Port Logist Res Ctr 727 Taejong ro Busan 49112 South Korea Korea Maritime & Ocean Univ Div Mech Engn 727 Taejong Ro Busan 49112 South Korea Ho Chi Minh City Univ Technol HCMUT Fac Mech Engn Dept Mechatron 268 Ly Thuong Kiet StDist 10 Ho Chi Minh City Vietnam Vietnam Natl Univ Ho Chi Minh City Linh Trung Ward Ho Chi Minh City Vietnam 

出 版 物：《ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE》 (Eng Appl Artif Intell)

年 卷 期：2025年第143卷

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：The "Regional Innovation Strategy (RIS) " through the National Research Foundation of Korea (NRF) - Ministry of Education (MOE) 

主　　题：Quay crane scheduling problem Gradient-based policy learning Deep reinforcement learning Proximal policy optimization Greedy randomized adaptive search procedure ant colony optimization 

摘      要：As global trade via maritime transport increases annually, competition among seaports necessitates dynamic scheduling in operational management to optimize port performance. Quay crane scheduling problem (QCSP) is a typical optimization problem for container terminal operations on the quayside. This paper proposes a modern approach based on model-free deep reinforcement learning (DRL) named proximal policy optimization (PPO), a gradient-based framework with a shared actor-critic network structure. Besides, a novel hybrid metaheuristic combined with a greedy randomized adaptive search procedure (GRASP) and ant colony optimization (ACO) is employed as an alternative approach. The performance of the artificial intelligence (AI) -powered solutions is verified through numerical simulation for single- and multi-agent scenarios with different starting positions. The results highlight the intelligent approach s remarkable performance in solving complex optimization problems and its flexibility compared to traditional metaheuristics, as it does not require clear insights into the system s dynamics.