|Table of Contents|

Reinforcement learning modeling and optimization for AGV charging strategy in energy self-consistent port(PDF)

长安大学学报（自然科学版）[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2024年5期
Page:
164-176
Research Field:
交通能源融合技术专题
Publishing date:

Info

Title:
Reinforcement learning modeling and optimization for AGV charging strategy in energy self-consistent port
Author(s):
LIU Shao-bo1; 2; 3;  HUANG Ying-jie1; 2; 3;  HE Wei-ming1; 2; 3
(1. Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan 430063,Hubei, China; 2. State Key Laboratory of Maritime Technology and Safety, Wuhan 430063, Hubei, China; 3. Sanya Science and Education Innovation Park, Wuhan University of Technology, Sanya 572000, Hainan, China)
Keywords:
traffic engineering;  energy self-consistent port;  clean energy consumption;  AGV charging strategy;  reinforcement learning;  simulation scenarios
PACS:
U691.3
DOI:
10.19721/j.cnki.1671-8879.2024.05.014
Abstract:
To optimize the charging strategy of automated guided vehicles(AGVs)in energy self-consistency ports and enhance the consumption of clean energy, a reinforcement learning modeling and optimization method for AGV charging strategy in simulation scenarios was proposed. By analyzing the scheduling process of AGV transportation and charging, a simulation-based framework for reinforcement learning strategy optimization was established. The charging decision-making process of AGVs was modeled as the Markov decision process(MDP), with the influence of AGV charging decisions on overall transportation operations articulated through the configuration of environmental space, action space, and reward function. A simulation model, reflective of a Chinese port's layout, was constructed to simulate the processes of container import and export transportation and AGV charging. The horizontal transportation link, including AGV selection, scheduling procedures, and charging protocols was defined, while capturing the constraints of energy, equipment quantity, and logistics scheduling rules present in actual transportation scenarios. Two clean energy generation curves with different feature were utilized as reward coefficients for charging, facilitating the optimization of charging strategies through the interaction of the DQN algorithm with the simulation model. The optimized charging strategies were compared and analyzed against two different rule-based charging strategies in different initial battery power scenarios. The results show that the clean energy consumption rates of the two optimized charging strategies remain 74% to 94% and 69% to 72% while the rule-based charging strategies are 38% to 62% and 33% to 55%. In terms of total energy consumption, the clean energy share of the optimized charging strategy is 35% to 47%, compared to 17% to 37% for the rule-based charging strategy. The optimized charging strategy is generally more favorable in terms of energy cost. The simulation durations for all four strategies across four different initial battery power scenarios are consistent, ranging from approximately 60,000 to 62,000 s, indicating that the proposed method does not prolong overall operational time while enhancing clean energy consumption capacity, thus confirming its effectiveness and economic efficiency.6 tabs, 11 figs, 25 refs.

References:

[1] 邰能灵,王萧博,黄文焘,等.港口综合能源系统低碳化技术综述[J].电网技术,2022,46(10):3749-3764.
TAI Neng-ling,WANG Xiao-bo,HUANG Wen-tao,et al.Review of lowcarbon technology for integrated port energy systems[J].Power System Technology,2022,46(10):3749-3764.
[2]方斯顿,赵常宏,丁肇豪,等.面向碳中和的港口综合能源系统(二):能源-交通融合中的柔性资源与关键技术[J].中国电机工程学报,2023,43(3):950-969.
FANG Si-dun,ZHAO Chang-hong,DING Zhao-hao,et al.Portintegrated energy systems toward carbon neutrality(Ⅱ):Flexible resources and key technologies in energy-transportation integration[J].Proceedings of the CSEE,2023,43(3):950-969.
[3]ZHAO Q R,JI S W,GUO D,et al.Research on cooperative scheduling of automated quayside cranes and automatic guided vehicles in automated container terminal[J].Mathematical Problems in Engineering,2019,2019:6574582.
[4]张亚琦,杨 斌,胡志华,等.自动化码头AGV充电与作业的集成调度研究[J].计算机工程与应用,2017,53(18):257-262,270.
ZHANG Ya-qi,YANG Bin,HU Zhi-hua,et al.Research of AGV charging and job integrated scheduling at automated container terminal[J].Computer Engineering and Applications,2017,53(18):257-262,270.
[5]范厚明,郭振峰,杨 宇.预约机制下送箱集卡多集装箱码头调度问题[J].系统仿真学报,2017,29(12):3051-3060.
FAN Hou-ming,GUO Zhen-feng,YANG Yu.Truck scheduling for delivering containers among multiple container terminals based on truck appointment system[J].Journal of System Simulation,2017,29(12):3051-3060.
[6]范厚明,郭振峰,李 阳.考虑碳排放和预约机制的送箱集卡多码头调度问题[J].同济大学学报(自然科学版),2018,46(9):1241-1252,1280.
FAN Hou-ming,GUO Zhen-feng,LI Yang.Truck scheduling in delivering containers from an outside yard to multiple container terminals considering carbon emission and the truck appointment system[J].Journal of Tongji University(Natural Science),2018,46(9):1241-1252,1280.
[7]李林蔓,李雨青,王孟雅,等.基于启发式规则的自动化码头换电式AGV调度优化方法[J].运筹与管理,2023,32(10):9-15.
LI Lin-man,LI Yu-qing,WANG Meng-ya,et al.Optimization method for powerchanging AGV scheduling of automatic terminal based on heuristic rules[J].Operations Research and Management Science,2023,32(10):9-15.
[8]HU H T,CHEN X Z,WANG T S,et al.A threestage decomposition method for the joint vehicle dispatching and storage allocation problem in automated container terminals[J].Computers & Industrial Engineering,2019,129:90-101.
[9]秦 琴,梁承姬.自动化码头考虑缓冲区的设备协调调度优化[J].计算机工程与应用,2020,56(6):262-270.
QIN Qin,LIANG Cheng-ji.Optimization of equipment coordination scheduling with considering buffer space in automated container terminal[J].Computer Engineering and Applications,2020,56(6):262-270.
[10]滕 浩,庄子龙,黄子钊,等.自动化码头水平运输调度的改进人工蜂群算法[J].计算机集成制造系统,2023,29(7):2449-2460.
TENG Hao,ZHUANG Zi-long,HUANG Zi-zhao,et al.Improved artificial bee colony algorithm for automatic terminal horizontal transportation scheduling[J].Computer Integrated Manufacturing Systems,2023,29(7):2449-2460.
[11]XU Y X,QI L,LUAN W J,et al.Load-in-load-out AGV route planning in automatic container terminal[J].IEEE Access,2020,8:157081-157088.
[12]初良勇,周于佩,姚艺飞,等.自动化码头多资源协同调度优化模型与算法[J].哈尔滨工程大学学报,2022,43(10):1539-1546.
CHU Liang-yong,ZHOU Yu-pei,YAO Yi-fei,et al.Co-scheduling optimization of multiple resources in the automated terminal considering AGV path conflict[J].Journal of Harbin Engineering University,2022,43(10):1539-1546.
[13]王 轩,汪 宝,陈艳萍,等.低碳自动化码头AGV物流调度和有序充电的联合经济优化[J/OL].上海交通大学学报,2023:1-24[2023-06-01].https://doi.org/10.16183/j.cnki.jsjtu.2023.027.
WANG Xuan,WANG Bao,CHEN Yan-ping.Joint economic optimization of AGV logistics scheduling and orderly charging in low-carbon automated terminal[J/OL].Journal of Shanghai Jiaotong University,2023:1-24[2023-06-01].https://doi.org/10.16183/j.cnki.jsjtu.2023.027.
[14]SINGH N,DANG Q V,AKCAY A,et al.A matheuristic for AGV scheduling with battery constraints[J].European Journal of Operational Research,2022,298(3):855-873.
[15]梁 晶,王超恒.基于改进NSGA-Ⅱ算法的自动化集装箱码头AGV调度优化[J].中国水运(下半月),2023,23(5):28-30.
LIANG Jing,WANG Chao-heng.AGV scheduling optimization of automated container terminal based on improved NSGA-Ⅱ algorithm[J].China Water Transport,2023,23(5):28-30.
[16]LI L M,LI Y Q,LIU R,et al.A two-stage stochastic programming for AGV scheduling with random tasks and battery swapping in automated container terminals[J].Transportation Research Part E,2023,174:103110.
[17]MA N,ZHOU C H,STEPHEN A.Simulation model and performance evaluation of battery-powered AGV systems in automated container terminals[J].Simulation Modelling Practice and Theory,2021,106:102146.
[18]彭云辉,张 朋,张 煜,等.考虑自动化集装箱码头充电桩分配策略的动态调度仿真分析[J].武汉理工大学学报(交通科学与工程版),2023,47(1):180-184.
PENG Yun-hui,ZHANG Peng,ZHANG Yu,et al.Dynamic scheduling simulation analysis considering charging pile allocation strategy of automated container terminal[J].Journal of Wuhan University of Technology(Transportation Science & Engineering),2023,47(1):180-184.
[19]马宁丽,胡志华.基于拥堵感知的自动化集装箱码头AGV充电策略[J/OL].计算机集成制造系统,2022:1-22[2022-03-08].https://kns.cnki.net/kcms/detail/11.5946.TP.20220305.1541.010.html.
MA Ning-li,HU Zhi-hua.Congestion-aware AGV charging strategy in automated container terminal[J/OL].Computer Integrated Manufacturing Systems,2022:1-22[2022-03-08].https://kns.cnki.net/kcms/detail/11.5946.TP.20220305.1541.010.html.
[20]傅正堂,胡志华,宗 康.集装箱码头AGV电量非饱和状态下的调度优化[J].大连海事大学学报,2017,43(3):58-62.
FU Zheng-tang,HU Zhi-hua,ZONG Kang.Scheduling optimization of container terminal AGV under electricity unsaturation condition[J].Journal of Dalian Maritime University,2017,43(3):58-62.
[21]ZHENG X Y,LIANG C J,WANG Y,et al.Multi-AGV dynamic scheduling in an automated container terminal:A deep reinforcement learning approach[J].Mathematics,2022,10(23):4575.
[22]CHOE R,KIM J,RYU K R.Online preference learning for adaptive dispatching of AGVs in an automated container terminal[J].Applied Soft Computing,2016,38:647-660.
[23]ANGELOUDIS P,BELL M G H.An uncertainty-aware AGV assignment algorithm for automated container terminals[J].Transportation Research Part E,2010,46(3):354-366.
[24]李 玉,苌道方,高银萍,等.基于数字孪生的自动化集装箱码头多AGV动态调度[J].计算机集成制造系统,2023,29(12):4175-4190.
LI Yu,CHANG Dao-fang,GAO Yin-ping,et al.Multi-AGV dynamic scheduling of automated container terminal based on digital twin[J].Computer Integrated Manufacturing Systems,2023,29(12):4175-4190.
[25]王盼龙,梁承姬,王 钰.自动化集装箱码头多层设备调度及仿真分析[J].计算机工程与应用,2023,59(24):345-359.
WANG Pan-long,LIANG Cheng-ji,WANG Yu.Multilayer equipment scheduling and simulation analysis of automated container terminal[J].Computer Engineering and Applications,2023,59(24):345-359.

Memo

Memo:
-
Last Update: 2024-10-20