﻿ 基于深度网络的船舶冰区航行路线规划方法
 舰船科学技术  2022, Vol. 44 Issue (18): 173-176    DOI: 10.3404/j.issn.1672-7649.2022.18.036 PDF

Research on ship route planning in ice area based on deep network
JIA Li-xiao, QIAO Qian-fang, NIU Tong
School of Nautical Technology, Jiangsu Shipping College, Nantong 226010, China
Abstract: The navigation route planning of ships in ice area based on deep network is studied. Based on deep Q network, the state space, action space and reward function of ship navigation are designed. The ice area model is established by raster method to construct the ice area state space. The ship action is learned and trained in each space and reward function process, and the optimal route is generated. After smooth processing, the sailing distance is shortened, and the sailing route is planned for the ship from the current position, which can avoid the collision with the floating ice in the ice area. The research results show that the proposed method can generate a good path after 8000 iterations, and the simulated route can avoid the floating ice accurately and the path node reduction is smooth.
Key words: deep network     ice area navigation     route planning     state space     action space     reward function
0 引　言

1 船舶冰区航行避障路线规划设计 1.1 基于深度Q网络的船舶路线规划准备

 $Q\left( {s,a} \right) = Q\left( {s,a} \right) + \alpha \left[ {r + \gamma \mathop {\max }\limits_a Q\left( {{s_{t + 1}},{a_{t + 1}}} \right) - Q\left( {s,a} \right)} \right]。$ (1)

 图 1 深度Q网络计算流程 Fig. 1 Calculation flow of deep Q network
1.2 船舶冰区避障路径设计 1.2.1 海洋环境建模

 $\left\{ \begin{gathered} x = \bmod \frac{k}{m} - 1，\\ y = {{\rm{int}}} \frac{k}{m}。\\ \end{gathered} \right.$ (2)

 $s = \left( {x,y} \right)。$ (3)
1.2.2 避障路线状态空间确定

 $S = \left\{ {{s_i} = \left( {{x_i},{y_i}} \right)\left| {i = 1,2, \cdots ,N} \right.} \right\} 。$ (4)
1.2.3 避障路线动作空间确定

 $\begin{cases}a=\mathrm{arg}{\mathrm{max}}_{a}Q\left(s,a\right)\left(\varepsilon 的概率\right)，\\ 开展新探索\left(1-\varepsilon 的概率\right)。\end{cases}$ (5)

1.2.4 设计奖励函数

 $R=\begin{cases}100\left(抵达终点\right)，\\ 0\left(没有与冰块发生碰撞\text{，}抵达终点\right)，\\ -100\left(与冰块发生碰撞\right)。\end{cases}$ (6)
1.2.5 航路平滑设置

2 结果与分析

 图 2 奖励值变化趋势 Fig. 2 Change trend of reward value

 图 3 损失函数变化 Fig. 3 Change of loss function

 图 4 平滑处理前后路径规划效果 Fig. 4 Effect of path planning before and after smoothing

 图 5 不同浮冰面积下船舶避障效果 Fig. 5 Obstacle avoidance effect of ships under different floating ice areas
3 结　语

 [1] 杨帆, 何正伟, 何帆. 基于深度网络和船舶交通流的航道水深预测方法研究[J]. 武汉理工大学学报(交通科学与工程版), 2019, 43(1): 130-135. [2] 韩志豪, 汪益兵, 张宇, 等. 基于深度强化学习的船舶航线自动规划[J]. 中国航海, 2021, 44(1): 100-105. DOI:10.3969/j.issn.1000-4653.2021.01.017 [3] 刚旭皓, 田于逵, 季少鹏, 等. 冰区航行船舶冰阻力数值研究进展[J]. 船舶工程, 2020, 42(9): 6-13+43. DOI:10.13788/j.cnki.cbgc.2020.09.02 [4] 童帮裕, 胡坚堃. 基于改进蚁群算法的船舶冰区航行路径规划[J]. 中国航海, 2020, 43(1): 24-28. DOI:10.3969/j.issn.1000-4653.2020.01.005 [5] 杨琪森, 王慎执, 桑金楠, 等. 复杂开放水域下智能船舶路径规划与避障方法[J]. 计算机集成制造系统, 2022, 28(7): 2030-2040. DOI:10.13196/j.cims.2022.07.009 [6] 朱坤财, 徐郑攀, 赵自奇, 等. 基于航迹预测的水面无人艇动态避障方法[J]. 中国测试, 2021, 47(11): 28-33+46. DOI:10.11857/j.issn.1674-5124.2021020095 [7] 罗贤程, 尹建川, 李宏宾. 基于动态分阶势场法的船舶自动避碰系统[J]. 上海海事大学学报, 2020, 41(1): 1-7. DOI:10.13340/j.jsmu.2020.01.001 [8] 洪晓斌, 徐郑攀, 魏新勇, 等. 基e于改进速度障碍法的水面无人艇动态避障[J]. 光学精密工程, 2021, 29(9): 2126-2139. DOI:10.37188/OPE.2021.0272