1   Introduction

Since the end of the last century, autonomous vehicle has become a popular research field as a move towards unmanned driving and safe operation (Jurgen and Ronald, 2013). Offshore vehicles, such as surface ships and underwater vehicles, also closely follow this autonomous operation trend. As compared to ground vehicles (e.g., cars and trains) (Weichselbaum et al., 2013; Yin et al., 2017; Jo et al., 2015), ships have less maneuverability due to their huge sizes, large masses, and complex evolving external disturbances. These diverse characteristics of ships cause exceptional difficulties in improving their autonomy level. Although there have been tests, experiments, and even products on ground vehicles, similar applications on surface ships are still limited.

Most offshore operations and construction are carried out by human operators in hazardous environments. Such work can be risky due to changing sea environments, harsh living conditions, and workplace hazards. This has motivated researchers' interest in the field of autonomous ships to obtain better safety and efficiency during sailing and operations. As long as the autonomy level onboard is improved, human operators can be supported or even replaced by artificial intelligence (AI) systems. Therefore, a significant part of offshore work can be replaced by onshore office work. Such a step provides the advantages of safer working and living conditions for human operators, as well as lower labor costs for companies.

Additionally, autonomy in ships can help reduce human errors during offshore operations. Surveys have shown that 80%–90% of shipping accidents happen due to human errors, either directly or indirectly (Heij and Knapp, 2018; Schroder-Hinrichs, 2010). These human-induced accidents may cause damage to ships and sometimes even harm to human operators. Thus, it is highly essential to improve the autonomy level of ships for safer offshore operations and construction.

The majority of available applications related to autonomous ships use traditional model-based algorithms (Gu et al., 2020). However, the complicated modeling of different types of autonomous ships has limited such applications. Furthermore, during sailing, ships have to deal with frequently changing environmental loads and slowly varying operational parameters caused by fuel consumption as well as tear and wear conditions. These issues of surface ships are difficult to address using traditional model-based or signal-based methods. On the plus side, the DL method can learn to approximate the model and be generalized to the unknown changing environment without a prior model. With the implementation of the DL technique, the process can reduce the expert-involved knowledge and automate the modeling process. Therefore, the development of the DL method in the last decade has provided an innovative solution to the aforementioned problems.

In recent times, the DL technique has been proven to be adaptable in the field of autonomy. The DL technique has been extensively used in autonomous vehicles, such as car-following (Zhu et al., 2018; Wang et al., 2018a) and car detection (Ammour et al., 2017) for autonomous cars. It is well known that automobiles are normally exposed to a more complicated driving environment compared to surface vessels. Therefore, most DL applications in autonomous cars are developed in the field of traffic and lane detection and logistics. Similar studies based on the application of the DL technique in autonomous surface vessels also confirmed its ability to deal with detection, control systems onboard, monitoring systems, and transportation and logistics.

In this paper, we have presented a detailed review of current progress in DL methods applied to ship autonomy. The first part of this review paper carefully addresses the state-of-the-art in surface ship autonomy. This clearly gives us deeper insight into the current issues and challenges encountered by autonomous surface ships. In the next part, a detailed discussion of the history and up-to-date theories of DL methods is provided. Afterward, the last part of this review paper deals with the applications and challenges of DL methods in autonomous ships.

To our knowledge, a careful analysis of AI literature makes it evident that the review studies completely dealing with DL in autonomous surface ships are still missing. Contrary to the existing review studies, a significant part of this work contributes to the following potential aspects:

• The plausible reasons, which motivated us to adapt DL in autonomous surface ships over the traditional model-based or signal-based algorithms, and the subsequent level of improvement in automation, achieved by DL algorithms.

• A wide range of DL applications in the field of autonomous surface ships, which have been further classified into four aspects, namely control systems onboard, ship navigation, fault detection, and transportation and logistics. The distinct characteristics of these different DL applications have been thoroughly discussed.

• The current challenges of using DL methods in autonomous surface ships are analyzed in detail. In addition, a brief note on potential applications of this subject and its future research scope is also provided as well.

The remaining parts of this review paper have been organized in the following manner. In Section Ⅱ, the current scenario of autonomous surface ships and their recent challenges are given. Subsequently, Section Ⅲ provides a brief introduction to DL, including its history and some of its most renowned methods. Next, a detailed survey of DL in control systems onboard is provided in Section Ⅳ, followed by summarizing the applications of various DL methods in navigation systems in Section Ⅴ. Further, Section Ⅵ provides the relevant literature review for DL in monitoring systems. Subsequently, the DL applications in transportation and logistics are added in Section Ⅶ. Furthermore, the research trends of DL in autonomous ships are discussed in Section Ⅷ. In Section Ⅸ, some of the more important points of this review study are readdressed as concluding remarks.

2   Autonomous ships and research scope

With the advancement of control systems and artificial intelligence (AI), the autonomy of offshore ships has become apparent in the eyes of the public. This section briefly discusses the characteristics and development of autonomous ships, followed by current, ongoing challenges.

2.1   Characteristics of the autonomous ship

The term "autonomous ship" has first become a research subject at the beginning of the 21^st century (Statheros et al., 2008). Subsequently, global organizations and maritime companies have started collaborative research works in standardizing "autonomous ships." The collaborative research project, MUNIN (Maritime Unmanned Navigation through Intelligence in Networks), was co-funded by the European Commission under its Seventh Framework Program. This project was majorly proposed to study and develop the concept of autonomous ships. Herein, autonomous ships are first defined as the "next generation modular control systems and communications technology that will facilitate wireless monitoring and control functions for both on and off the board. These features will be included in advanced decision support systems to provide a capability to operate ships remotely, under semi or fully autonomous control" (MUNIN, 2021).

In 2018, International Maritime Organization (IMO) defined Maritime Autonomous Surface ships (MASS) as "ships to a varying degree, which can operate independently of human interaction", and also defined four levels of autonomy for MASS (IMO, 2021). Similarly, Llyod's Register (2015) has defined six autonomy levels for ships. It can be noted that the defined autonomy levels by IMO, as well as Lloyd's Register, are summarized in Tables 1 and 2, respectively. Both entities defined the autonomy levels in a gradual manner, with the low level referring to the low autonomy with human operations, while the high level presents high autonomy with few or no human operations.

Currently, most surface ships' autonomy levels are still low (i.e., under autonomy level 1, as defined by IMO, or autonomy levels 1 and 2, as defined by Lloyd's Register). The majority of decisions and subsequent operations onboard are carried out by human operators.

Under such circumstances, autonomous surface ships have been widely discussed and studied in the past decades in the marine industry. In 2012, the first autonomous ship project in the marine industry was initiated in Europe (Munim, 2019). Earlier to this project also, there have been many research investigations were carried out on autonomous ships. Such research work includes the following essential aspects, investigation of control systems onboard (Wu and Yang, 2014), transportation and logistics (Davarzani et al., 2016), offshore navigation (Crisp, 2004; Han and Chong, 2004; Noel et al., 2019; Ozturk and Cicek, 2019), and fault detection (Howard, 1994).

2.2   Challenges in autonomous ship

Similar to most advanced technologies, the major challenges encountered by autonomous ships arise from two aspects. They are given as research and applications. Research of autonomous ships lies in an interdisciplinary subject. So, this not only needs massive effort in each subject topic but also requires collaboration between different scientific communities. On the other hand, a variety of applications for autonomous ships are also facing challenges in terms of legislation and regulations (Perera, 2018), design, operation, maintenance, and management. Although the Maritime Safety Committee (MSC) has approved Interim guidelines for MASS trials (IMO, 2019) at its 101 session in June 2019, the details of autonomous ship guidelines still need to be further consummated.

With the development of maritime technology, the structure and engines onboard the ships are getting increasingly complicated to fulfill the various tasks. In such circumstances, the traditional model-based control methods face challenges in adapting these complex models. All these adverse conditions open up a new window for DL-based algorithms to provide solutions without precise modeling. Thus, researchers have adapted it to improve the autonomy of surface ships.

2.3   Research scope

It is natural that when we first read the term "autonomous ship, " what comes to our mind is auto-pilot, as similar to auto-driving in autonomous cars. Yet we know that autonomous ships are far away from auto-pilot technology. Unlike auto-pilot or auto-driving, ship autonomy is a complex field that has relations to many different subjects. A fully autonomous ship requires research and applications of various potential aspects, majorly including ship design, control systems, auto-pilot and route planning systems, navigation systems, and monitoring systems in combination with system integration. Such work can be roughly divided into three categories: onboard, in port, and onshore. In this paper, we will focus on research work on more efficient usage of DL methods in autonomous surface ships, applied to systems onboard and in port.

As shown in Figure 1, a fully autonomous surface ship needs monitoring systems, such as control and navigation systems onboard, logistics control in port, and remote control, as well as supervisory onshore. In this paper, we will only consider the autonomy of a single ship and completely excludes control of multiple ships. As most up-to-date applications of DL of autonomous surface ships lie in the field of onboard and in-port systems, we will classify such applications into the following two aspects:

Figure  1  Various systems involved in an autonomous ship

Download: Full-Size Img

• Onboard systems that include three main parts: control systems, navigation systems, and monitoring systems.

• Systems in port, such as transportation and logistics.

3   DL introduction

This section briefly describes the development of DL-related methods in recent years, as well as some of the classical algorithms that are broadly adopted in the DL domain.

3.1   Introduction to machine learning (ML)

Humans have been dreaming of building machines that can "think" ever since ancient Greece. This "dream" is no longer a fantasy now with the invention of high-speed computers and the development of ML algorithms. For decades, ML has been an active research area with its excellent performance and wide range of applications in miscellaneous engineering fields. Nowadays, researchers in various fields are seeking opportunities to incorporate ML methods to solve the difficulties encountered in their respective fields.

In general, there are three different types of ML principles, supervised learning, unsupervised learning, and reinforcement learning (Figure 2).

Figure  2  Three different types of machine learning: supervised learning, unsupervised learning, and reinforcement learning

Download: Full-Size Img

• Supervised learning uses labeled datasets to train models to obtain the desired output. The applications of supervised ML can be concluded in two folds: they are regression to understand the relations between different variables and classification to assign data into multiple categories.

• Unsupervised learning, on the contrary, uses unlabeled datasets for training. It is normally used to solve clustering or association problems.

• Reinforcement learning (RL) is coined as another powerful ML algorithm. This deals with how intelligent agents should make decisions in a particular environment. The essence of RL is to make decisions based on different observations (environmental state) in the environment through the agent and to continuously interact with the environment to obtain different reward signals, as shown in Figure 3.

Figure  3  Schematic diagram demonstrating the reinforcement learning

Download: Full-Size Img

3.2   Brief history of DL

When ML was first introduced in the public domain, people were trying to use it to solve their problems. But such an end use of ML requires intensive computation, which is normally difficult for humans but easy for computers. The real challenge for ML, however, is to solve problems that are difficult to formulate into mathematical models, such as the awareness of different faces and different sounds. However, these experience-based tasks are easy for human beings, as they are challenging to machines. Artificial Neural Network (ANN) was introduced under such a background. Instead of describing the problems with formalized mathematical models, ANN discovers feature representations of data by combining the low-level features to form more abstract high-level representations.

The concept of DL originated from the investigation of ANN models in ML theory. The first ANN-related model, originating in 1943, came from McCulloch and Pitts' attempts to model nerve cells (McCulloch and Pitts, 1943). Hereafter, the so-called MP model was introduced in the research. Figure 4 shows the structure of an MP model. The model consists of linear weightings of input signals, summation, and nonlinear activation. With the application of the MP model, the first ANN model was proposed in 1958 by Rosenblatt (1958), and later it was used for completing simple classification tasks. This model was named "perceptrons." The convergence of the perceptrons was proven in 1962 (Rosenblatt, 1962). Not long after the perceptron rule was introduced, it was proven by Minsky that it only adapts to linear systems (Minsky, 1968). Thus, research on perceptrons came to a standstill.

Figure  4  Schematic diagram illustrating the MP (McCullon and Pits, 1943) model

Download: Full-Size Img

Afterward, another major extension of the ANN took place in the late 20^th century, when Werbos developed a backpropagation algorithm to train multilayer neural networks (Werbos, 1988). Such a mechanism is shown in Figure 5. In a study by Rumelhart et al. (1986), the original network structure is replaced with multiple hidden layers, and subsequently, the Sigmoid function was introduced for nonlinear mapping. Meanwhile, the backpropagation algorithm is used to train the model for nonlinear classification and learning. In 1989, LeCun et al. (1989) invented the convolutional neural network (CNN) by using the backpropagation algorithm to recognize handwriting in post office systems. However, as a heuristic optimization strategy, the backpropagation algorithm encountered problems with gradient vanishing (Vandeginste et al., 1998). Since 2006, such gradient vanishing problems can be effectively prevented by deep belief net with fine-tuning mechanism (Hinton, 2009) and Rectified Linear Unit as an activation function (Nair and Hinton, 2010). The aforementioned research outcomes have laid the foundation for DL. Afterward, the theoretical research and applications of DL stepped into a period of rapid development.

Figure  5  The functional features of the backpropagation algorithm

Download: Full-Size Img

In 2011 and 2012, Deep Neural Network (DNN) models were adopted in speech recognition by Microsoft and Google, with considerable achievement (Senior et al., 2012; Dahl et al., 2011). In the computer vision field, AlexNet, with dropout techniques accelerated by Graphics Processing Unit, won first place in the famous ImageNet Challenge in 2012 (Krizhevsky et al., 2012). Since then, DL, especially CNN, has drawn tremendous interest among all disciplines of global scientific communities, and soon it became one of the most popular ML methods. Later in 2014, Dauphin et al. proved that the local minima problem of neural networks is generally not a serious problem and removing the local minima haze hanging over neural networks [Dauphin et al., 2014]. From then on, the development of DL has received an explosive period, where the relevant techniques were going through astounding growth.

On the plus side, DL shows powerful information perception capabilities in supervised learning fields, such as image recognition, audio recognition, and natural language processing. In addition to supervised learning, DL exhibits superior performance in the field of RL as well. As said earlier, RL is another class of ML algorithms in the field of AI. The learning mechanism of RL requires learning from the collected interactive trajectories (learning experience), modification, and subsequent improvement in decision-making ability so as to obtain larger cumulative rewards.

Eventually, the emergence of DL methods has greatly improved the agent's information-extracting ability from a series of observations (e. g., visual/sound observations) in RL algorithms. With the addition of DL algorithms, deep RL shows that its control does not lose or sometimes even exceed the human-expert level in many fields. Therefore, deep RL algorithms were widely used in the field of games (Vinyals et al., 2019), robot control (Chen et al., 2018), and unmanned driving vehicles (Sallab et al., 2017). One of the most famous cases happened in 2016 when deep RL-based AlphaGo defeated the best Go players in Asia and became the best Go player (Silver et al., 2017).

Research on DL is still moving forward. Nowadays, in addition to the above fields, DL has also been applied to fields such as medical science (Albarqouni et al., 2016), praxeology (Phan et al., 2017), transportation (Lv et al., 2015). We believe that there will be more scientific disciplines that are involved in collaboration with ML, especially DL, in the near future. Much more effort will be expended to solve the problems that are difficult to formulate by mathematical representations with the attempt to use DL.

The combination of DL and autonomous ships is a new trend in the field of maritime and offshore engineering. Since the late 20^th century, a series of research studies have shown their variety of approaches to recognize the ship in graphics using neural network models (Inggs and Robinson, 1995). In the next section, we provide a detailed review of the applications of DL methods in autonomous ships. This part of the discussion will further focus on four disciplines: control systems onboard, transportation and logistics, navigation, and monitoring system.

4   Control systems onboard

Control systems of surface ships have always been a popular research field in ships and shipping autonomy. Control systems onboard normally include the following essential aspects: position control, motion prediction and detection, power management, and thrust control. Currently, most these control systems are still based on PID controllers to ensure the robustness and reliability of offshore structures (Roberts, 2007). Although PID controllers have proven to be trustworthy in the industry for decades, the traditional PID controllers have shown unsatisfactory performance under certain circumstances, e.g., when the controlled plant has a time-varying model or when the tuning process is difficult to proceed.

Under such an inevitable background, new control methods are proposed to deal with the performance difficulties encountered during offshore construction. DL is one of the most popular methods, especially attributed to its approximating ability to solve black box problems. Most DL applications with respect to marine control systems are known to be related to deep RL. But the use of DL in thrust control is still missing. Such applications mainly focus on three fields: position control, motion prediction and detection, and power management. In this section, we discuss applications of DL in these fields with respect to autonomous surface ships.

4.1   Position control

Position control refers to the systems that track and adjust the position of the controlled plants. For autonomous surface ships, position control systems can be divided into three aspects: auto-berthing and steering control, tracking and trajectory control, and dynamic positioning. In the last decade, numerous efforts have been made to control the ship position using DL.

1) Auto-berthing and steering control: Ship auto-berthing, as well as computer-aided berthing, were proposed in the late 20^th century to deal with complex environments and decrease human errors in ports (Zhang et al., 2022). In the 2010s, an adaptive neural network was applied to the auto-berthing of underactuated surface ships to handle the dynamic uncertainties and external disturbances (Zhang and Zhu et al., 2019). Herein, the radial basis function neural network is proposed to estimate the nonlinear dynamics. In order to deal with both external disturbances and nonlinear uncertainties, during turn-around-way auto-berthing, the deep-rooted information (DRI) and Multilayer Perceptron (MLP) models were utilized to estimate the parameters in the linearized uncertainty functions.

Then (Kim et al. (2020) applied the DL-based Skip-ENet (Kim et al., 2019) to the vision-based monitoring system to ship berthing. Such an NN-based algorithm was used as an encoder and decoder bottleneck to compress the representation of image data. The size of the receptive field in the dilated convolution is adjusted to segment the small objects with an unchanged computation amount. In this way, the performance of auto-berthing was improved by means of detecting complex marine obstacles.

Later in 2017, the deep RL was first applied to the steering control of an underactuated ship using policy-based deep RL algorithms, i. e., deep deterministic policy gradient (DDPG) and normalized advantage functions (NAF) (Tuyen et al., 2017). Herein, the DDPG generates continuous action using the actor-critic framework to overcome the difficulty using Q-learning with continuous action domain, while the NAF deals with the value-function estimation bias problems. Such a control algorithm could assist the ship in sailing past certain locations. However, more complex steering control targets could not be achieved by relying solely on DDPG and NAF.

2) Tracking and trajectory control: Apart from steering control, the deep NN was also applied in tracking and trajectory control of autonomous ships (Wang et al., 2018b; Zhang et al., 2020b; Chen et al., 2019; Guo et al., 2020b; Sun and Gao, 2021).

Wang et al. (2018b) combined the DDPG algorithm with NN control to deal with uncertain models during USV course tracking. Herein, the actor-critic mechanism was replaced with one NN with loss functions to simplify the training process. The simulation results showed that the performance of the course track was significantly improved with DDPG, as compared to only NN. However, the tracking error still remained high during sinuous tracking.

Another attempt was made by Sun and Gao (2021) to combine the deep NN prediction model with the PID controller to achieve auto-tuning for trajectory control. Two deep NNs were proposed in this paper to evaluate the waypoint behavior effect and guidance parameters. Evaluation is done by using a 6-input-5-output 8-layer deep NN, while the parameter valuation model is a 4-input-2-output deep NN. Such a system is based on MOOS-IvP (Mission Oriented Operation Suite - Interval Programming), a set of open-source C++ modules for providing autonomy on robotic platforms, in particular for autonomous marine vehicles. Although the overtuning problem has not been solved in this application, it is still a bold attempt to use deep NN in ship tracking and trajectory.

Further studies using deep RL-based control methods were proposed by Chen et al. (2019) and Guo et al. (2020b) for intelligent path planning of autonomous ships.

Chen et al. (2019) proposed the q-Learning-based RL path planning method while the steering of the rudder was controlled. The obstacle avoidance problem was simplified with the action space set to limited options. In this work, the navigation rules and economic efficiency were first modeled as the reward components. Although the proposed RL method could successfully learn and plan the path under different scenarios, it can only be adapted to low-speed ships with no dynamic obstacles.

Further studies with RL in path planning were proposed by Guo et al. (2020b). This quantified the crew experience and COLREGS to ensure the planned path was in line with IMO rules and to fulfill the global optimum. Based on the experience of relay memory, a deep RL with the combination of artificial potential field (APF) and DDPG path planning system was proposed. Such an APF-DDPG-based method has a faster learning time and also higher convergence speed compared to DDPG, DQN, and Q-learning.

However, none of the above-mentioned studies has done research on Lyapunov stability analysis. Overall, Lyapunov stability was difficult to achieve in most ML-based control algorithms. Zhang et al. (2020b) combined RL with the Lyapunov function and baseline controller in a tracking control of the autonomous ship. For the first time, overall closed-loop stability was ensured by using deep RL, with model uncertainty handled at the same time.

3) Dynamic Positioning: Dynamic positioning systems are essential for offshore construction vessels and offshore platforms to maintain the position and heading via propulsion systems in the open sea (Desai, 2015). Traditional DP systems, like any other marine control systems, are usually based on PID controllers and Kalman filters (García and Vázquez, 2008). Such a system is sensitive to sudden changes in environmental loads and cannot handle time-variant models (Ye et al., 2021). In a work by Lee et al. (2020), an RL-based adaptive PID controller was proposed for DP systems. As compared to the traditional PID controller, the proposed algorithm by Lee et al. (2020) could obtain optimal PID gains from learning, which assists the PID controller in coping with the varying environmental loads without human input.

4.2   Motion prediction and detection

Ship motion prediction is widely used onboard to navigate or for special offshore operations (Jose M. and Juan F., 2010). With the development of DL methods, deep NN has been extensively applied in ship motion prediction systems (Liu et al., 2020).

Liu et al. (2020) proposed two RNN-based ship motion prediction systems with Long Short Term Memory (LSTM) based on impulse response function and auto-correction function, respectively. Both algorithms showed equivalent performance in training efficiency, but auto-correction function-based RNN has displayed slightly better prediction accuracy.

Table 3 provides a summary of DL-based applications in marine position and motion control systems. In general, the current applications of DL methods in maritime control systems are mostly based on an NN, and most of them use deep RL methods.

5   Ship recognition in navigation

Ship navigation refers to the study of knowing the current position and getting to the destination. The main purpose of ship navigation systems is to provide the global and local location of the target ship and its surroundings (Lazarowska, 2019). As shown in Figure 6, traditional navigation systems use Global Navigation Satellite System (GNSS) to locate the ship and to detect obstacles as well. Although the GNSS is extensively used in ship navigation, it has a few drawbacks. First of all, it cannot support the requirements of all phases of ships without augmentation systems. Furthermore, the accuracy of GNSS is not sufficient when it comes to some special ships. Under such circumstances, DL-based navigation has been studied to improve the current systems. Such applications include ship detection and identification from Synthetic Aperture Radar (SAR) images and photos, traffic prediction, image fusion, etc. In this section, we provide a detailed discussion based on the aforementioned DL-based navigation systems, especially for ship detection.

Figure  6  Ship navigation system

Download: Full-Size Img

5.1   Ship detection with SAR images

Ship detection is essential in marine navigation systems. In order to get the position and to plan the trajectory of the ship, as well as to plan routes of multiple vessels in port, one has to detect the ship from satellite images or from camera data. There are generally two types of ship detection systems. One is to locate the ships for navigation purposes, and the other one is to detect other ships to avoid collisions. Herein, the ship detection systems based on SAR images are most widely studied. These types of ship detection systems focus on locating the ships in the radar image. The first part of this section demonstrates the various investigation reports dealing with ship detection systems by using SAR images.

With the development of DL algorithms, image and sound recognition methods have experienced a fast improvement in the last two decades. Such image and sound recognition technology has also been applied to the navigation of autonomous surface ships. In recent studies, it has been reported that the combination of up-to-date ML technology with navigation systems was used to detect and classify the different types of ships.

1) Deep NN-based ship detection and classification: Not until 2016 the navigation systems based on SAR ship detection were studied, and multiple DL methods were applied to this field (Schwegmann et al., 2016). In Schwegmann et al.' s study, deep highway networks with LSTM were proposed to detect the ships from sub-images of the SAR dataset. Such detections are classified into three categories: positive, false positive (ship-likes), and negative. The proposed method was able to detect the ships with an accuracy rate of over 90%. Joseph et al. (2018) also achieved high accuracy with morphological-based DNN for ship detection and classification, with an accuracy rate close to 95%. In their study, the ship detection and classification algorithm has been simplified by comparing it with predefined point models.

Apart from the aforementioned methods, the deep NN-based RetinaNet was also a popular algorithm in ship detection and classification, as described elsewhere (Wang et al., 2019a), (Gao et al., 2019a). The so-called RetinaNet also adapts a feature pyramid network to sub-sample the images into small-size figures for detection purposes.

2) CNN-based ship detection and classification: On the other hand, the aforementioned NN-based SAR ship detection and classification systems do not take spacial information into account, leading to an increase in the difficulty of learning the procedure of the systems. In order to overcome the loss of spatial information with previously proposed algorithms, as well as to deal with an unbalanced and small dataset, the CNN was studied for ship and trajectory detection (You et al., 2019; Fan et al., 2019). Sharifzadeh et al. (2019) integrated the CNN-based ship type classification and multiple hidden layers (MLP) to decrease the false alarms during their detection with SAR images. To achieve better precision, a multiscale rotated bounding box was proposed by Li et al. (2018b) to replace the traditional horizontal bounding box in CNN-based detection systems. In 2018, CNN combined with constant false alarm rate (CFAR) was first introduced in ship detection by Wang and Li et al. (2018), and such an approach has been proven to be better than using CFAR only.

Later, attempts were made to decrease the detection time of the system by using grid CNN (Zhang and Zhang, 2019). Another example of using CNN to decrease detection time integrates the concatenation mechanism and the anchor box mechanism (Zhang and Zhang, 2019b). Such algorithms can be applied to the real-time detection of high-speed ships. CNN was proposed in ship classification by Li et al. (2018a) to improve the density of datasets. Such a method was proposed with data augmentation and ratio batching to deal with the unbalanced dataset of different ship types. Another attempt to handle unbalanced datasets was made by Shao et al. (2018) with a weighted distance measure loss function.

CNN-based ship detection and classification with fine-tuning in the top layers was first proposed by Wang et al. (2018c) to handle small datasets. All these methods can provide high accuracy and sufficient detection time on the detection of ships. However, the tuning process is difficult. Overfitting may occur during the learning procedure. For traditional anchor-based CNN, the performance depends largely on the tuning of the anchor boxes, which can be time-consuming and also complex for objective detection. To avoid the drawback of such anchor-based CNN, an anchor-free CNN was proposed for SAR ship detection with improved detection accuracy and speed [Gao et al., 2020].

In view of the aforementioned CNN-based studies mostly focus on low-speed ships or static ships. In recent works (Zhang et al., 2019; Zhang and Zhang, 2019), a depthwise separable CNN and a grid CNN were developed to decrease the detection time, thus tracking and detecting high-speed ships in real time.

The aforesaid examples only deal with low-resolution SAR images. Bentes et al. (2018) showed how CNN can handle high-resolution images in TerraSAR-X with nonlinear data and the development of normalization function in preprocessing as well.

3) Regression Convolutional Neural Network (RCNN) based, Faster-RCNN based, and master-RCNN based ship detection and classification: More advanced DL methods were proposed to deal with the overfitting and underfitting problems during the learning process, and thereby to improve the detection speed. Herein, one of the most popular algorithms is RCNN. It was first adapted to SAR ship detection in 2017 (Kang et al., 2017). In 2018, an attention module and angle RCNN SAR ship detection algorithm was proposed to estimate the orientation of the ship during simulations (Wang and Lu et al., 2018). However, the related dataset was not established until 2019 (Wang et al., 2019b). A dataset was constructed for ship detection by Wang et al. (2019b), and subsequently, the RCNN algorithm was modified to identify ships.

Later, Faster-RCNN was studied to improve the up-to-date detection speed and accuracy. In 2018, Faster-RCNN was first applied to multiscale and multiscene SAR ship detection (Jiao et al., 2018), with sensitive negative samples that are specifically selected and labeled. In order to deal with these negative samples, a large number of negative samples containing only land areas were proposed by Gao et al. (2019b) based on Faster-RCNN. Such detection algorithms were improved by Lin et al. (2019b) using squeeze and excitation rank Faster-RCNN.

Mask Region Convolution Neural Network (Mask-RCNN) has also been widely studied to improve classification accuracy under complicated backgrounds (Lin et al., 2019a). This algorithm was compared with F-RCNN and was proven to be more dominant than F-RCNN (Zhang et al., 2020a).

4) YOLO-based ship detection and classification: Another widely used CNN-based method in ship detection and classification is YOLO (You Only Look Once). A CNN model named YOLOv2 with a reduced number of layers was proposed by Chang et al. (2019) to achieve short detection time and high accuracy. Similar applications were also developed by other researchers (Chen et al., 2020b; Zou et al., 2020). Chen et al. (2020b) combined YOLOv2 with a modified generative adversarial network. Such CNN-based detection systems have an accuracy rate of 97.2% during simulations. Zou et al. (2020) adapted YOLOv3 with multiscale Wasserstein distance and gradient penalty in ship detection to improve the accuracy of target detection.

5) Ship detection in the Arctic area: More complicated ship detection and recognition applications using ship detection can be found in the Arctic area, where floating ice can influence the accuracy of classification and detection. In 2016, CNN was first applied to ship-iceberg discrimination in SAR images (Bentes et al., 2016). The result was compared to Support Vector Machine (SVM) and was proven to be better at classification accuracy with large training data. In 2019, pseudolabeling was presented by Rane and Sangili (2019) to deal with the insufficient amount of training data. In 2020, YOLOv3 was proposed in iceberg-ship discrimination to increase classification accuracy [Hass and Arsanjani, 2020]. The above-mentioned algorithms only consider the SAR data while detecting ships from icebergs. Heiselberg (2020) took into account both the SAR data and multispectral satellite images (MSI) data to increase the accuracy of ship classification in the Arctic area.

A summary of SAR ship detection and classification systems using DL can be found in Table 4. Most methods are either based on different types of neural networks or combine neural networks with other DL methods.

5.2   Other ship detection systems

DL is widely studied in the field of ship detection and classification. The studies related to ship detection systems from images other than SAR images can be traced back to 1995 when NN was first investigated and applied in radar ship target recognition [Inggs and Robinson, 1995]. In this study, three types of neural networks were studied in detail, namely, the feedforward network with back propagation, Kohonen's supervised learning vector quantization network, and Simpson's fuzzy min-max NN. All these three methods provided classification accuracy of over 85%. The aforementioned algorithms require a large range of radar image data to achieve better performance.

1) Detection from high-resolution optical images: For the detection of ships from high-resolution images with interference, such as clouds and waves, the NN-based DL method was proposed and tested (Huang et al., 2020). Similar applications were applied for ship identification from optical images using CNN. Gallego et al. (2018) proposed to use a CNN for feature extracting, with a k-nearest neighbor to classify the ships from the dataset. CNN with fine tuning is applied to deal with small datasets (Shi et al., 2018; Zhao et al., 2020). Huang et al. (2019) adopted similar applications using Resnet and feature pyramid networks. Another attempt has been made by Jiang et al. (2019) and Chen et al. (2020a) to classify ships in high-resolution images. Jiang et al. (2019) trained and transferred high-resolution optical images to low-resolution infrared images before classification. In comparison, Chen et al. (2020a) applied a coarse step and fine-tuned step to CNN to obtain a short classification time. Relevantly, in a study by Wang et al. (2021), high-resolution images went through a Gaussian heatmap regression before being classified using CNN.

One of the commonly faced difficulties in ship detection from images is the rotation of the ship. In [Fu et al., 2018], a deep RL-based aerial optical image ship rotation detection system was proposed with a feature fusion pyramid network.

2) Detection from real-time cameras: Ship detection from photos and from real-time cameras was also a popular application of DL since the beginning of this century. In another study, three multi-modal early, middle, and late fusion CNN-based architectures were adapted to image fusion before ship detection (Farahnakian and Heikkonen, 2020). The NN-based detection and classification systems were also proposed for real-time cameras under different weather conditions (Zhao et al., 2019). Later, a coarse-to-fine cascaded CNN was introduced to obtain the short classification time (Chen et al., 2020a). The CNN-based YOLOv2 and YOLOv3 were also applied onboard for ship detection and tracking to achieve short detection time and real-time tracking as well (Liu et al., 2019; Li et al., 2020]. For ship detection in video surveillance, a k-nearest neighbor-based SVM was proposed by Cao et al. (2020), with a CNN-based ship feature extraction.

3) Detection using acoustic data: With the development of DL methods and the increasing demand for ship type recognition in the deep sea for both military and commercial purposes, the NN-based ship classification has become an essential subject topic in both research and industry. In 2017, a new ship classification method based on feedforward NN and SVM was proposed by Niu et al. (2017). Such a method is able to detect the different types of ships using acoustic data. The classifiers perform well up to the 10 km range. In 2020, CNN was proposed to classify surrounding ship types with acoustic data (Shen et al., 2020). Such a classification system was robust to ship operative conditions during simulations. In 2018, a ship type classification system was first introduced with CNN via auditory detection (Shen et al., 2018).

A summary of non-SAR-based ship detection, and classification systems, using the DL method can be found in Table 5. Such applications can be divided into three categories: radar image detection, camera image detection, and auditory data detection. All the above-mentioned detection and classification systems are based on different types of neural network models, while some of these applications are combined with neural networks and SVM.

5.3   Other ship navigation systems

Apart from the applications mentioned above, there are a few other systems using DL methods for ship navigation systems, including navigation-based image fusion (Guo et al., 2020a), inland ship navigation (Zhong et al., 2019), and traffic prediction for navigation purpose (Kim and Lee, 2018).

A deep NN-based data fusion system was proposed by Guo et al. (2020a) to merge marine radar and electronic chart data. Such applications can support navigation with comprehensive information.

Inland ship navigation can be even more challenging due to the complex environment and interfering noise. For inland ship trajectory restoration and AIS (Automatic Identification System) data repairing, RNN was proposed to improve the ship navigation systems (Zhong et al., 2019).

Safety and sea-port operational efficiency are noted to be the main concern in crowded harbors. Kim and Lee (2018) investigated such problems using CNN-based DL algorithms. Herein, ship movement and ship attribute features are retrieved by using CNN and fully-connected NN, respectively. The extracted data are subsequently analyzed in another NN for the prediction of the number of ships in the interested area.

A summary of DL in other ship navigation systems is provided in Table 6.

6   Transportation and logistics

It is well known that offshore transportation and logistics play an important role in the global economy. Compared to the control of a single ship, controlling multiple ships and regulating the stowage onboard are more challenging tasks for maritime engineering. Herein, the nonlinear and time-varying model of such systems makes it difficult to control via traditional Lyapunov-based control algorithms. DL has been applied to such problems to deal with their complicated models. These aforementioned applications are mostly related to obstacle avoidance (Cheng and Zhang, 2018; Zhao and Roh, 2019).

DL methods have been applied to such fields since 2018. Particularly for obstacle avoidance, the unknown dynamics of ships and environmental loads have always been known to be a challenging task. By using the traditional control algorithms, robust control was adapted to deal with such unknown characteristics. However, these algorithms are mostly complex in nature and require detailed model data. In a study by Cheng and Zhang (2018), deep RL was first introduced to handle the aforementioned unknown environmental dynamics for obstacle avoidance on an underactuated unmanned marine ship. This is the so-called CDRLOA (concise deep reinforcement learning obstacle avoidance) system, and it was tested in simulations. It can be noted that the real-life applications of a system like CDRLOA still remain worth exploring.

On the other hand, collision avoidance for multiple ships is much more complex relative to single-ship collision avoidance due to the increased number of degrees of freedom. In 2019, the deep Q-learning (DQN) system and deep NN in the multiship collision avoidance system were combinedly used (Zhao and Roh, 2019). In the study, the path following and collision avoidance reward functions are considered separately in DQN-based collision avoidance systems. The proposed method is able to avoid collision in simulations of complex situations, e.g., four ships heading to the center point. But the model uncertainties were not considered in this work.

The amount of literature on the effective utilization of the DL technique in marine transportation and logistics is limited. According to the details summarized in Table 7, the deep NN and DQN systems can be adapted to collision avoidance.

7   Comparing DL over the traditional methods in autonomous surface ships

Traditional model-based or model-free methods were known to be widely studied in autonomous surface ships prior to the development of DL.

7.1   Control systems onboard

Traditional control systems onboard are either usually based on model-free PID controllers or model-based Lyapunov control algorithms (Shi et al., 2017). These methods have been widely applied for construction vessels to improve shipping autonomy, and some of them have proven to be reliable in the marine industry. However, these traditional systems don't require fine-tuning or precise knowledge of controlled plant models. Under certain circumstances, e.g., for construction vessels with complex dynamics, these methods are difficult to apply onboard due to their time-varying controlled plant models.

7.2   Ship navigation

Before DL was proposed, most ship recognition systems were based on support vector machines (Morillas et al., 2015). As compared to the DL-based methods, this algorithm has a shorter training time and is less likely to be overturned. However, SVM-based methods are less competitive in classification relative to DL, while DL is better at handling large datasets.

7.3   Transportation and logistics

In particular, control systems with respect to ship transportation and logistics are mostly based on model-based algorithms (Do, 2011). Herein, controlling multiple ships is more complex than controlling a single ship in the harbor due to the increasing degrees of freedom and nonlinearity. This makes the modeling of the ships an essential tool in controlling systems. On the plus side, the DL technique does not require detailed models, except a large amount of data is necessary for tuning and training.

8   Trends of DL in autonomous ship

Being one of the most effective tools in perception, cognition, and control algorithms, DL has shown its beneficial features as well as near promising future toward further developments in ship autonomy.

As discussed earlier, the DL was known to be most widely used in the field of ship localization and navigation, especially subjected to object detection and classification by using SAR images. It is mainly owing to the strong feature extraction and generalization capability toward image-like matrix signals (e. g., image and SAR). Such a family of algorithms based on the DL technique provides great potential in ship detection in ports, open seas, and even in arctic areas, including accuracy and time efficiency.

Meanwhile, DL is also moved into the applications in terms of improving control and fault detection systems onboard. These applications vary from position and motion control to fault detection and prediction. Among many other DL methods, model-free deep RL is the most popular algorithm. For offshore surface ships, the control plant models are usually uncertain due to construction work and varying environmental loads. Thus makes greater use of model-free DRL.

In conclusion, it can be noted that there are still plenty of interdisciplinary research works which can be done in the combination of DL and autonomous ships in terms of both theoretical and practical significance. Such parts of work include:

• Learning procedure. For most marine systems that use DL, the learning procedure takes time, good data quality, and parameter tuning as well. In the near future, research based on the reduction of model training time, the required quality of data, and the parameter sensitivity can be extensively studied for further performance enhancement of DL-based marine systems.

• Performance. The performance assessment (such as accuracy and precision) of DL methods has been an important reference tool since the origin of classification/detection tasks. Currently, in most applications, autonomous surface ships can reach an accuracy of over 90%. However, in consideration of safety and industry requirements, these performances still need to be improved to ensure the safety and reliability of autonomous surface ships.

• Data collection. Nowadays, the data-driven DL approach highly relies on the amount and quality of the dataset. Therefore, an effective data collection approach still poses a big problem in the case of autonomous surface ships due to the mass of data onboard and the high cost of data collection, data management, and data processing. In such cases, adopting the maritime database is an intuitive approach to increase the level of autonomy of marine surface ships. In addition, the learning efficiency of DL can be treated as a supplementary research direction besides the existing data collection problem.

• Deployment. Currently, DL-based algorithms are regarded as the most popular algorithms. This can be attributed to their ease of application and no requirement for detailed modeling of the controlled plant. Such features favor researchers working in different scientific fields. However, the complicated structure of different neural networks, the requirement of a large amount of training data, and the long duration of training time make it difficult to deploy on ships. Although such DL methods work well in simulations and on scaled models, there is still a large gap between their performance on real ships and simulators. In addition, the subsequent challenges from International Regulations for Preventing Collisions at Sea (COLREGs) also make it difficult to implement the DL algorithms onboard.

• Applications in real ships. In fact, most DL-based algorithms are tested in simulations. So, the applications of such methods in scaled ships or full-scale ships are still rarely adopted. This is mainly due to the following essential factors. Firstly, DL algorithms require large amounts of data, and then the required computing power to process the data is still an expensive setting in maritime practice. Meanwhile, due to the high maintenance cost and long solution leading time, shipbuilding manufacturers are still cautious toward such novel technologies. In view of these aspects, the application of DL methods in ship practice is still short and mostly implemented via simulation.

• Regulations. DL in autonomous surface ships also faces a series of challenges from the industry and government sides. The regulations of DL in autonomous surface ships and the safe adaptation of such algorithms on real ships are still under investigation. On the other hand, how to standardize the data collection and learning procedure of DL methods in full-scale autonomous surface ships and how to assess the overall performance of DL methods are some of the questionable aspects that still need to be answered.

Apart from the aforementioned future research tasks, there are many other potential applications that can be further investigated in autonomous surface ships by using DL, e.g., control systems with complex uncertain models, ship maintenance prediction, shipping management, and automated ship production. These systems either contain internal or external uncertainties that are difficult to model via traditional modeling methods or produce a large amount of data. Using DL algorithms on such systems, one could make the most use of available data and also deal with the unknown model at the same time. Furthermore, DL can be a useful tool when combined with traditional control methods. For example, in the case of complicated marine operations, such as auto-berthing and obstacle avoidance, using experts' data to access DL-based high-level control decisions could handle the problem of model uncertainties. At the same time, the low-level control methods can still be based on traditional control algorithms to achieve a robust control performance. Another example of combined DL and traditional control algorithms involves DL-based self-tuning in traditional Lyapunov-based control. Such a combination of DL and traditional control algorithms would be expected to ensure both Lyapunov stability and optimal tuning in different control applications.

9   Concluding remarks

This review study has provided an outline of DL applications in ship autonomy. We first introduced the history and basic knowledge of DL and autonomous surface ships. Subsequently, some of the most important points collected from the marine literature studies are highlighted. In the end, a comprehensive discussion on the current progress in development and future trends of DL in autonomous surface ships is provided.

Although many of the DL methods have been applied in autonomous surface ships, especially in the field of ship detection and classification, further research work on those methods is highly essential for further improving the overall performance of such DL-based systems. Some of the potential DL applications for complicated maritime systems can also be investigated to achieve a higher autonomy level. In general, it is always a challenging task to make the DL method learn and act in human-like ways. In traditional engineering fields, such as marine engineering and marine technology, the systematic improvement in autonomy and reduction of human actions are becoming the major subjects of intense scholarly debate. Furthermore, the combination of DL and autonomous surface ships can be a promising research field, but this needs a great deal of effort from experts and researchers working in the fields of ship autonomy and ML.