1   Introduction

With the continuously developing of ICT (Information and Communications Technology), the IOT (Internet of Things) is built as a new infrastructure for the next wave of innovation (Hermann et al., 2016). It has started to transform some traditional low-tech industries into high-tech ones, e. g., transportation logistics. Large volume of information is accumulated with high speed and/or high variety, which is normally defined as the concept of "big data" (De Mauro et al., 2015). The proper storage and management of big data are necessitated so that it can be properly analyzed to facilitate a better decision-making and the quality control of the data is a prerequisite for the correctness of analysis results (Blake and Mangiameli, 2011; Bates, 2019). Such a revolution in both "big data" and digitalization is called Industry 4.0 in the onshore manufacturing industry (Hermann et al., 2016).

Similarly, shipping is conducting the same transition, calling Shipping 4.0 or cyber-shipping (Røseth, 2016). The current digitalization of shipping covers a wide range of activities, such as the navigation at sea and cargo operations in port. The digitalization of ship operations implies that a large amount of data are collected on board and then are sent to offices onshore. Data from operations can be used to better understand, optimize and improve navigation, energy consumptions, emissions and maintenance of vessels. Data can be based on noon reports from vessels, semi-automatic data combined with data logged automatically on board, or fully automatic systems which allow for data to be sent ashore with short time intervals, such as Automatic Identification System (Cai et al., 2021). An overview of vessel performance at daily level can be presented via noon reports. A noon report is a manual report prepared by the crew on board on a daily basis, summarizing relevant sailing information, e.g., sailing time, sailed dis‐ tance, course, engine running time, power produced and total fuel consumption. Auto-log data are automatically recorded and logged in real time on board via sensors and ICT technology, which will provide more sufficient and hopefully accurate data than noon reports. However, it is still infeasible in the short term due to practical reasons such as costs and lack of regulations.

These vessel data can be used for vessel performance monitoring and improvement of operations, which has recently become a focus in shipping due to tightening environmental regulations and fierce market competition. For instance, there are today a considerable number of commercial software applications for vessel performance available at market using vessel operational data, e. g. Vessel Performance Solutions(VPS, 2021), Kongsberg Vessel Performance (Kongsberg, 2021), and the performance monitoring system SeaTrend (FORCE Technology, 2021). These systems enable reporting and analysis of vessel performance on several key performance indicators (KPIs), but have a common challenge regarding varying quality of operational data, which impacts the validity of their recommendations. It is current an industry practice to improve DQ from datasets by pre-processing methods such as removing outliers and filling null values, introducing a large risk for further information processing. Therefore, there is a need to develop a DQ quantifying method tailored to the needs of monitoring vessel operations and performance. A detailed literature review about DQ will be conducted in Section 2.

To fill this gap, a new methodology for measuring and improving DQ so as to better monitor vessel performance is developed in this paper. With this method, a set of data quality metrics have been developed for ship operations, which can be applied to first categorize data from specific dimensions, and next to score the data quality of an entire dataset so that the data quality of the entire dataset will be revealed through different severity levels. The rest of this paper is structured as follows. In Section 2, a detailed literature review on data quality assessment is conducted. In Section 3, the proposed method for the evaluating of DQ in vessel operations is introduced, including the data categorizes, data dimension metrics and the scoring criterion. A case study is further conducted on a noon report dataset in Section 4, applying the proposed method. In Section 5, discussions and limitations of the proposed DQ assessment method are provided, and finally concluding remarks are drawn in Section 6.

2   Literature review on the data quality assessment

Data quality is a multidimensional, complex and morphing concept and has evoked much interest in academia and industry since 1980s (Tejay et al., 2004). The effective collection, representation, and application of data are important to an organization because these activities facilitate business operations and business analytics. Errors in data items from organizational databases, data warehouses, and data streams may lead to costly errors in business decisions (Dey and Kumar, 2010). Redman (1998) argued that poor data quality would impact setting strategy, data execution, ability to derive values from data and ability to align the organization. With the booming of Artificial intelligence (AI) and machine learning application in different engineering fields (Cai et al., 2022), the data quality issue is increasingly concerned by researchers. Karagiannidis and Themelis (2021) indicated that the accuracy of his proposed ship fuel consumption model based on operational data has been significantly increased when a proper DQ control was conducted in advance.

Therefore, significant research effort has been spent on estimation of the quality of data. The systematic study of data quality started in late 1990s in the information system field. It is found that the most effective way to evaluate data quality is through attributes or dimensions such as accuracy, reliability, timeliness, accessibility and consistency, representing a single aspect of data quality (Wang and Strong, 1996; Chengalur-Smith et al., 1999). Theoretically, Tejay et al. (2004) argued that the nature and scope of data quality dimensions changed with the changing of different semiotic levels (empirics, syntactics, semantics and pragmatics). Semiotics elucidates the intricacies associated with a sign as it moves from the physical world, where it is created, to the social world of norms. According to this research, people often mixed the dimensions of communication quality (empirics), information quality (semantics) and knowledge quality (pragmatics) into data quality (syntactic). Gradually, investigations of data quality were extended to data warehouses (Wang et al., 2006), e-commerce (Knight and Burn, 2005; Peltier et al., 2013), e-learning systems (Alkhattabi et al., 2011), collaborative business processes (Falge et al., 2012), web data (Caro et al., 2008; Yerva et al., 2012) and wireless sensor networks (Coen-Porisini and Sicari, 2012). The development of DQ assessment methods has been widely applied in various industries, e.g., finance (Pipino et al., 2002), manufacturing (Shankaranarayan et al., 2003) and supply chain and logistics (Hazen et al., 2014).

In the transportation industry in general, data are increasingly becoming a valuable asset. The bad quality of data has caused problems in transportation operations, planning, traffic congestion information, transit and emergency vehicle management, and commercial truck operations (Ahn et al., 2008). To address data quality issues, Turner (2004) gave a definition of data quality as "the fitness of data for all purposes that required it". Measuring data quality requires an understanding of all intended purposes for the dataset. A framework for data quality management was developed and applied by Federal Highway Administration (US Department of Transportation, 2021), which was based on six fundamental measures of traffic data quality namely accuracy, completeness, validity, timeliness, coverage and accessibility. An important use of this framework was to estimate travel time, which is a key measure for the performance of transportation systems, therefore, receiving much attentions from transportation engineers (Eisele and Rilett, 2002). Richardson and Smith (2012) proposed a novel approach to measure errors so that the distribution of the errors can be quantified. Focusing on the automatic data collection systems, Liao and Davis (2012) explored several statistical analysis methods to detect measurement drifts. In their study, a mixture modelling technique using expectation maximization algorithm and cumulative sum (CUSUM) methods were explored for data quality. Jones-Farmer et al. (2014) further provided an overview on the analysis of data for use in process improvement and control charting.

In the specific shipping industry, a few studies have also been found. Røseth (2016) gave an overview of big data, data quality issues and possible solutions. Perera and Mo (2020) proposed a framework to handle high dimensional vessel data. Recently, the widely investigation of data-driven models for the prediction and monitoring of ship opera‐ tional performance has been conducted on both noon reports and sensor data, such as Soner et al. (2018), Soner et al. (2019), Yan et al. (2020), and Karagiannidis and Themelis (2021). Among these applications, DQ was of great importance to determine the model accuracy and their generalization ability, which were pre-processed to some extent by these researchers and should be highly concerned.

According to such a literature review, it is found that visible efforts have been made to improve the data quality in different industries. However, few studies have focused on the quality dimension prioritization and quantitative quality assessment for vessel operational data, to the best knowledge of the authors. Therefore, the investigation to improve the DQ of operational data from vessels is urgently required. With a quantitative assessment method, the DQ of data in current shipping companies from operation can be constantly monitored and improved, which will further facilitate the development of data-driven models and eventually the operational decision-makings.

3   Methodology for assessing DQ of vessel operations

In this section, a specific method is proposed for the evaluation of DQ based on practice of vessel operations and the existing literature for managing DQ. In this method, data are first categorized based on a categorical framework from practical operation functions. Then, a set of quality dimensions are applied on every data category. Afterwards, the quality of each data item after categorization is scored based on proposed validation rules.

3.1   Data categories

In vessel operations, vessel performance can be measured by processing data collected on board such as the noon report and autolog data. Three types of raw data are relevant for the domain of vessel performance: environmental data, vessel response data and machinery data, as seen in Figure 1. Environmental data consist of weather information such as wind, wave, current, temperature variation, and ice, which are normally recorded by crew on board. These data are, in some systems, also be provided by meteorological services.

Figure  1  Data supply chain of vessel operations

Download: Full-Size Img

The vessel response data include the motions due to external environment (e.g., heave and pitch.), and the vessel status (e.g., position, course, speed, draft, trim and speed). GPS and log systems, on-board sensors/devices, or even manual observation are commonly used to record vessel response data. The third main type of raw data source is machinery, which covers all equipment on board such as main engine, auxiliary engine(s), boiler, propeller, air-conditioning, hotel functions, cargo heating/cooling systems, fuel/water pumps and cargo handling equipment. The corresponding data from machinery include information such as engine speed, consumption of fuel and lubrication oil, shaft power, torque, temperatures and pressures.

Based on the data supply chain in vessel operations, a concrete categorical framework is proposed in this paper. The proposed framework consists of 10 general categories, as seen in Table 1. Examples for explanations are given in the following. For the "Main Engine" category, it consists of relevant data items such as the main engine running hours, the revolutions per minute (RPM), main engine power and the main engine fuel consumption. The category "Performance Indicators" includes vessel performance metrics such as SFOC and added resistance of vessels.

3.2   Data dimension metrics

In this section, the data are further assessed along a set of selected dimension metrics. The metrics are presented in Table 2, including the dimension names, concrete descriptions of dimensions and types. The two basic types of data quality metrics are intrinsic quality and contextual quality, respectively, which are originally from the concept of multi-dimensional nature of data quality (Wang and Strong, 1996; Lee et al., 2002). Intrinsic quality denotes the dimensions that are objective and native to the data, while contextual quality refers to the dimensions that are dependent on the context of the task at hand.

Six different metrics are selected in our method for vessel data based on literature, as shown in Table 2. Among them, completeness is defined as to what degree the data records are complete in content without missing data. Validity metric is defined as the degree to which data are conforming to the related rules, including value formatting, data type and realistic range. These rules are often determined by data-owner or users' knowledge on a vessel's navigational, technical and operational characteristics. For instance, the speed of a cargo vessel is measured in knots and can not exceed the value of 30. Once relevant rules are set, validity dimension can be measured for received datasets. Timeliness is defined as whether data are reported at the desired time, according to the industry practice (e.g., noon reports should be sent to shore at noon every day). The fourth-dimension, consistency, can be considered as the presenting of large variety according to the comparison with some equivalent cases that share similar conditions. For instance, if a vessel suddenly consumes twice as much fuel as ordinary during sailing, then this data record is not consistent with the others. The accuracy metric is defined as to what degree the data are equivalent to their corresponding "real" values, which can be accessed via comparison with external values that are known to be correct. The last dimension metric, integrity, is defined as to what degree data of one variable conform to relevant relationship rules with other data variables. For instance, the torque meter on board is used to measure the main engine power value. If the registered values do not conform with, for example, registered values from RPMs, the integrity metric is violated.

3.3   DQ scoring

To quantify the quality of each data item, scoring methods should be finally used. Thus, a scoring method with three levels, namely Good, Warning and Alarm is proposed. Each level is further assigned with a numerical value, which is 0, 10, and 20, respectively, so a total score can be calculated for every data source. As shown in Table 3 for an example, the specific level of data quality (severity) is described. For a data item within validity metric, the level is considered as Good when the calculated values are within a valid range set by vessel operators. The scoring method can be then applied on raw datasets of vessels after properly categorized with different metrics in Sections 3.1 and 3.2.

3.4   System of validation rules and root causes

Using the data categories and the data dimensional metrics from the previous sections, different validation rules are formed, which can be directly applied on the raw dataset for further analysis. Therefore, a concrete DQ validation rule system for vessel operations and performance is produced, which may consist of more than 100 validation rules (the number of rules depend on vessel and company requirements). Table 4 shows an example of the DQ rules for the vessel performance in terms of added resistance and data internal dependencies. A specific ID is assigned to each rule. For the sake of clarity, other validation rules are not presented here.

Dependency relations among rules can be also revealed based on their semiotic hierarchy, as indicated in the last column of Table 4. With the dependencies of validation rules, it is possible to drill down from high level performance metrics to identify lower level root causes of poor data quality. As shown in Figure 2, it is an example of the rule dependency for diagnosis when accounting for the high-level added resistance. In this figure, each circle pie with its ID denotes a specific rule, and the six sub-pieces of each pie denote the DQ severity for noon report in 6 consecutive days. The three different colors indicate the data quality scoring levels, namely red for alarm, yellow for warning and green for good. The added resistance rules (56 and 57), belonging to the category of performance indicator, are the high-level monitoring rules for vessel performance. For instance, when there is a DQ alarm in rule 56 (the top-left sub-piece), the root causes could be traced as the missing of wind speed (14) and/or the missing of wave height (17). Meanwhile, the corresponding validity rules are also alarmed (rules 42 and 43). Another simplified example about the rule dependency is illustrated in Figure 3 for SFOC diagnosis, where the range of SFOC (71) is determined by the range of power (65), the range of fuel consumption (52) and engine consistency (94). These rules are further determined by other low-level validation rules, such as engine run hours (63 and 25), engine RPM (64 and 18) and the missing of engine power or not (26). Such diagnostic results will be directly shared with crew on board, which will facilitate them to improve their data recordings.

Figure  2  An example of relevant rules for the vessel performance in terms of added resistance

Download: Full-Size Img

Figure  3  An example of relevant rules for the vessel performance in terms of SFOC

Download: Full-Size Img

4   Case study and industry application

In this section, a case study is presented based on a real dataset from vessels in the Danish tramp shipping company TORM (TORM, 1889). The proposed method is first integrated into a cloud-based vessel performance analysis engine used by the company. The original noon report dataset will be first described, and then the implementation of the DQ method will be introduced, followed by comparison results.

4.1   The noon report dataset

The datasets consists of data between 2013 and 2017 from the fleet of TORM. Note that, due to the confidential reason, the original dataset will not be shared. Figure 4 shows an overview of the variation of fleet size and the number of noon reports each year. The number of noon reports increase with the increase of fleet size. When old vessels are demolished, e.g., between 2016 and 2017, the corresponding noon report size is decreased. Only six vessels (Vessel A, Vessel B, …, Vessel F) are selected for this case study and the vessel performance in terms of added resistance is used for DQ visualization and comparison for brevity.

Figure  4  Overview of fleet size and noon reports

Download: Full-Size Img

4.2   Implementation of DQ assessment method

The implementation of the DQ method was realized via an industry application. First, the assessment framework with all the validation rules for vessel performance is built. In this step, experienced domain knowledge is needed so as to have proper range checking values for data validation rules. Second, the assessment framework is integrated into the existing vessel performance analysis engine and then connects to its cloud database, which enables the analysis of DQ of all historical noon reports. Lastly, an automatic assessment procedure is built to handle new incoming datasets. Whenever a noon report arrived, the preliminary diagnosis of DQ will be conducted, displaying the quality levels by colors in time-series.

4.3   Comparison results of DQ

Figure 5 illustrates the DQ severity (Good, Warning and Alarm) levels for the noon reports from the selected TORM's vessels between April 10 to April 16 in 2017. Only the data categorized by validation rules related to added resistance are presented. The different color bars denote different severity levels. As observed, the overall data quality in this week is quite good, indicated by the majority of green color. Specifically, there are not alarms or warnings for data in terms of loading condition (category 5), whereas warnings and alarms start to appear for the performance indicator (category 8). There are a few alarms for data in categories 3 and 4. Accounting for both data dimensions and categories, it is found that the noon report data from Vessel A has a perfect quality, while the data quality is getting worse for Vessels C, D, E and F. The root causes of low data quality can be reflected from the diagram. For instance, due to the violations of rules 14, 17, 42 and 43, the performance rules (56 or 57) are displayed with a warning or an alarm. The method can not only instantly signal potential validity issues for vessel performance, but also points towards potential root causes of detected problems. The method thereby enables crew on board and onshore staff to work with continuous improvement of source data and KPI data quality for any (shipping) companies or operators.

Figure  5  DQ (severity) levels for the noon reports from the selected TORM's vessels between April 10 to April 16 in 2017

Download: Full-Size Img

Figure 6 illustrates the number of alarms triggered for all 6 vessels over 15 months. Only the data categorized by validation rules related to added resistance are presented. A monthly interval is adopted in this diagram and only DQ breaches with "alarm" is included. It is observed that there is 414 alarms for rule 57 in Feb 2016 (specific number is measured by the bar height), at the time when the DQ assessment method starts to apply. Then, there is a constant improvement of the overall data quality. For instance, the percentage of data severity in terms of vessel performance (category 8, calculated by overall counts of all DQ levels) drops from 33.4% to 25%, and again drops from 9.0% to 5.8% over one year time for all vessels in general. The DQ assessment thus makes it visible that the DQ of the 6 vessels of the shipping company has been improved significantly from 2016 to 2017. A major reason is the implementation and integration of the DQ assessment method into the vessel performance monitoring platform. It can therefore be concluded that this method is an effective instrument to drive ongoing improvements of quality of source data. Meanwhile, joint efforts from crew and personnel are also needed for such an improvement. It should be mentioned that, due to project closure, we do not have further access to the database any more. Hence, only data before 2017 are presented in this paper.

Figure  6  DQ variation after the assessment system applied in one year

Download: Full-Size Img

5   Discussion

DQ is of great importance in the ongoing digitalization process of shipping industry. Poor DQ costs extra time and energy, which largely compromises proper decision-making. With the exponential growth of data nowadays in shipping companies, the need of advanced DQ monitoring methods, such as the one proposed in this paper, is becoming urgent. The structured DQ assessment in this paper provides a mechanism for continuous monitoring and improvement, being able to capture the data value chain deficiencies to some extent in real time.

5.1   Limitations of the proposed method

In order to make the proposed DQ assessment method work, as we have described in the previous section, the shipping company made a thorough effort to deploy a practical system and keep it running in reality accounting for proper thresholds for validation rules. There is a risk that the method provides inaccurate signals on data quality if there is no mastering of vessel knowledge and operation, which makes technical staff and crew take either wrong or no actions. Therefore, one limitation of this method is that sufficient domain knowledge is always needed for a proper implementation.

In the proposed DQ method, each data item has been scored by three different levels. However, the boundaries of the defined levels are blurry due to changing environment and operation states. The defined boundary values for the validity check rules are somewhat overlapping depending on different vessels or different shipping companies. As seen in Figure 7 for the validity ranges, the boundaries of each DQ level are overlapping due to the differences among physical extremes, realistic extremes and typical extremes. There is a small possibility that the identified alarms (Errors in Figure 7) could be still within the warning domain (Inaccuracy in Figure 7) due to overlapping. Such limitation can't be avoided. Normally, the setting of accurate boundary or threshold values should be done by industry professionals. Based on such reasons, a dedicated ongoing effort is required for a robust vessel performance monitoring system to implement the proposed method, which might be a barrier for shipping industry.

Figure  7  The overlapping of validity ranges

Download: Full-Size Img

5.2   Remaining challenges for DQ assessment in vessel operations

Challenges in further developing and refining of the DQ assessment methods exist. The current method was applied on the noon report datasets from vessel operation. With the digitalization of shipping industry, autolog datasets in high-frequency from sensors on board or datasets from AIS become more and more popular. The direct application of this method on such datasets may be challenging. Further development of the current framework is needed, for instance, the update of DQ dimensions and corresponding validation rules. Besides, how to properly implement advanced mathematical models, such as statistical process control (SPC) methods or other control chart methods, is another challenge that we have to figure out in the following research. The root causes of the identified data problems are crucial. How to smartly diagnose root causes based on machine learning algorithms and AI is a big chal‐ lenge as well. Last but not the least, vessel performance by nature is a very complex phenomenon due to different factors from mechanical responses, environmental effects and operational uncertainties. Therefore, sufficient domain knowledge is needed to have a better assessment. A reliable vessel performance analysis requires not only knowledge from naval architecture but also rich experience from sailing practice. How to properly define the boundaries of the data quality levels, as we have mentioned above, needs extra concerns.

6   Conclusions

In this paper, a well-structured DQ assessment methodology has been proposed. The data quality of the raw dataset from vessel operations can be properly analysed. In this method, 10 different data categories and 6 different data dimension metrics are proposed for the dividing of raw data. More than 100 validation rules are produced within the framework, which can be directly used to assess the DQ of raw datasets via scoring. Three quality levels, namely good, warning, and critical alarm are adopted.

The proposed DQ method has been successfully applied on a real noon report dataset from the Danish shipping company TORM. The DQ of the dataset is assessed and visualized. The comparison results are analysed and results indicate that this method is effective to monitor and thus improve the data quality of noon reports from shipping operations. Root causes could be identified once the internal dependence among validation rules have been clarified, which facilitate actions to improve operations in practice.

This research will help shipping industry reap full benefits by the improvement of DQ in digital ship operations in the near future. Further research is needed to be done in order to address new challenges. For instance, advanced diagnostics should be investigated on the monitoring of unstable periods. Smart diagnostics of the root causes of abnormal alarms should be studied based on new technology such as machine learning algorithms.