1 Introduction

Soil moisture is a critical component of the climate system and serves as an integrative and unifying theme in physical geography (Legates et al., 2011). Evapotranspiration, near-surface atmospheric moisture availability and temperature, and planetary boundary layer instability are all influenced by soil moisture (Pal and Eltahir, 2001; Taylor et al., 2012; Ford and Quiring, 2014; Stéfanon et al., 2014; Wang et al., 2017; Liao et al., 2018). In situ soil moisture has been widely used to investigate the impacts of soil moisture on climate on a variety of timescales (Dirmeyer, 2011; Seneviratne et al., 2013), to validate land surface models (LSMs; Balsamo et al., 2009; Xia et al., 2014) as well as satellite soil moisture products (Gruhier et al., 2010; Collow et al., 2012; Parrens et al., 2012; Holgate et al., 2016), and to provide drought monitoring and early warning (Tang and Piechota, 2009; Wang et al., 2011; Pozzi et al., 2013). Despite the importance of soil moisture in various fields, little has been done to collect and harmonize in situ soil moisture measured in different networks. The International Soil Moisture Network (ISMN) was initiated in 2009 to assemble and homogenize in situ soil moisture measurements from operational networks and validation campaigns on a global scale (Dorigo et al., 2011). There are 55 networks in total that contain over 2200 stations globally available in the ISMN (https://ismn.geo.tuwien.ac.at/en/). Additionally, the North American Soil Moisture Database (NASMD) was initiated in 2011 to provide the harmonized and quality-controlled in situ soil moisture data for different users with various purposes (Quiring et al., 2016). Different from ISMN, the NASMD is established primarily for North America and has approximately twice as many stations in North America as ISMN.

Several quality control (QC) procedures have been applied to NASMD (Quiring et al., 2016), which flag soil moisture measurements that fall outside the range of 0 to 0.6 m³ m^–3, record the same value over 10 days, or deviate more than three times the standard deviation during a 30-day period. However, these do not address all sources of errors, of which an important one is related to complications in frozen soil. Except for the cosmic-ray method, other sensors cannot accurately measure the soil moisture content when soil is partially or completely frozen. In addition, since the dielectric conductivity of ice is significantly lower than that of liquid water, in situ sensors in frozen soil systematically underestimate the soil water content (Hallikainen et al., 1985). However, quality checks based on the frozen/unfrozen soil status have not been conducted when NASMD was developed. Furthermore, using a single maximum threshold, such as 0.6 m³ m^–3, may be inadequate for a robust evaluation of soil moisture QC, because the soil porosity differs substantially among the observational stations in North America (Xia et al., 2015c).

The North American Land Data Assimilation System (NLDAS) aims to generate the spatially and temporally consistent high-quality LSM forcing datasets by using the best available observations, and to provide multi-model output in support of modeling activities. After upgrading the observation-based atmospheric forcing, parameters, and code of LSMs, the NLDAS phase 2 (NLDAS-2) produced a soil temperature dataset that agrees better with observations over much of the conterminous United States than that in the NLDAS phase 1 (Xia et al., 2012a, b, 2013). With the QC procedures of Dorigo et al. (2013), soil moisture measurements in ISMN are quality-controlled 1) based on the geophysical dynamic range and consistency of the observations, and 2) by detecting outliers, breaks, saturation of the signal, and unresponsive sensors based on the spectrum of soil moisture time series. As with ISMN in which Dorigo et al. (2013) developed a QC algorithm to flag spurious soil moisture observations, Xia et al. (2015c) proposed an automated QC methodology to identify spurious and erroneous soil moisture observations in 421 NASMD stations (approximately 23% of all the NASMD stations) based on the NLDAS-2 Noah soil temperature and soil porosity. With the QC procedures of Xia et al. (2015c), soil moisture measurements are flagged if 1) the soil moisture content does not fall within the geophysical range of 0 to 0.6 m³ m^–3, 2) the value in the shallow soil layers (< 60 cm) exceeds the corresponding Noah soil porosity, and 3) the corresponding soil temperature is below zero. The results suggest that the QC approach is efficient to process the in situ soil moisture in NASMD, and the quality-controlled data indeed have a positive influence on assessment of the simulated soil moisture in the four NLDAS-2 LSMs [i.e., Mosaic, Noah, Sacramento (SAC), and Variable Infiltration Capacity (VIC)]. However,Kishné et al. (2017) and Morgan et al. (2017) recently pointed out that the soil hydraulic parameters used in the Noah model were out-of-date, and vertically homogeneous soil properties applied in the Noah model could lead to significant biases in model-simulated fluxes. Therefore, using the same soil porosity value for different soil layers in the QC method of Xia et al. (2015c) may deteriorate the efficiency of QC. To address the existing concerns, this study builds on the prior experience in soil moisture QC and conducts an updated automated QC procedure for all NASMD stations to provide a quality-controlled in situ soil moisture dataset to serve a diverse user community.

The rest of the paper is organized as follows. Section 2 provides a detailed description of the NASMD, NLDAS-2,and datasets that are used for evaluating and generating QC procedures, followed by the experimental design and evaluation of QC procedures. The results of the evaluation in various QC processes based on the benchmark data are shown in Section 3, and the application of QC to NASMD is described in Section 4, followed by the summary and discussion in Section 5.

2 Data, experimental design, and evaluation process 2.1 Data

The data used in this study include the daily in situ soil moisture observations and soil properties provided by NASMD, and simulated daily soil temperature and soil porosity from NLDAS-2 Noah LSM. NASMD provides a harmonized, dense, and quality-controlled network of the soil moisture dataset in North America (Quiring et al., 2016). It has integrated data from more than 32 observation networks comprising over 1800 observation stations in the United States and Canada. However, there are only 1290 observation stations from 22 networks that are currently available for downloads from the NASMD website (http://nationalsoilmoisture.com/About.html). Figure 1a shows the location of the 1290 observation stations included in NASMD, and Table 1 lists the measurement information of the 22 networks, i.e., the station density, record periods, and measurement depths. The number of stations varies from 1 in Central Plains Experimental Range (CPER) to 352 in Snow Telemetry (SNOTEL). However, as each network tends to address a unique objective, measurement techniques and depths are varied in different networks. Metadata, including the location, country, state, observation network, measurement depths, percent sand/silt/clay, soil texture, and bulk density at each depth, have been reported for each station in NASMD. In fact, the in situ observed soil texture data are available for only approximately 69% of the stations included in NASMD (Quiring et al., 2016). Soil texture information for the remaining stations is estimated based on the National Cooperative Soil Survey (NRCS) Soil Survey Geographic Database (SSURGO).

NLDAS-2 has generated land surface water/energy products from four LSMs (i.e., Mosaic, Noah, SAC, and VIC). The NLDAS-2 Noah soil temperature are simulated at the middle of four Noah soil layers (5 cm for the 0–10-cm layer, 25 cm for the 10–40-cm layer, 70 cm for the 40–100-cm layer, and 150 cm for the 100–200-cm layer) (Xia et al., 2015a). However, Mosaic and SAC do not have the soil temperature simulation, whereas VIC typically has three soil layers but their depths vary from grid to grid. The Noah simulated soil temperatures are chosen in this study because the number of layers in the Noah model is the largest among the four models, and the depths are uniform over the NLDAS domain (Xia et al., 2014). The NLDAS-2 Noah model is forced by meteorological data that integrate the North American Regional Reanalysis dataset with gauge observations. The NLDAS domain (25°–53°N, 67°–125°W) with a 0.125° grid covers the central United States, southern Canada, and northern Mexico. Simulations span from 1979 up to date with an hourly temporal resolution. The NLDAS-2 Noah soil temperature has been validated by using in situ observations, which reasonably agrees with the observed soil temperature for different soil layers and on different temporal scales (Xia et al., 2013). Furthermore, Xia et al. (2015b) found that the NLDAS-2 soil texture matches well with observed soil texture. Therefore, the soil temperature simulated from the NLDAS-2 Noah model will be used to flag spurious soil moisture observations recorded under frozen conditions, and the Noah soil porosity data supplemented with the vertical soil porosity data from selected/corresponding stations will be used to flag those exceeding the soil porosity.

For the in situ soil moisture observations from 78 Soil Climate Analysis Network (SCAN) stations, extensive quality-controlled procedures including a visual inspection and automatic detection of spurious observations were conducted by Liu et al. (2011) (Fig. 1b). Problematic observations such as data beyond the physically reasonable range, data showing discontinuity without physical causes, data with inconsistency due to sensor-related changes, and data for frozen soil, were all excluded from the dataset. This manually quality-controlled dataset has previously been used as the ground truth to evaluate the accuracy of soil moisture simulations from LSMs (Cai et al., 2014; Kumar et al., 2014a, 2016; Xia et al., 2014; De Lannoy and Reichle, 2016), and is used in this study as the benchmark to assess performance of the automated QC methods.

2.2 Experimental design

To exclude spurious and erroneous soil moisture observations from the records, the most direct approach is to manually inspect the observed records one by one. However, it is not practical to apply the visual inspection to datasets containing large volumes of records. The more feasible solution is to develop a methodology to automatically identify and flag spurious observations. Xia et al. (2015c) developed an automatic methodology to flag spurious and erroneous soil moisture observations in 421 NASMD stations by using the NLDAS-2 Noah soil temperature and soil porosity. In this study we extend their methodology, with modifications, to all the available NASMD stations (1290 stations in total on the NASMD website). Rules of the QC algorithm are delineated as follows:

(1) Ensuring that measurements are within a geophysical range (GR) between 0.0 and 0.6 m³ m^–3 (GR check). If a soil moisture measurement is less than the minimum threshold 0.0 m³ m^–3 or larger than the maximum threshold 0.6 m³ m^–3, it is flagged as a spurious observation.

(2) Ensuring that measurements do not exceed the in situ soil porosity (SP). Based on the soil texture recorded for each depth at which the soil moisture is measured in NASMD, we estimate the soil porosity for each station based on the field capacity table (Table 2) used in the Noah LSM. If a soil moisture measurement is larger than the soil porosity at the corresponding measurement level and site, it is flagged as a spurious observation.

(3) Ensuring that measurements are not taken in frozen soils (ST0). Based on the NLDAS-2 Noah soil temperature simulated at the four soil layers (i.e., 5, 25, 70, and 150 cm), we generate the daily soil temperature series for each station according to its measurement site by using the nearest-neighbor method. However, depths at which the soil moisture is measured are network-dependent in NASMD (Table 1). Therefore, for each given station, we linearly interpolate the modeled soil temperature to depths of the soil moisture measurement. A soil moisture measurement is flagged as a spurious observation if the corresponding soil temperature drops below 0°C.

The above methodology is similar to Xia et al. (2015c), but with several differences. First, Xia et al. (2015c) used 12 while we use 78 SCAN stations that pass the manual inspection to assess performance of the aforementioned QC approach including GR, SP, and ST0 checks (the first experiment, hereafter called Exp1; see Table 3). Then we repeat these QC procedures for all the stations in NASMD. Second, Xia et al. (2015c) assumed that the soil porosity was vertically uniform in the shallow (< 60 cm) soil layers for each station, while we estimate the soil porosity for each specific soil layer at a given station based on the corresponding soil texture recorded in NASMD. To diagnose the effect of soil porosity on quality-controlled data, we conduct a sensitivity analysis (the second experiment, referred to as Exp2 hereafter) in which the SP check is removed from Exp1. In other words, only GR and ST0 checks are implemented in Exp2. Furthermore, to test the effect of soil texture datasets (vertically uniform in NLDAS-2 versus inhomogeneous in NASMD) on the quality-controlled data, another sensitivity analysis (the third experiment, referred to as Exp3 hereafter) is added. The NLDAS-2 Noah soil porosity is interpolated to each station by using the nearest-neighbor method and then applied to all the soil depths at each station from the various networks in the SP check of Exp3.

2.3 Evaluation metrics

To evaluate how well the QC method performs, a direct comparison between the soil moisture observations before and after the QC and benchmark data at each individual site is conducted for the 78 SCAN stations. Evaluation metrics used in this study include the bias, mean absolute error (MAE), root-mean-square error (RMSE), anomaly correlation (AC), and Taylor skill score (S). The analysis period is from 2000 to 2012, and the observed soil moisture anomaly is the temporal anomaly after removing the mean seasonal cycle. Soil moisture observations before and after QC are compared to benchmarks based on the bias, MAE, RMSE, AC, and S criterion (Taylor, 2001):

$ {\rm{bias}} = \frac{1}{N}\sum\nolimits_{i = 1}^N {\left( {{\rm{OB}}{{\rm{S}}_i} - {\rm{B}}{{\rm{M}}_i}} \right)} , \!\quad\quad $

(1)

$ {\rm{MAE}} = \frac{1}{N}\sum\nolimits_{i = 1}^N \left| {{\rm{OB}}{{\rm{S}}_i} - {\rm{B}}{{\rm{M}}_i}} \right|, \!\!\!\!\!\quad\quad $

(2)

$ {\rm{RMSE}} = \sqrt {\sum \nolimits_{i = 1}^N \frac{{{{\left({{\rm{OB}}{{\rm{S}}_i} - {\rm B}{{\rm M}_i}} \right)}^2}}}{N}}, $

(3)

$ {\rm{AC}} = \frac{{\frac{1}{N}\mathop \sum \nolimits_{i = 1}^N \left({{\rm{OB}}{{\rm{S}}_{{{\rm A}_i}}} - \overline {{\rm{OB}}{{\rm{S}}}}}_{\rm A} \right)\left({{\rm{B}}{{\rm{M}}_{{{\rm A}_i}}} - \overline {{\rm{B}}{{\rm{M}}}}}_{\rm A} \right)}}{{\sqrt {\frac{1}{N}\mathop \sum \nolimits_{i = 1}^N {{\left({{\rm{OB}}{{\rm{S}}_{{{\rm A}_i}}} - \overline {{\rm{OB}}{{\rm{S}}}}}_{\rm A} \right)}^2}} \sqrt {\frac{1}{N}\mathop \sum \nolimits_{i = 1}^N {{\left({{\rm{B}}{{\rm{M}}_{{{\rm A}_i}}} - \overline {{\rm{B}}{{\rm{M}}}}}_{\rm A} \right)}^2}} }}, $

(4)

$ S = \frac{{4\left({1 + R} \right)}}{{{{\left({\sigma + \dfrac{1}{\sigma }} \right)}^2}\left({1 + {R_0}} \right)}}, \quad\quad\quad\quad\quad $

(5)

where N is the total number of days of the soil moisture observation, OBS_i ( $ {{\rm OBS}_{\rm A}}_{_i}$ ) is the observed soil moisture (anomaly), $\overline {{\rm{OB}}{{\rm{S}}}}_{\rm A}$ is the temporal average of observed soil moisture anomaly, BM_i ( $ {{\rm BM}_{\rm A}}_{_i}$ ) is the benchmark (anomaly), $\overline {{\rm{B}}{{\rm{M}}}} _{\rm A}$ is the temporal average of benchmark anomaly, R is the correlation coefficient between the observed soil moisture and benchmark, R₀ is the maximum correlation coefficient in theory (assumed to be 1), and σ is the standard deviation of observed soil moisture normalized by the standard deviation of benchmarks.

Overall, the bias, MAE, and RMSE are evaluated based on absolute values, whereas AC and S are based on the anomalies. Bias is used to evaluate the systematic error in quality-controlled data whereas MAE and RMSE are used to assess its overall error. AC is used to evaluate the capacity of quality-controlled data to capture the daily variability of benchmark data whereas S is to evaluate its capacity to capture the seasonal variance of benchmark data. If the variability and variance of quality-controlled data approach those of benchmarks, AC and S approach 1 (a perfect score). Otherwise, AC and S decrease toward zero, and AC can even be below zero (no skill). In addition, the significance of differences of these evaluation metrics between soil moisture observations after and before QC are tested by using a two-sample t test at the 95% confidence level.

3 Evaluation of various QC processes based on the benchmarks for 78 SCAN stations 3.1 Evaluation of Exp1

Table 4 summarizes the overall percentages of flag occurrences at the five measured soil layers for the 78 SCAN stations. Although the geophysical range of soil moisture (between 0.0 and 0.6 m³ m^–3) has already been used as a rule in the QC procedure in the NASMD development (Quiring et al., 2016), we still detect some observations that fall outside the range. Percentages of the observations flagged as beyond the geophysical range are 1.61%, 0.32%, 0.07%, 0.32%, and 0.29% at the 5-, 10-, 20-, 50-, and 100-cm soil layer, respectively. To investigate this in more details, we analyze the NASMD observations (before the QC procedure of this study is applied, referred to as raw observations hereafter) for the 78 SCAN stations, and find that the flagged measurements mainly come from stations ND_2020 and MS_2110, which are located in North Dakota and Mississippi, respectively. Approximately 80% of the soil moisture measurements at the 5-cm soil layer at station ND_2020 fall outside the geophysical range, whereas the percentage varies from 9.57% to 30.27% at the five measured soil layers at station MS_2110. The result suggests that the GR check for NASMD is still necessary.

For the SP check in Exp1, percentages of the observations flagged as exceeding the in situ soil porosity are 3.17%, 2.69%, 3.68%, 6.33%, and 12.55% at the 5-, 10-, 20-, 50-, and 100-cm soil layer, respectively. However, percentages of the flag occurrences at the first four soil layers are larger than those reported by Xia et al. (2015c) (0.20%, 0.40%, 0.20%, 0.80%, and 9.06% at the 5-, 10-, 20-, 50-, and 100-cm soil layer, respectively). One of the reasons for this difference is that stations with a significant portion of the flagged records in this study are not included in Xia et al. (2015c). Most of the flagged observation records come from the SCAN stations located in Mississippi and Missouri, whereas no station from these two states are included in Xia et al. (2015c). Percentages of the flag occurrences are even up to 50% in some SCAN stations, e.g., stations MS_2032, MS_2110, and MS_2064.

For the ST0 check in Exp1, percentages of the observations flagged for the 78 SCAN stations are 11.17%, 10.64%, 9.58%, 8.73%, and 8.00% at the 5-, 10-, 20-, 50-, and 100-cm soil layer, respectively. Percentages of the flag occurrences monotonically decrease as the soil depth increases, which indicates that the impacts of frozen soils on soil moisture observations are reduced at the deep soil layer because soil at the greater depth freezes less frequently. In addition, the flagged measurements due to the ST0 check mainly come from the SCAN stations in western United States, e.g., Montana, North Dakota, Utah, and Kansas. Because of the high altitude in the western mountainous area, soil freezing occurs more frequently and hence influencing more observations. Overall, the flagging process in Exp1 removes about 15.81% of measurements at the 78 SCAN stations.

3.1.1 Spatial variation of evaluation metrics based on the 78 SCAN stations

Daily AC at the five measured soil layers for the 78 SCAN stations is calculated by using soil moisture observations before and after QC against benchmarks for the period 2000–12. Stations located in Colorado (i.e., CO_2017), Mississippi (i.e., MS_2024 and MS_2025), and Alabama (i.e., AL_2056, AL_2057, and AL_2059) show considerably low AC values (less than 0.5) for the 5-cm soil moisture. By contrast, most of the other stations show good performance with AC values up to 0.9 at the five measured soil layers before the QC procedure, and further improvement is derived from QC in Exp1. Figure 2 shows the spatial distribution of the AC difference between quality-controlled soil moisture observations and raw observations at the five measurement depths for the 78 SCAN stations. Some decrease of AC values, albeit small, is detected at several stations. Instead of reflecting potential weakness of the automatic QC implemented in this study, it may well indicate that even the manual QC may not be able to capture all spurious data, as the rules implemented here are rather fundamental to the physical realism of soil moisture. At most stations, AC values tend to increase for the quality-controlled data, especially for the stations located in Montana, Utah, Missouri, and Mississippi, where the AC difference is significant at the 95% confidence level at the 10-, 20-, and 50-cm soil depths.

The metric S also indicates an improvement in soil moisture observations after QC (Fig. 3). Numbers of stations with positive and negative differences are almost equal at the first four soil layers. However, negative differences of S values are relatively small and not significant. In contrast, positive differences of S values are larger, especially for the stations located in Missouri and Mississippi where the S difference is significant at the 95% confidence level at the first four soil layers.

To evaluate the observation error of soil moisture data before and after the QC, bias, MAE, and RMSE at the five measured soil layers are calculated for the 78 SCAN stations. For raw observations, the bias values show a considerable overestimation at the 5-cm soil layer at stations in North Dakota (because of falling outside the geophysical range), and at the 10-, 20-, 50-, and 100-cm depths in Mississippi. In contrast, the 5- and 10-cm soil moistures are underestimated at the stations in Alabama and Colorado. Bias of the quality-controlled observations generally becomes closer to zero after QC. Figure 4 shows the spatial distribution of the bias difference between quality-controlled soil moisture observations and raw observations at the five measured soil layers for the 78 SCAN stations. As a whole, the bias values consistently decrease at the five soil layers, especially for the stations located in Mississippi where the bias difference is significant at the 95% confidence level. MAE and RMSE values for quality-controlled soil moisture observations are also smaller than those for the NMSMD dataset (figure omitted). Overall, the QC conducted in this study is effective in improving soil moisture observations for the 78 SCAN stations according to the metrics of AC, S, bias, MAE, and RMSE.

3.1.2 Aggregated results

Figure 5 shows boxplots of the metrics of AC, S, bias, MAE, and RMSE at the five soil layers for all observational records of the 78 SCAN stations. Mean values of all metrics of soil moisture observations at all the five layers after the QC procedure are better than those for NASMD. AC (S) values increase by 0.017 (0.025), 0.005 (0.013), 0.008 (0.013), 0.005 (0.029), and 0.001 (0.019) at the 5-, 10-, 20-, 50-, and 100-cm soil layer, respectively, after the QC procedure of Exp1 is conducted. Mean values of bias (MAE, RMSE) decrease by 0.027 (0.026, 0.028), 0.003 (0.003, 0.003), 0.003 (0.003, 0.003), 0.006 (0.004, 0.005), and 0.009 (0.007, 0.007) m³ m^–3 at the 5-, 10-, 20-, 50-, and 100-cm soil layer, respectively, when compared with NASMD. Additionally, ranges and numbers of outliers of the five metrics decrease at the five soil layers after the QC procedure. Overall, mean values of AC and S are closer to 1, whereas bias, MAE, and RMSE are closer to 0 after the QC procedure for the 78 SCAN stations.

To evaluate the seasonal cycle and interannual variability of the quality-controlled soil moisture observations at different depths, quality-controlled data are averaged over all the 78 SCAN stations. The monthly mean soil moisture and its climatology from the benchmarks, raw observations, and quality-controlled observations for the 5-, 10-, and 20-cm soil layers are shown in Fig. 6. Comparisons for the 50- and 100-cm soil layers are excluded because few SCAN stations have the benchmark data at either of these two depths. The results show that raw observations cannot capture the peak of benchmarks at different soil depths, especially at the 5-cm soil depth. However, quality-controlled data closely follow the monthly mean benchmarks for the top three soil depths. For the climatological seasonal cycle, quality-controlled data are much closer to benchmarks, especially in winter and early spring as the QC procedure has flagged unrealistic soil moisture observations caused by soil freezing when compared with raw observations.

Figure 7 compares the benchmarks, raw observations, and quality-controlled observations at stations MS_2110, MS_2032, and MO_2061 at the 5-, 10-, and 20-cm soil layers. Most spurious and erroneous observations are flagged by GR and SP checks at stations MS_2110 and MS_2032. Taking the 5-cm soil layer at station MS_2110 as an example, values of bias, MAE, and RMSE decrease from 0.073, 0.084, and 0.109 to –0.001, 0.023, and 0.026, respectively, whereas values of AC and S increase from 0.865 and 0.617 to 0.946 and 0.959, respectively. As a result, the monthly averaged soil moisture overall becomes closer to benchmarks after the QC procedure for these stations. For station MO_2061, most of spurious observations are flagged by SP and ST0 checks. The monthly averaged quality-controlled observations are much closer to benchmarks, especially in winter and early spring. As a whole, the overall results further suggest that the QC approach is effective in detecting spurious and erroneous observations in NASMD.

3.2 Results from sensitivity tests Exp2 and Exp3

With the SP check removed in Exp2, percentages of the observations flagged due to the effect of frozen soils in Exp2 are slightly larger than those in Exp1 (Table 4). Mean values of AC and S for the 78 SCAN stations in Exp2 are consistently smaller than those in Exp1, whereas mean values of bias, MAE, and RMSE are consistently larger (Fig. 5). For stations where the soil moisture measurement is larger than soil porosity, e.g., MS_2110 and MS_2032 (Fig. 7), many spurious observations cannot be detected by either GR or ST0 check. As a result, the monthly averaged quality-controlled data are consistently larger than benchmarks at these stations. The quality-controlled data without the SP check is not so good as the data that go through the SP check.

With the soil porosity estimated based on the in situ soil texture provided by NASMD replaced with the NLDAS-2 Noah soil porosity in the SP check, Exp3 still shows very similar results to Exp1. Percentages of the observations flagged as exceeding the soil porosity at the five soil measurement layers in Exp1 are slightly larger than those in Exp3 (Table 4). Interestingly, we notice that the percentage of flag occurrences at the 100-cm soil layer is still quite large in this test. The result suggests that for deep soil moisture measurements, there are still a large number of observations exceeding the soil porosity. The performance of Exp3 is as good as Exp1, even at deep soil layers. In addition, there are no obvious differences in the monthly mean soil moisture and climatologically averaged cycle between Exp1 and Exp3 at different soil depths for the 78 SCAN stations (Figs. 6, 7). Similar results from different soil texture datasets suggest that applying the Noah soil porosity throughout the whole soil layers for all stations in Exp3, while not accurate, may be less problematic than deemed. However, the soil texture does vary vertically throughout the soil column (0–200 cm). As a more detailed reflection, the mean absolute difference between the NLDAS-2 Noah soil porosity and estimated soil porosity averaged for the 78 SCAN stations is 0.018 m³ m^–3, whereas the maximum difference (0.056 m³ m^–3) is found at station UT_2132. Therefore, for better precision, in the QC procedure we use the soil porosity estimated from the in situ soil texture at each measured depth in NASMD for the SP check, as opposed to applying the 10-cm soil porosity from the Noah model into all layers throughout the whole soil column as done in Xia et al. (2015c).

3.3 Sensitivity to potential errors in the benchmark data

The metrics presented above evaluate the effectiveness of QC methodology against the benchmark data. Obviously, how accurate they are in reflecting the QC effectiveness depends on the accuracy of benchmark data. As indicated earlier, even the benchmark data may not be free of errors. To check this, the QC methodology was applied to the benchmark data as well. A small fraction of the benchmark data are indeed flagged, with a vast majority of the flagged data found in winter, which is related to uncertainties in the temperature data used; in summer, very few data points are flagged, and 62 out of the 78 benchmark stations are not flagged at all. To examine how potential errors in the benchmark data may influence the perceived effectiveness of QC methodology and to eliminate uncertainties related to the quality of soil temperature data, the performance metrics are re-evaluated based on data from the 62 error-free stations during summer only. While quantitative differences are expected, qualitatively the results are the same as those based on all the benchmark data, and applying QC to NASMD data improves performance metrics. For example, for the 5-cm soil layer at station MS_2032, values of bias, MAE, and RMSE decrease from 0.055, 0.065, and 0.079 to 0.011, 0.029, and 0.034, respectively, whereas values of AC and S increase from 0.870 and 0.404 to 0.923 and 0.702, respectively. In comparison, for the same station and soil layer, when performance metrics are evaluated based on all the benchmark data (including the flagged data in winter), values of bias, MAE, and RMSE decrease from 0.076, 0.080, and 0.090 to 0.022, 0.033, and 0.038, respectively, whereas values of AC and S increase from 0.831 and 0.313 to 0.886 and 0.630 respectively. This indicates that the impact of potential errors in the benchmark data on the perceived effectiveness of QC methodology is rather small.

4 Application of QC to NASMD 4.1 Comparison of mean daily results

There are 1290 observation stations from 22 networks which are currently available for downloads from NASMD. However, it is found that 81 of these stations are located outside the NLDAS domain ranging from 25° to 53°N and 125° to 67°W, and 86 of these stations do not have soil moisture measurement records. Therefore, the automated QC approach is applied to the remaining 1123 NASMD stations. However, for COSMOS, since cosmic-ray techniques can measure the total soil moisture content in frozen soil, the ST0 check of QC procedure is skipped for this network. The spatial distribution of the percentage of flagging by GR, SP, and ST0 checks based on the 1123 NASMD stations for different soil depths is shown in Fig. 8. Stations flagged by the GR check are relatively fewer in number and are distributed randomly in space. The reason for spurious measurements falling outside the geophysical range may be caused by erroneous readings or sensor failures. However, the GR check can effectively remove these spurious values. In general, stations flagged by the SP check are mainly located in the eastern United States, especially for the MAW-Missouri, ICN, and ECONET, whose soil types are silt loam or sandy loam and percentages of flagging by the SP check are 16.31%, 11.27% and 10.30%, respectively. By contrast, stations flagged by the ST0 check are mainly located in western and northern United States. Similar to the evaluation results for the 78 SCAN stations, data for the 1123 NASMD stations are more reliable in states with soils frozen less frequently than those in cold regions. As a result, higher percentages of flagged values happen at high altitudes and latitudes.

Table 5 presents the percentages of flag occurrences for different networks in NASMD. The number of available soil moisture measurements varies from 12, 826 in the CHILI network to 2,529,487 in the SNOTEL network, depending on the years of record, number of stations available in each network, and number of the measured soil depths. Compared with SP and ST0 checks, the percentage of flag occurrences in the GR check is the lowest, varying from 0.00% in AARD, ARM, DEOS, and OM to 5.69% in SDAWN. The percentage of flag occurrences in the SP check varies from 0.00% in AARD to 16.31% in MAW-Missouri, whereas the percentage in the ST0 check varies from 0.00% in CHILI to 46.72% in SNOTEL. Less than 1% of the observations in AARD, AWDN, and SNOTEL are flagged by GR and SP checks together. In contrast, 39.68%, 24.54%, and 46.68% of the observations in these three networks are flagged by the ST0 check, respectively. For networks located in the warmer climate like CHILI, ECONET, and WTM, the ST0 check flags less than 2% of the observations. Generally, including the GR, SP, and ST0 checks, percentages of the flagged observations in ARM, CHILI, OM, and WTM, which are minimally affected by frozen soils, are the smallest (< 5%). We compare the percentages of observations flagged in NASMD with results fromDorigo et al. (2013) who applied similar QC procedures to several soil moisture measurement networks including the ARM, ICN, SNOTEL, and SCAN over North America in ISMN. Percentages of the flag occurrences in GR and SP checks are comparable in these networks, whereas percentages of flag occurrences in the ST0 check of our study are larger than those in Dorigo et al. (2013). The reason for this difference may be caused by the different soil temperature datasets used [Gobal Land Data Assimilation System (GLDAS) Noah soil temperature used in Dorigo et al. (2013) versus NLDAS-2 Noah soil temperature used in this study] and different spatial resolutions (0.25° in GLDAS versus 0.25° in NLDAS-2). Overall, approximately 24.47% of the soil moisture observations in NASMD are removed by the QC procedure.

To further evaluate the effect of QC procedure, we select six networks with 50 or more checked stations to compare the averaged soil moisture observation before and after the QC procedure. Figure 9 presents the comparison of quality-controlled observations (with QC procedure) and averaged raw observations (without QC procedure) at the top soil layer for CRN, MAWN, OM, SNOTEL, SCAN, and WTM. Compared with the raw observations, soil moisture contents in CRN, MAWN, and SCAN generally decrease after the QC check. However, effects of the QC procedure on raw observations are relatively small in OM and WTM. In contrast, most of the flags over SNOTEL, which is located in mountains, at the 5-cm soil depth, occur as expected due to the ST0 check, and soil moisture measurements during winter and early spring are removed after the QC procedure.

In addition, the comparison of quality-controlled observations and averaged raw observations at the shallow soil layers (< 50 cm) for CRN, MAWN, SNOTEL, and SCAN are shown inFig. 10. Effects of the QC procedure on raw observations at deep soil layers are relatively small for the stations in CRN and SCAN, particularly during the warm season. However, effects of the QC procedure on raw observations are substantial for the 10-cm soil water content in MAWN and 50-cm soil water content in SNOTEL. To investigate why averaged raw observations are so wet at the 50-cm soil layer in SNOTEL (Fig. 10f), the original data in NASMD is analyzed, and it is found that there are some extremely large values existing at several stations for a certain period in SNOTEL. The corresponding values of quality-controlled measurements decline as these erroneous measurements are removed by the GR check. Overall, the GR and SP checks flag approximately 11% of the raw observations in MAWN, making the quality-controlled observations smaller than raw observations due to effects of drier soils (Figs. 9b, 10d). Generally, the comparison results suggest that the QC procedure can effectively detect and flag the spurious and erroneous soil moisture measurements in NASMD.

4.2 Availability of QC data

We have generated a quality-controlled daily in situ soil moisture database based on NASMD. The quality-controlled database includes 1123 observation stations from 18 networks that are located in the NLDAS domain and have soil moisture measurement records in the original NASMD. Soil moisture measurements are flagged as “–9999” if 1) the value falls outside the geophysical range of 0 to 0.6 m³ m^–3, 2) the value exceeds the corresponding soil porosity estimated based on the in situ soil texture, and 3) the corresponding soil temperature drops below zero. The quality-controlled database is available for free download at http://www.geosimulation.cn/QualityControlNASMD.html.

5 Summary and discussion

In this study, we revise the automated QC methodology proposed by Xia et al. (2015c), and extend the revised methodology to all the 1123 NASMD stations to develop a more vigorously quality-controlled in situ soil moisture dataset for North America. The revised methodology includes a geophysical range check (GR check), soil porosity check (SP check), and soil temperature check (ST0 check). Soil moisture observations at the 78 example stations in SCAN, which have been used to assess the accuracy of soil moisture simulation from LSMs in many previous studies, are used here as the benchmark to evaluate performance of the QC procedure in this study. The evaluation results show that errors of the quality-controlled data are generally reduced, and anomaly correlation (AC) and Taylor skill score (S) generally increase after the QC procedure. This improvement suggests that the GR, SP, and ST0 checks can effectively detect and flag most spurious and erroneous soil moisture observations for all the 78 stations.

The revised QC approach is further applied to 1123 NASMD stations covering 22 networks. The QC results indicate that the reliability of raw soil moisture observations is the lowest in networks of SNOTEL (47.55% flagged), SDAWN (41.21% flagged), AARD (39.68% flagged), and MAW-Missouri (34.90% flagged), whereas the highest in networks of OM (2.90% flagged), CHILI (3.82% flagged), WTM (4.51% flagged), and ARM (4.68% flagged). As expected, the networks located in high altitudes and latitudes have the highest percentage of flagged observations. Overall, QC checks effectively ensure a geophysically realistic range of soil moisture dynamics (GR and SP checks), and remove the observational records negatively affected by soil freezing (ST0 check).

Different from Xia et al. (2015c), for the SP check in this study, the soil porosity estimated based on the in situ soil texture throughout the whole soil layers for all available NASMD stations, is applied. The sensitivity test removing the SP check from the QC procedure shows that the SP check can effectively flag the soil moisture measurement that exceeds the soil porosity, and the quality-controlled soil moisture values decrease considerably when compared with the raw observations. In addition, another sensitivity test replacing the soil porosity estimated based on NASMD measured soil texture with the NLDAS-2 Noah soil porosity in the SP check suggests that the impact of different soil texture datasets on the quality-controlled data is limited, and the 10-cm NLDAS-2 soil porosity can well detect and flag the spurious observations in the top four soil layers. However, the soil texture more or less vertically varies with the depth throughout the soil column (0–200 cm). Using the soil porosity estimated from the in situ soil texture at each measured depth in NASMD for the SP check is more realistic and desirable.

It is important to note that the manually checked data from the 78 SCAN stations used as benchmarks probably still contain errors, yet which are the best data available for the purpose of this study. Field samples processed by using the thermo-gravimetric method would provide the best benchmark data, but unfortunately such data are not available for the NASMD stations. On the other hand, we would like to point out that this study focuses on flagging the data points that violate thephysical realism of soil moisture, and does not involve re-calibrating the NASMD data based on the benchmark data. Therefore, while potential errors in the benchmark data certainly influence the perceived effectiveness of QC methodology, they do not influence the QC results. For the accuracy of QC results, the quality of soil temperature and soil porosity data are all that matter. Potential errors in the soil temperature and soil porosity data may cause the soil moisture data to be over-flagged or under-flagged. To this end, the National Soil Moisture Network is collaborating with its partners to collect the measured soil temperature for many NASMD sites where corresponding data are available. When these measured soil temperature becomes available in the future, a re-processing procedure using the measured soil temperature will be conducted to further enhance the soil moisture quality. Improving the quality of the soil moisture data is an ongoing process that requires continuous efforts from the community. At the current stage, we believe that the QC product documented here is suitable for use in evaluating the soil moisture from remote sensing, numerical models, and reanalysis data.

Soil moisture is an important component of the coupled land–atmosphere system. It can influence subsequent meteorological conditions (Koster et al., 2016), and hence the more realistic soil moisture initialization can help improve seasonal and subseasonal climate predictions (Kumar et al., 2014b) and operational flood forecast (Wanders et al., 2014). Reliable soil moisture data is critical for agricultural drought monitoring (Anderson et al., 2012) and irrigation demand estimation (El Sharif et al., 2015), and therefore benefiting the water resources planning and management. Other applications using soil moisture data include the landslides and erosion prediction, epidemic risk monitoring, and rainfall estimation (Brocca et al., 2017; Wu et al., 2018). As an important data source for these application systems, quality soil moisture observations are essential and can influence results from application studies. Accurate soil moisture observations over large areas during long periods are very important to both practical applications and fundamental research. To this end, this study makes contributions through quality controlling an important observational soil moisture database for North America.

Recently, remote sensing based soil moisture data have gained increasing recognition, and have been incorporated into many application systems owing to the more complete coverage in both space and time. However, the uncertain data quality and limited soil depth of remote sensing data are impeding their broader application. In contrast, in situ observations are usually more accurate and can reach greater soil depth, but the limited spatial coverage and relatively coarse temporal resolution are limiting factors. Combining the in situ and remotely sensed data can lead to better soil moisture products that can address challenges of both types of data. Such efforts should take advantage of quality controlled products such as what is produced in this study.

Acknowledgments. We thank the scientists who maintain the soil moisture networks used in this study. We appreciate Steven M. Quiring and his group for collecting in situ soil moisture observations to form NASMD. Without their efforts and support, it would be impossible to accomplish this work. We are also grateful to the four anonymous reviewers for their constructive comments that have improved this article significantly.