Gio Global Land Component - Lot I ”Operation of the Global Land … · GIO-GL Lot 1, GMES Initial Operations Date Issued: 18.04.2017 Issue: l2.10 Gio Global Land Component - Lot

GIO-GL Lot 1, GMES Initial Operations Date Issued: 18.04.2017 Issue: l2.10

Gio Global Land Component - Lot I ”Operation of the Global Land Component”

Framework Service Contract N° 388533 (JRC)

QUALITY ASSESSMENT REPORT

AREA OF WATER BODIES

SPOT/VGT AND PROBA-V

VERSION 2

Issue 2.10

Organization name of lead contractor for this deliverable: HYGEOS

Book Captain: Mathieu Compiègne (HYGEOS)

Contributing Authors:

GIO-GL Lot 1, GMES Initial Operations Date Issued: 18.04.2017 Issue:l2.10

Document-No. GIOGL1_QAR_WBV2 © GIO-GL Lot1 consortium

Issue: l2.10 Date:18.04.2017 Page: 2 of 42

Dissemination Level PU Public X PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the consortium (including the Commission Services) CO Confidential, only for members of the consortium (including the Commission Services)




Document Release Sheet

Book captain: Mathieu Compiègne Sign Date 18.04.2017

Approval: Roselyne Lacaze Sign Date 18.04.2017

Endorsement: Michael Cherlet Sign Date

Distribution: Public




Change Record

Issue/Rev Date Page(s) Description of Change Release

15.06.2016 All Quality assessment of VGT and PROBA-V global Water Bodies Version 2 products

I1.00

I1.00 05.10.2016

14–17; 32-36

Revision after JRC review:

- information about reference dataset - recommendations

I1.10

I1.10 19.01.2017 19-44 Update according to recommendations of external review

I2.00

I2.00 18.04.2017 42 Finalization for public release I2.10




TABLE OF CONTENTS

1 Background of the document ................................................................................................. 10

1.1 Executive Summary ........................................................................................................ 10

1.2 Scope and Objectives ..................................................................................................... 10

1.3 Content of the document ................................................................................................. 11

1.4 Related documents ......................................................................................................... 11

1.4.1 Applicable documents .............................................................................................. 11

1.4.2 Input ........................................................................................................................ 11

1.4.3 Output ...................................................................................................................... 11

2 Review of Users Requirements .............................................................................................. 12

3 Quality Assessment Method .................................................................................................. 14

3.1 Overall procedure ........................................................................................................... 14

3.2 The Products under Analysis: ......................................................................................... 14

3.2.1 The PROBA-V WB V2 ............................................................................................. 14

3.2.2 The VGT WB V2 ...................................................................................................... 16

3.2.3 The 32-year global scale water history record (GWHR) and Water Surface Ratio map 16

3.3 Evaluation procedures, sample & Metrics ....................................................................... 18

3.3.1 Water surface QA .................................................................................................... 22

3.3.2 Water detection QA ................................................................................................. 24

4 Results ................................................................................................................................... 25

4.1 CGLS-WB Coastal pixels ................................................................................................ 25

4.2 Characterisation of the sample ....................................................................................... 25

4.3 Water Surface QA ........................................................................................................... 30

4.3.1 Temporal variability .................................................................................................. 32

4.3.2 VGT/PROBA-V overlap period ................................................................................. 36

4.4 Water detection QA ........................................................................................................ 38

5 Conclusions ........................................................................................................................... 40

6 References ............................................................................................................................ 42




List of Figures

Figure 1: Global Water History Record for July 2015. Pink areas show pixels where water was detected during the month. The resolution is 30 meters. The map is centered on lat/lon = 46.06°N./-79.58°E and has a size of ~125x125km. ................................................................ 17

Figure 2: Water Surface Ratio (WSR) map as computed for July 2015 from GWHR product (Figure 1). The grayscale corresponds to the water surface ratio going linearly from 0 (black) to 100% (white) .................................................................................................................................... 18

Figure 3: Biome stratification of Earth's land masses. The 14 eco-regions of Olson et al. (2001) were aggregated into 8 general biomes, with additional “Greater Lakes” and “glacier” classes.

.............................................................................................................................................. 20

Figure 4: Land/Sea mask used to reject coastal zone from the Quality Assessment ..................... 21

Figure 5: Limits for the higher latitude cutoff as a function of the month of the year in JRC GWHR product (pixels with latitude higher than these limits are stamped as unobserved). ................ 22

Figure 6: CGLS-WB V2 product over Britain area (France). Water (70) is grey, Sea (0) is black and Land (255) is white. Left panel is PROBA-V product, right panel is VGT product. .................. 25

Figure 7: Pixel repartition (excluding dry land) in GWHR-WSR (whatever their status in CGLS-WB product). The left panel corresponds to VGT time series and the right panel corresponds to PROBA-V time series. ........................................................................................................... 26

Figure 8: Temporal variability of the total surface covered by the pixel sample used for the validation. Left panel corresponds to the global surface for the entire time series for VGT dataset. Right panel is a close up for the last year of VGT dataset with detail for each biome.

.............................................................................................................................................. 27

Figure 9: Temporal variability of the surface covered by the pixel sample used for the validation for PROBA-V dataset. ................................................................................................................. 27

Figure 10: WSR repartition for pixels classified as water in the CGLS-WB product. Upper panel corresponds to the VGT sample and lower panel corresponds to the PROBA-V sample. ...... 29

Figure 11: Overall accuracy, commission error and omission error for entire VGT (left panel) and PROBA-V (right panel) dataset as detailed for each biome (plus Great Lakes). The dashed lines show the global results. ................................................................................................. 31

Figure 12: CE and OE for the Northern hemisphere averaged over the entire VGT sample as a function of the biome and for each month over the year. ........................................................ 33

Figure 13: CE and OE for the Northern hemisphere averaged over the entire PROBA-V sample as a function of the biome and for each month over the year. ..................................................... 34




Figure 14: CE and OE for the Southern hemisphere averaged over the entire VGT sample as a function of the biome and for each month over the year ......................................................... 35

Figure 15: CE and OE for the Southern hemisphere averaged over the entire PROBA-V sample as a function of the biome and for each month over the year. ..................................................... 36

Figure 16: Omission error for both VGT and PROBA-V during the overlapping period for the two datasets. The dashed lines show the values corresponding to the entire overlapping time series. .................................................................................................................................... 37

Figure 17: Commission error for both VGT and PROBA-V during the overlapping period for the two datasets. The dashed lines show the values corresponding to the entire overlapping time series. .................................................................................................................................... 37

Figure 18: Omission error for water detection QA as a function of the WSRthres ............................ 38

Figure 19: Commission error for water detection QA as a function of the WSRthres ........................ 39




List of Tables

Table 1: GCOS Requirements for water bodies as Essential Climate Variables [GCOS-154, 2011]. .............................................................................................................................................. 13

Table 2: Surface types as defined in the GL WB Version 2 products............................................. 15

Table 3: Legend of the Quality Layer of PROBA-V WB V2 ............................................................ 15

Table 4: Aggregation of the Olson et al. (2001) biomes into eight general biomes used for the present study plus two additional areas (shaded in grey: Greater lakes and Glacier). For each biome, the corresponding surface after projection on CGLS-WB grid is given. ....................... 19

Table 5: Definition for the confusion matrix elements for Water Surface QA. WSRi is the water surface ratio (between 0 and 1) of pixel i. Ai is the surface of pixel i (varying with Latitude). Atot is the total surface of the sample ............................................................................................ 23

Table 6: Definition for the confusion matrix elements for Water detection QA. .............................. 24

Table 7: Pixel number repartition for the whole sample corresponding to VGT ............................. 28

Table 8: Pixel number repartition for the whole sample corresponding to PROBA-V ..................... 28

Table 9: Confusion matrix for the entire VGT dataset. Values are unitless. ................................... 30

Table 10: Confusion matrix for the entire PROBA-V dataset. Values are unitless. ........................ 30

Table 11: Minimum WSRthres values for which PROBA-V and VGT products are compliant with GCOS requirements in terms of omission error. ..................................................................... 41




List of Acronyms

ATBD Algorithm Theoretical Basis Document CEOS Committee for Earth Observation Satellite CGLS Copernicus Global Land Service ECV Essential Climate Variables ESA European Space Agency GCOS Global Climate Observing System GIO GMES Initial Operations GL Copernicus Global Land service GMES Global Monitoring for Environment and Security GTN-L Global Terrestrial Network for Lakes GWHR Global Water History Record dataset HSV Hue, Saturation, Value IFOV Instantaneous Field of View JRC Joint Research Centre NDWI Normalized Difference Water Index PUM Product User Manual PROBA-V VEGETATION sensor on-board Project for On-Board Autonomy (PROBA)

satellite RGB Red Green Blue SPOT/VGT VEGETATION sensor on-board Satellite Pour l'Observation de la Terre (SPOT) SVP Service Validation Plan SWB Small water bodies UNFCCC United Nation Framework Convention on Climate Change WB Water Bodies WBDA Water Body Detection Algorithm WBO Water Body Occurrence WGS World Geodetic System WSR Water Surface Ratio




1 BACKGROUND OF THE DOCUMENT 1.1 EXECUTIVE SUMMARY

The Global Land (GL) Component in the framework of GMES Initial Operations (GIO) is earmarked as a component of the Land service to operate “a multi-purpose service component” that will provide a series of bio-geophysical products on the status and evolution of land surface at global scale. Production and delivery of the parameters are to take place in a timely manner and are complemented by the constitution of long term time series.

Quality Assessment constitutes the only means of guaranteeing the compliance of generated products with user requirements. It concerns new products which must pass an exhaustive scientific evaluation before to be implemented operationally. The procedure follows, as much as possible, the guidelines, protocols and metrics defined by the Land Product Validation (LPV) group of the Committee on Earth Observation Satellite (CEOS) for the validation of satellite-derived land products. They are described in the Service Validation Plan [GIOGL1_SVP].

Many of the products of the Global Land service are derived from SPOT/VEGETATION sensor data which has been switched off at the end of May 2014. To ensure the continuity of the service at 1km resolution, the methodologies and the processing lines have been adapted to the data of the PROBA-V sensor.

The Global Land service provides a Water Bodies (WB) product which maps the pixels identified as water bodies. The Version 1 of products covers the African continent only. The Version 2 of products covers the whole globe. Version 1 is derived using the Small Water Body (SWB) algorithm (Gond et al., 2004), calibrated for arid and semi-arid regions Details can be found in the ATBD [GIOGL1_ATBD_WBV1]. Version 2 methodology relies on a Red-Green-Blue (RGB) to Hue-Saturation-Value (HSV) transformation (Pekel et al., 2014). Thresholds have been updated to SPOT/VGT [GIOGL1_ATBD_WB-V2-VGT] and PROBA-V data [GIOGL1_ATBD_WB-V2-VGT].

1.2 SCOPE AND OBJECTIVES

The document presents the results of the quality assessment (QA) of Copernicus Global Land Service (CGLS) WB V2 products. This includes the VGT and PROBA-V products.

The quality assessment is performed over the whole globe. The spatial grid is regular in Longitude and Latitude with a resolution of 1x1 km2 at the Equator. The temporal resolution is 10 days covering the period from January 1st, 1999 to May 31th, 2014 for VGT data and January 1st, 2014 to October 31th, 2105 for PROBA-V.

The objective is to check the quality (e.g. overall accuracy; commission error, omission error) of the CGLS WB V2 product evaluating if their performance is sufficient to be disseminated to users.




1.3 CONTENT OF THE DOCUMENT

This document is structured as follow:

• Chapter 2 recalls the users requirements, and the expected performance • Chapter 3 describes the methodology for quality assessment, the metrics and the criteria of

evaluation • Chapter 4 presents the results of the analysis • Chapter 5 summarizes the main conclusions of the study

1.4 RELATED DOCUMENTS

1.4.1 Applicable documents

AD1: Annex II – Tender Specifications to Contract Notice 2012/S 129-213277 of 7th July 2012

AD2: Appendix 1 – Product and Service Detailed Technical requirements to Annex II to Contract Notice 2012/S 129-213277 of 7th July 2012

1.4.2 Input

Document ID Descriptor

GIOGL1_SSD Service Specifications of the Global Component of the Copernicus Land Service.

GIOGL1_SVP Service Validation Plan of the Global Land Service

GIOGL1_ATBD_WBV1 Algorithm Theoretical Basis Document of the WB V1 product.

GIOGL1_ATBD_WB-V2-VGT Algorithm Theoretical Basis Document of the SPOT/VGT WB V2 product

GIOGL1_ATBD_WB-V2-PROBAV

Algorithm Theoretical Basis Document of the PROBA-V WB V2 product.

GIOGL1_QAR_WBV1 Quality Assessment Report of the Version 1 of Water Bodies products over Africa

1.4.3 Output

Document ID Descriptor

GIOGL1_PUM_WBV2 Product User Manual the WB Version 2 product




2 REVIEW OF USERS REQUIREMENTS

According to the applicable document [AD2], the user’s requirements relevant for the areas of the WB products are:

• Definition: permanent and seasonal water bodies, natural and man-made, independently of their size. Include but are not restricted to the lakes of the Global Terrestrial Network for Lakes (GTN-L).

• Geometric properties:

o The baseline pixel size shall be 1 km. o The target baseline location accuracy shall be 1/3 of the at-nadir instantaneous field

of view. o Pixel co-ordinates shall be given for the centre of pixel.

• Geographical coverage:

o Geographic projection: regular lat-long o Geodetical datum: WGS84 o Pixel size: 1/112° - accuracy: min 10 digits o Coordinate position: centre of pixel o Global window coordinates:

Upper Left:180°W-74°N Bottom Right: 180°E 56°S

o African window coordinates: Upper Left: 26°W – 38°N Bottom Right: 60°E - 35°S

• Time definition:

As a baseline, the biophysical parameters are computed by and representative of dekad, i.e. for ten-day periods (“dekad”) defined as follows, for each month of the year: days 1 to 10, days 11 to 20 and days 21 to end of the month.

o The output data shall be delivered in a timely manner, i.e. within 3 days after the end of each dekad.

• Ancillary information: o The number of measurements per pixels used to generate any synthesis period o The per-pixel date of the individual measurements or the start-end dates of the

period actually covered o Quality indicators, with explicit per-pixel identification of the cause of anomalous

parameter results




• Accuracy requirements:

o Baseline

o

: wherever applicable the bio-geophysical parameters should meet the internationally agreed accuracy standards laid down in document "Systematic Observation Requirements for Satellite-Based Products for Climate". Supplemental details to the satellite based component of the "Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC (GCOS-154, 2011)" (see Table 1) Target

Regarding this latter accuracy requirement, the water surface biophysical variable corresponds to two different Essential Climate Variables (ECV), i.e. the “land cover” and the “lakes”.

: considering data usage by that part of the user community focused on operational monitoring at (sub-) national scale, accuracy standards may apply not on averages at global scale, but at a finer geographic resolution and in any event at least at biome level.

Table 1: GCOS Requirements for water bodies as Essential Climate Variables [GCOS-154, 2011].

Variable/ Parameter

Horizontal Resolution

Vertical Resolution

Temporal Resolution Accuracy Stability

Maps of land-cover type 250 m N/A 1 year

15% (maximum error of omission

and commission in mapping individual classes), location accuracy better

than 1/3 IFOV with target IFOV 250m


and commission in mapping individual classes), location accuracy better

than 1/3 IFOV with target IFOV 250m

Areas of GTN-L Equivalent to 250 m N/A monthly


and commission in lake area maps), location accuracy better than 1/3 IFOV with target

IFOV 250m

5% (maximum error of omission and

commission in lake area maps),

location accuracy better than 1/3 IFOV with target

IFOV 250m




3 QUALITY ASSESSMENT METHOD

3.1 OVERALL PROCEDURE

The Quality Assessment (QA) follows the procedures described in the GL Service Validation Plan [GIOGL1_SVP]. Despite the areas of Water Bodies are not included into the focus areas of the Land Product Validation group of CEOS, the validation protocol relies on similar aspects: checking the spatial and temporal consistency, and quantifying the accuracy of the water detection. This quality is performed taking the 32-year global scale water history record (GWHR) dataset by EC JRC (Pekel et al., 2016) as the reference data. This dataset is provided globally with a spatial resolution of 30 meters and almost covers the full Copernicus Global Land Service Water Bodies (CGLS-WB) products time range. The QA is then performed pixel by pixel globally and for nearly the full period covered by CGLS-WB product.

Because of the small spatial extent of the inland water surfaces (< 3% of the global land surfaces), and because some of them are seasonal, a validation approach based on a set of pixels selected randomly at global scale would yield too few samples containing water surfaces, and thus would not allow a proper estimation of detection performance. Such a problematic is also present for burned area validation and requires a sophisticated sampling for the QA to be statistically representative (see Boschetti et al., 2016). Fortunately, the reference data that we use here covers the whole globe at a spatial resolution higher than the Copernicus Global Land Service (CGLS) Water Bodies products. No random selection of the pixels is necessary as we use all valid pixels as the sample for the validation.

Regarding temporal consistency, the reference GWHR dataset has a temporal frequency of 1 month (flag whether water was detected during a given month). Each of the CGLS-WB dekad will be compared with the reference data for the month it falls into.

3.2 THE PRODUCTS UNDER ANALYSIS:

3.2.1 The PROBA-V WB V2

The methodology to retrieve the PROBA-V WB V2 products is described in the ATBD [GIOGL1_ATBD_WB-V2-PROBAV]. The RGB to HSV transformation was used by Pekel et al. (2014) to detect water bodies in MODIS time series. This approach has been adapted to PROBA-V dataset. The Water Bodies Detection Algorithm (WBDA) consists of four major steps: 1) the PROBA-V daily Top-Of-Canopy reflectances are first composited in a 10-daily synthesis (MC10), associated with a status map; 2) the SWIR, NIR and RED bands are then transformed to HUE,




SATURATION and VALUE using a RGB to HSV transformation; 3) subsequently, the application of specific threshold values on HUE and VALUE applied per pixel while taking into account some masks (MC10 status map, WB Potential mask, Permanent Glacier mask and the Volcanic Soils mask) allows water body detection; 4) finally, a Water Body Occurrence (WBO) is calculated and as such provides additional qualitative information. For each pixel, the detection algorithm assigns a surface type according to Table 2.

Table 2: Surface types as defined in the GL WB Version 2 products

Value Legend 0 Ocean 70 Water 251 No data 255 No water (Dry land)

Table 3: Legend of the Quality Layer of PROBA-V WB V2

Value Label

0 Sea

71 Very Low

72 Low

73 Medium

74 High

75 Very High

76 Permanent

151 Lowland vegetation

152 Mountain vegetation

241 Glacier

242 Volcanic

243 Mountain no vegetation

244 Lowland no vegetation

251 No data

252 Cloud

253 Snow

254 SZA > 45°




The PROBA-V WB V2 products are described in the Product User Manual [GIOGL1_PUM_WBV2]. They include two (2) single bands GEOTIFF format file containing the following layers:

• WB: the water bodies detection (Table 2) • QL: the Quality Layer represents the Water Bodies Occurrence (WBO) if the base layer

indicates ‘Water’ or the source of masking if the base layer indicates ‘No water’ or ‘No data’. The values of the digital numbers in the QL are summarized in Table 3. The WBO values are shaded blue in the Table 3.

We perform this Quality Assessment on 66 dekads corresponding to the period from January 1st 2014 to October 31th 2015.

Remark: The algorithm is applied for all pixels considered as “land” from the original PROBA-V land-sea mask where the coastal strip is included. As a consequence, the coastal zone is stamped “water” in the WB product. On the other hand, the “ocean” part of this land sea mask includes large lakes. As a consequence, the coastal zone of large lakes is detected as “water” but not their central part which is labeled as “ocean”.

3.2.2 The VGT WB V2

The methodology used to retrieve the VGT WB V2 products is nearly the same as for PROBA-V only some thresholds have been adapted [GIOGL1_ATBD_WB-V2-VGT]. The product format is identical to the PROBA-V one.

We perform this Quality Assessment on 555 dekads corresponding to the period from January 1st, 1999 to May 31th, 2014.

Remark: The detection algorithm is applied for all pixels considered as “land” from the FAO’s GAUL coast line vector data. Using this mask, there is no coastal strip stamped as “water” in the product as it is in PROBA-V. The “ocean” part of this land sea mask also includes some large lakes but not any as in the PROBA-V mask. For example, African lakes are all and entirely labeled as water bodies.

3.2.3 The 32-year global scale water history record (GWHR) and Water Surface Ratio map

The 32-year global scale water history record (GWHR) dataset is given at a resolution of 30 meters and can be considered as high resolution regarding the resolution (1km) of the CGLS Water Bodies products. Classification performance, measured using over 40,000 reference points, confirmed that the classifier produces less than 1% of false water detections, and misses less than 5% of water. The GWHR dataset consists of global maps for each month starting from 1985 to October 2015. For a given month, two binary (0/1) maps are provided (i) one telling if a valid




observation (clear sky situation) was available and (ii) one telling if water was detected. No time stamp is given regarding the water detection within a month. Figure 1 shows an example for the water mask in July 2015.

Figure 1: Global Water History Record for July 2015. Pink areas show pixels where water was

detected during the month. The resolution is 30 meters. The map is centered on lat/lon = 46.06°N./-79.58°E and has a size of ~125x125km.

In order to perform the quality assessment, GWHR product is re-sampled (using Google Earth Engine) into the same grid as the CGLS WB products. For each month, two maps are computed:

(i) the fraction, in %, of 30 meters pixels, located into the 1km pixels, that have a valid observation. It is called the Observed Surface Ratio (OSR) map.

(ii) the fraction, in %, of 30 meters pixels, located into the 1km pixels, that were identified as water. It is called Water Surface Ratio (WSR) map. Figure 2 shows the WSR map obtained from the map of Figure 1.




Figure 2: Water Surface Ratio (WSR) map as computed for July 2015 from GWHR product (Figure 1).

The grayscale corresponds to the water surface ratio going linearly from 0 (black) to 100% (white)

3.3 EVALUATION PROCEDURES, SAMPLE & METRICS

The validation is performed at global scale for the period from January 1st 2014 to October 31th 2015 for PROBA-V and from January 1st, 1999 to May 31th, 2014 for VGT. Although the GWHR dataset is only varying on a monthly basis, the validation may be reported dekad by dekad. Three CGLS decadal maps are compared with a single GWHR map.

Results are presented globally and also for individual eco-regions (biomes). We use the eco-region map given by Olson et al. (2001). We merged their 14 biomes into 8 general biomes as reported on Table 4. In addition to the 8 general biomes, we also use the “Greater Lakes” and “Glacier” class from Olson et al. (2001). Figure 3 shows the geographical repartition of those biomes.




Table 4: Aggregation of the Olson et al. (2001) biomes into eight general biomes used for the present study plus two additional areas (shaded in grey: Greater lakes and Glacier). For each biome, the

corresponding surface after projection on CGLS-WB grid is given.

General biomes Corresponding Olson 2001 biomes Surface (106 km2)

Tropical Forest • Tropical and subtropical moist broadleaf forests • Tropical and subtropical dry broadleaf forests • Tropical and subtropical conifer forests • Mangrove

24.28

Temperate forest • Temperate Broadleaf and missed Forests • Temperate conifer forests

17.13

Boreal forest and tundra • Boreal forests/Taiga • Tundra

22.96

Tropical Savanna • Tropical and sub-tropical grasslands, savannas and shrublands

20.54

Temperate savanna • Temperate Grasslands, savannas and shrublands • Montane grasslands and shrublands

15.49

Flooded grasslands and savanna

• Flooded grasslands and savanna 1.10

Mediterranean • Mediterranean 3.27

Desert and Xeric shrublands

• Desert and Xeric shrublands 28.32

Greater Lakes • Greater Lakes 1.05

Glacier • Glacier 1.20




Figure 3: Biome stratification of Earth's land masses. The 14 eco-regions of Olson et al. (2001) were

aggregated into 8 general biomes, with additional “Greater Lakes” and “glacier” classes.

For pixels to enter the validation sample, it has to verify the following conditions:

• being stamped as Water (70) or Dry Land (255) in the CGLS WB product • having a GWHR “observed surface ratio” (OSR) of 100% (all GWHR 30 meter pixels

subsequent to the 1km CGLS pixel are observed once during the month)

The validation sample is made of most valid pixels in the CGLS-WB / GWHR-WSR data. No issue related to a random spatio-temporal sampling of a sub-set of pixels is to be considered here.

As described in section 4.1, coastal zones are stamped as inland water in the CGLS PROBA-V WB product. So, for this analysis, we reject these coastal zone pixels applying the land/sea mask in Figure 4. Not doing that rejection would clearly bias the result of the QA that must focus on inland water bodies. The Land/Sea mask rejects some Great Lakes (see North America) from the QA by classifying it as “Sea”. This mask is applied to all data: the WSR maps, the PROBA-V WB and the VGT WB products.




Figure 4: Land/Sea mask used to reject coastal zone from the Quality Assessment

Some limitations related to the present methodology must be raised or reminded:

(i) The computed omission and commission error are based upon the comparison to the high spatial resolution GWHR product, not regarding a ground truth.

(ii) The GWHR is given monthly while the CGLS SPOT/VGT and PROBA-V products are provided every 10 days. Three dekads of CGLS products are compared to the same GWHR each month.

(iii) The GWHR does not fully cover the same latitude as the SPOT/VGT and PROBA-V products. It is more conservative in cutting zones potentially covered with snow during winter time. The northern limit at winter time goes down 30° (see Figure 5).




Figure 5: Limits for the higher latitude cutoff as a function of the month of the year in JRC GWHR

product (pixels with latitude higher than these limits are stamped as unobserved).

In the following, we separate between two different approaches for the validation. Those approaches reveal different characteristics of the CGLS-WB data that depends on the intended use of it:

1. The CGLS-WB product is seen as a map of the area covered with water. Then, a pixel flagged as “water” in the CGLS-WB product is considered as fully covered with water. In that situation, we may compare the land/water area coincidence between the CGLS-WB and the GWHR dataset. In the following, we refer this case as the “water surface” QA.

2. The CGLS-WB is seen as a detection flag within each pixel. A pixel flagged as “water” in the product means water was detected in the pixel area. In that situation, the occurrence of land/water pixel may be checked regarding the actual pixel content given a threshold WSR. In the following, we refer this case as “water detection” QA.

3.3.1 Water surface QA

Here, the confusion matrix is a two-way matrix composed of the proportion of water and land surfaces correspondence between the CGLS-WB product and the reference GWHR-WSR for pixels of the sample.

Table 5 shows the definition for the confusion matrix. p11, p12, p21 and p22 may be computed for each dekad or for the entire time series, for a given biome or globally.




Table 5: Definition for the confusion matrix elements for Water Surface QA. WSRi is the water surface ratio (between 0 and 1) of pixel i. Ai is the surface of pixel i (varying with Latitude). Atot is the total

surface of the sample

GWHR Reference

Water Land

CGLS

data

Water 𝑝11 =∑ 𝑊𝑆𝑅𝑖 𝐴𝑖𝑖∈[𝐶𝐺𝐿𝑆 𝑤𝑎𝑡𝑒𝑟]

𝐴𝑡𝑜𝑡 𝑝12 =

∑ (1 −𝑊𝑆𝑅𝑖)𝐴𝑖𝑖∈[𝐶𝐺𝐿𝑆 𝑤𝑎𝑡𝑒𝑟]

𝐴𝑡𝑜𝑡

Land 𝑝21 =∑ 𝑊𝑆𝑅𝑖 𝐴𝑖𝑖∈[𝐶𝐺𝐿𝑆 𝑙𝑎𝑛𝑑 ]

𝐴𝑡𝑜𝑡 𝑝22 =

∑ (1 −𝑊𝑆𝑅𝑖)𝐴𝑖𝑖∈[𝐶𝐺𝐿𝑆 𝑙𝑎𝑛𝑑]

𝐴𝑡𝑜𝑡

The four elements can also be defined as follow:

• P11 is the sum over all pixels i being classified as “water” in the CGLS-WB product, of the pixel surfaces (Ai) weighted by the Water Surface Ratio WSRi of the pixels (from the GWHR-WSR, see 3.2.3), divided by the total surface of the sample. This corresponds to the surface fraction of the sample being rightly classified as “water” in CGLS-WB.

• P12 is the sum over all pixels i being classified as “water” in the CGLS-WB product, of the pixel surfaces (Ai) weighted by the land surface ratio (1-WSRi) of the pixels, divided by the total surface of the sample. This corresponds to the surface fraction of the sample being mistakenly classified as “water” in the CGLS product.

• P21 is the sum over all pixels i being classified as “land” in the CGLS-WB product, of the pixel surfaces (Ai) weighted by the water surface ratio WSRi of the pixels, divided by the total surface of the sample. This corresponds to the surface fraction of the sample being mistakenly classified as “land” in CGLS-WB product.

• P22 is the sum over all pixels i being classified as “land” in the CGLS-WB product, of the pixel surfaces (Ai) weighted by the land surface ratio (1-WSRi) of the pixels, divided by the total surface of the sample. This corresponds to the surface fraction of the sample being rightly classified as “land” in the CGLS_WB product.

where WSRi is the water surface ratio (between 0 and 1) of pixel i, Ai is the surface of pixel i (varying with Latitude) and 𝐴𝑡𝑜𝑡 = ∑ 𝐴𝑖𝑖∈[𝑓𝑢𝑙𝑙 𝑠𝑎𝑚𝑝𝑙𝑒] is the total surface of the sample.

The confusion matrix elements are used to generate the overall accuracy (OA), the omission error (OE) and the commission error (CE) as follow:

• OA = p11+p22 • OE = p21 / (p11+p21) • CE = p12 / (p11+p12)




3.3.2 Water detection QA

In the present case, the confusion matrix is a two-way matrix composed of the proportion of water and land pixels correspondence between the CGLS-WB product and the reference GWHR-WSR for pixels of the sample. Pixels of the GWHR-WSR dataset are classified as “water” or “land” according to a threshold WSRthres.

Table 6: Definition for the confusion matrix elements for Water detection QA.

GWHR Reference

Water Land

CGLS

data

Water 𝑝11 =∑𝑝𝑖𝑥 ∈ [𝐶𝐺𝐿𝑆 𝑤𝑎𝑡𝑒𝑟 ∩ 𝑊𝑆𝑅 > 𝑊𝑆𝑅𝑡ℎ𝑟𝑒𝑠]

𝑁𝑡𝑜𝑡 𝑝12 =

∑𝑝𝑖𝑥 ∈ [𝐶𝐺𝐿𝑆 𝑤𝑎𝑡𝑒𝑟 ∩ 𝑊𝑆𝑅 ≤ 𝑊𝑆𝑅𝑡ℎ𝑟𝑒𝑠]𝑁𝑡𝑜𝑡

Land 𝑝21 =∑𝑝𝑖𝑥 ∈ [𝐶𝐺𝐿𝑆 𝑙𝑎𝑛𝑑 ∩ 𝑊𝑆𝑅 > 𝑊𝑆𝑅𝑡ℎ𝑟𝑒𝑠]

𝑁𝑡𝑜𝑡 𝑝22 =

∑𝑝𝑖𝑥 ∈ [𝐶𝐺𝐿𝑆 𝑙𝑎𝑛𝑑 ∩ 𝑊𝑆𝑅 ≤ 𝑊𝑆𝑅𝑡ℎ𝑟𝑒𝑠]𝑁𝑡𝑜𝑡

Table 6 shows the definition for the confusion matrix. p11, p12, p21 and p22 may be computed for each dekad or for the entire time series, for a given biome or globally. WSRthres may be varied.

The four elements can also be defined as follow:

• P11 is the sum of pixels being classified as “water” in the CGLS-WB product and having a WSR greater than WSRthres, divided by the total number of pixels in the sample. This corresponds to the fraction of pixels in the sample being rightly classified as having detected water.

• P12 is the sum of pixels being classified as “water” in the CGLS-WB product and having a WSR lower or equal to WSRthres, divided by the total number of pixels in the sample. This corresponds to the fraction of pixels being wrongly classified as having detected water.

• P21 is the sum of pixels being classified as “land” in the CGLS-WB product and having a WSR greater than WSRthres, divided by the total number of pixels in the sample. This corresponds to the fraction of pixels in the sample being wrongly classified as dry land.

• P22 is the sum of pixels being classified as “land” in the CGLS-WB product and having a WSR lower or equal to WSRthres, divided by the total number of pixels in the sample. This corresponds to the fraction of pixels being rightly classified as dry land.

The confusion matrix elements are used to generate the overall accuracy (OA), the omission error (OE) and the commission error (CE) as follow:

• OA = p11+p22 • OE = p21 / (p11+p21) • CE = p12 / (p11+p12)




4 RESULTS 4.1 CGLS-WB COASTAL PIXELS

The PROBA-V CGLS-WB product covers a band of few kilometres offshore. These areas, along the coastline are identified as “water”’ (70) rather than sea (0) as seen on Figure 6.

This labelling can cause two difficulties:

• A major issue arises when a user wants to perform blind, automatic processing of the product because the coastal zone will be considered as inland water. In that case, an auxiliary data (Land/Sea mask) is mandatory to reject the coastal zone.

• Further, the visual inspection led to general feeling that the product is contaminated because of the overwhelming majority of costal pixels compared to inland water pixels.

Figure 6: CGLS-WB V2 product over Britain area (France). Water (70) is grey, Sea (0) is black and

Land (255) is white. Left panel is PROBA-V product, right panel is VGT product.

4.2 CHARACTERISATION OF THE SAMPLE

The CGLS sample for QA is made of 66 dekads for PROBA-V products and 555 dekads for VGT products. The sample corresponding to the PROBA-V dataset is made of 5 955 896 129 pixels while the VGT sample is made of 30 106 613 494 pixels. The total area corresponding to the pixel sample for VGT and PROBA-V times series is 24242.106 km2 and 4842.106 km2, respectively.




Figure 7 shows the pixel repartition (excluding dry land) in GWHR-WSR whatever their status in CGLS-WB product. The left panel corresponds to the VGT time series and the right panel to the PROBA-V time series. The number of pixels decreases as WSR increases for 0<WSR<1 and raise sharply for WSR ~ 1.0.

Figure 7: Pixel repartition (excluding dry land) in GWHR-WSR (whatever their status in CGLS-WB product). The left panel corresponds to VGT time series and the right panel corresponds to PROBA-V

time series.

The Figure 8 and Figure 9 show the temporal evolution of the area covered by pixels of the sample over the entire time series. We see a clear seasonal variability for the global area that is mostly related to the variation of covered area at higher latitude depending on the period of the year. Tropical savanna and tropical forest sample area are not affected by this seasonal variability while biomes extending to higher latitude like boreal forest and tundra or temperate forest are strongly affected. The area covered by the sample also varies from a year to another mostly because of a variability of the number of pixels having an “observed surface ratio” (GWHR-OSR) of 100%. This variation may be related to the Landsat data itself as seen with the sudden increase of surface corresponding to the Landsat 8 service start during 2013 in the left panel of Figure 8.




Figure 8: Temporal variability of the total surface covered by the pixel sample used for the validation. Left panel corresponds to the global surface for the entire time series for VGT dataset. Right panel is

a close up for the last year of VGT dataset with detail for each biome.

Figure 9: Temporal variability of the surface covered by the pixel sample used for the validation for

PROBA-V dataset.

Table 7 and Table 8 show the pixel number repartition in the sample regarding their status in the CGLS-WB product and GWHR-WSR. The status of CGLS pixels is either “land” or “water” while the WSR is separated in 14 classes, including WSR=0 for dry land. Most pixels (~95%) belong to




the dry land/dry land category (detected as dry land in both CGLS-WB and WSR/GWHR product) essentially because dry land represents the vast majority of inland pixels.

Table 7: Pixel number repartition for the whole sample corresponding to VGT

CGLS Status Land Water

GWHR WSR

0 28662041040 71468534 0.00<WSR≤0.10 735004771 18095918 0.10<WSR≤0.20 138815952 10097990 0.20<WSR≤0.30 64952029 8151365 0.30<WSR≤0.40 40223181 8027397 0.40<WSR≤0.50 25164928 7803250 0.50<WSR≤0.60 17775687 8543294 0.60<WSR≤0.70 11809378 8864791 0.70<WSR≤0.75 4685426 4970102 0.75<WSR≤0.80 3920999 5351637 0.80<WSR≤0.85 3065012 5409351 0.85<WSR≤0.90 2859701 6770342 0.90<WSR≤0.95 2567323 8803248 0.95<WSR≤1.00 7412947 213957901

Table 8: Pixel number repartition for the whole sample corresponding to PROBA-V

CGLS Status Land Water

GWHR WSR

0 5658742659 8810940 0.00<WSR≤0.10 161215323 2064630 0.10<WSR≤0.20 32265287 1162166 0.20<WSR≤0.30 15400611 1032393 0.30<WSR≤0.40 9637827 1184393 0.40<WSR≤0.50 5995057 1360702 0.50<WSR≤0.60 4070346 1749883 0.60<WSR≤0.70 2449665 2077454 0.70<WSR≤0.75 854118 1242377 0.75<WSR≤0.80 634742 1357056 0.80<WSR≤0.85 430452 1368532 0.85<WSR≤0.90 344491 1675407 0.90<WSR≤0.95 256584 2086420 0.95<WSR≤1.00 407585 36019029




In GWHR-WSR dataset, 4.8% (4.6%) of inland pixels has WSR>0 (i.e. contains water) over PROBA-V (VGT) time series.

Around 1.1% (1.3%) of inland pixels is detected as “water” in CGLS PROBA-V (VGT) sample. This corresponds to 63 191 382 WB pixels over the PROBA-V data sample and 386 315 120 WB pixels over the VGT sample (globally, over all dekads).

Figure 10: WSR repartition for pixels classified as water in the CGLS-WB product. Upper panel

corresponds to the VGT sample and lower panel corresponds to the PROBA-V sample.




Figure 10 shows the repartition of the WSR (including WSR=0 for dry land) of pixels detected as water in CGLS data for the whole (globally over all dekads) VGT (upper panel) and PROBA-V (lower panel) samples. Most of the WB pixels detected in CGLS (57% for PROBA-V, 55.4% for VGT) has a WSR>0.95.

4.3 WATER SURFACE QA

Table 9 shows the confusion matrix for the entire VGT dataset. From this confusion matrix, the overall accuracy is 99.37%, the omission error is 28.86% and the commission error is 29.61%.

Table 9: Confusion matrix for the entire VGT dataset. Values are unitless.

GWHR Reference

Water Land

CGLS

data

Water 0.007681 0.003231

Land 0.003116 0.985972

Table 10 shows the resulting confusion matrix for the entire PROBA-V dataset. From this confusion matrix, the overall accuracy is 99.45%, the omission error is 32.18% and the commission error is 25.60%.

Table 10: Confusion matrix for the entire PROBA-V dataset. Values are unitless.

GWHR Reference

Water Land

CGLS

data

Water 0.006659 0.002291

Land 0.003160 0.987890




Figure 11: Overall accuracy, commission error and omission error for entire VGT (left panel) and PROBA-V (right panel) dataset as detailed for each biome (plus Great Lakes). The dashed lines show the global results.

Figure 11 shows the overall accuracy, commission error and omission error for the whole PROBA-V and VGT dataset as detailed for each biome. The results vary significantly depending on the biome:

• Tropical forest has an OE greater than the global value with around 38% in VGT and PROBA-V sample while it has a CE lower than the global value with around 21% in both sample.

• Temperate forest has an OE greater than the global value with around 39% in both VGT and PROBA-V sample. It has a CE near global value for VGT (29%) sample and PROBA-V sample (24%).

• Boreal forest and tundra has the worst OE among biomes with 45% and 49% for VGT and PROBA-V sample, respectively. Its CE is rather different for VGT and PROBA-V with 49% and 33%, respectively (both greater than global value).

• Tropical savanna has an OE lower than the global value with 25% and 28% for VGT and PROBA-V sample, respectively. Its CE is better than the global value for VGT with 24% and near the global value for PROBA-V with 26%.

• Temperate savanna has both OE and CE lower than global values in both VGT and PROBA-V sample. Its OE is 22% in both VGT and PROBA-V sample. Its CE 21% in both sample.




• Flooded grassland and savanna has an OE lower than global values in both VGT and PROBA-V sample with 26%. It has around 35% for CE in both sample which is higher than global value.

• Mediterranean biome has an OE around 35% in both VGT and PROBA-V sample that is greater than the global value. It has a CE of 25% for VGT sample (lower than global value) and CE of 29% for PROBA-V (greater than global value).

• Deserts and xeric shrublands have an OE of 21% for both VGT and PROBA-V sample that is lower than global value. It has a CE of 41% and 44% for VGT and PROBA-V, respectively (greater than global value).

Not surprisingly, the CE and OE are low (~1%) for the great lakes areas. Except for the CE in the boreal forest and tundra, the tendencies in biome to biome CE and OE variability are rather similar between the VGT and PROBA-V samples.

4.3.1 Temporal variability

Figure 12 to Figure 15 show the CE and OE averaged over the full sample as a function of the biome and month of the year. Figure 12 and Figure 13 are restricted to the northern hemisphere while Figure 14 and Figure 15 to the Southern hemisphere. This separation allows one to disentangle seasonal effects in each hemisphere. A significant seasonal variability of CE and OE is seen for most biomes in both hemispheres.

In the northern hemisphere (Figure 12 and Figure 13):

• For the “Tropical forest”, OE and CE variability is moderate. We see a slight seasonality with CE being higher in May/June/July and OE being higher in March/April (especially in VGT sample, Figure 12) and October/November.

• For the “Boreal forest and tundra”, the seasonality is strong. We notice that no data are available for that biome from November to February because it is restricted to high latitude. CE rises during spring and autumn. The OE increases from March to peak in July, then decreases until October.

• For “temperate forest” and “temperate savanna”, errors evolve rather similarly. CE is strongly peaked in March/April. Small peaks appear in October and December (especially in temperate forest for VGT sample, Figure 12). The OE shows a smooth variability over the year with a maximum at summer time and minimum during the winter.

• For “tropical savanna”, the CE shows a seasonal variability with higher values in January/February and July/August/September and lower values in March/April/May and October/November/December. This tendency is enhanced in PROBA-V products (Figure 13). The OE variability is quite low. It is slightly higher during spring and autumn.

• For “flooded grasslands and savannas”, CE shows a minimum in July/August/September. It is maximum in January. In VGT products (Figure 12), we see a peak in April that is not




present in PROBA-V products (Figure 13). Although OE varies relatively less, it is interesting to notice that the variability is rather different between the two products. In particular, OE rises in November/December/January in PROBA-V product while the opposite occurs in VGT products in January.

• For “Mediterranean”, the CE is rather stable over the year except in October/November/December. In VGT products, it peaks in December (Figure 12). The OE varies smoothly over the year. It is maximum at summer time and minimum during the winter. We notice that, likewise for CE, OE shows a raise in December in VGT sample, not seen in PROBA-V products (Figure 13).

• For “deserts & xeric shrublands”, the variability of CE and OE is relatively limited regarding other biomes.

Figure 12: CE and OE for the Northern hemisphere averaged over the entire VGT sample as a

function of the biome and for each month over the year.




Figure 13: CE and OE for the Northern hemisphere averaged over the entire PROBA-V sample as a


In the southern hemisphere (Figure 14 and Figure 15):

• For “tropical forest”, the variability is rather limited. Only OE in PROBA-V products seems to show a little seasonality with lower values in April/May and November/December (Figure 15).

• For “temperate forest”, we see a rather similar seasonality for CE and OE. Both show a peak around May/June/July. CE is quite constant over the rest of the year. OE has a secondary peak in December/January.

• For “boreal forest and tundra”, no data are available in the southern hemisphere. • For “tropical savanna”, CE is greater around July/August and February/March. OE is rather

constant but shows a little peak in January and is slightly lower in June/July. • For the “temperate savanna”, the CE variability is quite different between the two samples.

In particular, the strong raise in August/September in VGT products (Figure 14) is very limited in the PROBA-V one (Figure 15). The OE varies smoothly with a minimum around July/August/September and a maximum around November/December/January.




• For “flooded grasslands and savannas”, CE is higher in February/March/April and in around August/September. OE varies smoothly with a minimum around May/June/July and maximum around November/December/January.

• For the “Mediterranean”, CE variability is quite different between the samples. We see a maximum in May/June/July in VGT products (Figure 14) while the same period corresponds to a minimum in the PROBA-V products (Figure 15). Conversely, we see a maximum of CE in March/April in PROBA-V products while it is minimum in the VGT products for the same period. OE varies smoothly over the year. It is minimum in May/June/July/August and maximum in November/December/January/February.

• For “Desert & xeric shrublands”, CE is minimum around June/July/August and maximum around November/December/January. Its variation is smoother in the VGT products. OE follows a rather similar tendency as CE in the VGT products. In PROBA-V sample, its variability is less pronounced. Moreover, it shows some differences regarding the VGT sample, like a decrease in December.

Figure 14: CE and OE for the Southern hemisphere averaged over the entire VGT sample as a function of the biome and for each month over the year




Figure 15: CE and OE for the Southern hemisphere averaged over the entire PROBA-V sample as a


In the Northern hemisphere, OE and CE variabilities are rather anti-correlated (or poorly correlated) while in the southern hemisphere, OE and CE variabilities are rather correlated for some biomes.

4.3.2 VGT/PROBA-V overlap period

PROBA-V and VGT dataset recover in time for 15 dekads for the period January 1st, 2014 to May 31th, 2014. For that period, the VGT OE and CE are 26.1% and 34.8%, respectively. The PROBA-V OE and CE are 25.3% and 31.2%, respectively. VGT shows slightly worse results but the two dataset errors are rather consistent.

Figure 16 shows the omission error temporal evolution for both VGT and PROBA-V for the overlapping period between the two samples. Figure 17 shows the commission error temporal evolution for both VGT and PROBA-V for the overlapping period between the two samples. We see that despite a different overall value, the temporal variability is quite similar for the two samples.




Figure 16: Omission error for both VGT and PROBA-V during the overlapping period for the two datasets. The dashed lines show the values corresponding to the entire overlapping time series.

Figure 17: Commission error for both VGT and PROBA-V during the overlapping period for the two datasets. The dashed lines show the values corresponding to the entire overlapping time series.




4.4 WATER DETECTION QA

Figure 18 shows the OE for both VGT and PROBA-V as a function of the WSRthres. Looking at the result for global scale, we see that in both VGT and PROBA-V products OE decreases as the WSRthres increases. Not surprisingly, the CGLS-WB products misses less pixels containing water as the fraction of water that covers the pixel is large. For WSRthres=0.1, OE is up to 52.1% and 58.2% for VGT and PROBA-V, respectively. It drops to 3.3% and 1.1% when WSRthres=0.95 for VGT and PROBA-V, respectively. All biomes have a common tendency regarding the OE variation as a function of WSRthres. On the other hand, we see that the ranking of biomes regarding the OE is similar to the one obtained when looking at water surface QA (see Figure 11). “Boreal forest and tundra” has the worst OE. “Tropical forest”, “temperate forest” and “Mediterranean” has an OE greater than the global value. However, we note that for high WSRthres OE of “tropical forest” and “Mediterranean” drops more than the one for “Boreal forest and tundra” and “temperate forest”. “Flooded grasslands and savannas”, “Tropical savanna”, “temperate savanna” and “deserts & xeric shrublands” have a better OE than global scale.

Figure 18: Omission error for water detection QA as a function of the WSRthres

Figure 19 shows CE for both VGT and PROBA-V as a function of the WSRthres. Looking at global scale, we see that for both products, the CE increases as the WSRthres increases. As we consider greater and greater WSRthres as being land in the reference data, we increase the CE of our CGLS-WB products rather artificially. For a WSRthres = 0.1, the CE is 23.2% and 17.2% for VGT and PROBA-V respectively. The biome repartition is coherent with the one seen for the water surface QA (see Figure 11).




Figure 19: Commission error for water detection QA as a function of the WSRthres




5 CONCLUSIONS

Omission and commission errors of CGLS-WB for VGT and PROBA-V are derived using the GWHR dataset (Pekel et al., 2016) as the reference. The GWHR-dataset is given globally at a resolution of 30m. Classification performance, measured using over 40,000 reference points in the GWHR dataset confirmed that their classifier produces less than 1% of false water detections, and misses less than 5% of water. We transform the GWHR dataset to a Water Surface Ratio for each pixel in CGLS product grid.

The validation is performed at global scale and over the period January 1st, 1999 to May 31th, 2014 for VGT (555 dekads) and January 1st 2014 to October 31st 2015 (66 dekads). The sample corresponding to the PROBA-V dataset is made of 5 955 896 129 pixels while the VGT sample is made of 30 106 613 494 pixels. The total area corresponding to the pixel sample for VGT and PROBA-V times series is 24242 106 km2 and 4842 106 km2, respectively.

The validation sample is made of most valid pixels in the CGLS-WB / GWHR-WSR data. No issue related to a random spatio-temporal sampling of a sub-set of pixels is to be considered here.

Users have to keep in mind the following limitations inherent to the used methodology:

(i) The computed omission and commission error are based upon the comparison to the high spatial resolution GWHR product, not regarding a ground truth.

(ii) The GWHR product is given monthly while the CGLS SPOT/VGT and PROBA-V products are provided every 10 days. Three dekads of CGLS products are compared to a single GWHR product each month.

(iii) The GWHR product does not fully cover the same latitude as the SPOT/VGT and PROBA-V products. It is more conservative in cutting zones potentially covered with snow during winter time. The northern limit at winter time goes down 30° (see Figure 5).

We provide a first characterisation of the CGLS-WB sample regarding the GWHR reference data in section 4.2. In particular, we give the repartition of WSR (into 14 classes including WSR=0 for dry land) for pixels (either land or water) of the CGLS-WB. As a first result, we show that for pixels classified as “water” in the CGLS-WB:

• 18.5% and 13.9% are land (WSR=0) in VGT and PROBA-V respectively • 13.5% and 10.8% have 0 < 𝑊𝑆𝑅 ≤ 0.5 for VGT and PROBA-V respectively • 68.0% and 75.3% have 0.5 < 𝑊𝑆𝑅 ≤ 1.0 for VGT and PROBA-V respectively




We separate between two different approaches for the derivation of confusion matrix and subsequent overall accuracy (OA), omission error (OE) and commission error (CE):

1. Water surface Quality Assessment (results in section 4.3): the CGLS-WB product is seen as a map of the area covered with water. Then, a pixel flagged as “water” in the CGLS-WB product is considered as fully covered with water. In that situation, we compare the land/water area coincidence between the CGLS-WB and the GWHR datase. For the global VGT sample, we obtain an overall accuracy of 99.37%, the omission error is 28.86% and the commission error is 29.61%. For the global PROBA-V sample, we obtain an overall accuracy of 99.45%, the omission error is 32.18% and the commission error is 25.60%. Regarding this approach, the PROBA-V and VGT products are not compliant with the GCOS requirements (Table 1) in terms of omission and commission errors. We also give results separately for 8 different major biomes as well as the temporal evolution of OE and CE.

2. Water detection Quality Assessment (results in section 4.4): the CGLS-WB is seen as a detection flag within each pixel. A pixel flagged as “water” in the product means water was detected in the pixel area. In that situation, the occurrence of land/water pixel is checked regarding the actual pixel content given a threshold WSR to classify the GWHR-WSR in two categories (land/water). For WSRthres=0.1, OE is up to 52.1% and 58.2% for VGT and PROBA-V, respectively. It drops to 3.3% and 1.1% when WSRthres=0.95 (i.e. WB dimension of the order of the pixel size) for VGT and PROBA-V, respectively. To be compliant with the GCOS requirements (Table 1), the WSRthres minimum values are given in Table 11. The CE is minimum for the smallest WSRthres. For WSRthres = 0.1, the CE is 23.2% and 17.2% for VGT and PROBA-V respectively. As such, the PROBA-V and VGT products are not compliant with the GCOS requirements (CE lower than 15%).

WSRthres

GCOS Maximum error PROBA-V VGT

Maps of land cover types Omission = 15% 0.526 0.545

Areas of GTN-L Omission = 5% 0.739 0.865

Table 11: Minimum WSRthres values for which PROBA-V and VGT products are compliant with GCOS requirements in terms of omission error.




6 REFERENCES Boschetti, L., Stehman, S. V. & Roy, D. P. A stratified random sampling design in space and time for regional to global scale burned area product validation Remote Sensing of Environment, 2016, 186, 465 – 478. GCOS-143 (2010). Guideline for the Generation of Datasets and Products Meeting GCOS Requirements, WMO, Geneva, Switzerland. GCOS-154 (2011). Systematic Observation Requirements for Satellite-based Products for Climate Supplemental details to the satellite-based component of the Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC - 2011 Update, WMO, Geneva, Switzerland. Gond, V., Bartholomé, E., Ouattara, F., Nonguierma, A., & Bado, L. (2004). Surveillance et cartographie des plans d’eau et des zones humides et inondables en régions arides avec l'instrument VEGETATION embarqué sur SPOT-4. International Journal of Remote Sensing. doi:10.1080/0143116031000139908. Olson, D.M., Dinerstein, E., Wikramanayake, E.D., Burgess, N.D., Powell, G.V., Underwood,E.C., D'Amico, J.A., Itoua, I., Strand, H.E., Morrison, J.C., Loucks, C.J., Allnutt, T.F.,Ricketts, T.H., Kura., Y., Lamoreux, J.F., Wettengel, W.W., Hedao, P., Kassem, K.R., 2001. Terrestrial ecoregions of the world: a new map of life on earth a new global map of terrestrial ecoregions provides an innovative tool for conserving biodiversity. Bioscience 51 (11), 933–938. Pekel, J.-F., Cottam, A., Gorelick, N, Belward, A., High resolution mapping of the global surface water and its long-term changes, Nature, 540, 418-422, 2016 Pekel, J.-F., Vancutsem, C., Bastin, L., Clerici, M., Vanbogaert, E. Bartholomé, E. & Defourny, P. (2014). A near-real time water surface detection method based on HSV transformation of MODIS multi-spectral times series data. Remote Sensing of Environment, 140, 704-716. Doi:10.1016/j.rse.2013.10.008.

Documents

Gio Global Land Component - Lot I ”Operation of the Global Land … · GIO-GL Lot 1, GMES Initial Operations Date Issued: 18.04.2017 Issue: l2.10 Gio Global Land Component - Lot