Skip to main content
All CollectionsWEkEO Data DownloadHDA REST API
How to use the Harmonized Data Access REST API with cURL?
How to use the Harmonized Data Access REST API with cURL?

Let's see how to use the WEkEO HDA REST API with cURL through a concrete example showing how to download data from a query.

Alexandre avatar
Written by Alexandre
Updated over 4 months ago

Context

The Harmonized Data Access (HDA) API allows uniform access to the whole WEkEO catalogue, including subsetting and downloading functionalities. To learn more on the HDA API, please check this article: What is the Harmonized Data Access (HDA) API?

The HDA API is REST-based and is published at this base URL:

https://gateway.prod.wekeo2.eu/hda-broker/.

📌Note: all endpoints defined in this article are relative to this URL.

We introduce here usage examples of this API in WEkEO's JupyterHub service (accessible from Sign in > Dashboard > JupyterHub).

How to use cURL?

cURL is a command line tool that you can use to transfer data via network protocols.

You can access cURL in several ways:

  • In WEkEO JupyterHub: open a terminal, by default cURL is installed on the base environment

  • In a local terminal: chances are you have cURL installed by default.

⚠️ Commands have only been tested in Linux.

  • Windows Command Prompt often requires the use of double quotes instead of single quotes. Single quotes may not be interpreted correctly.

  • Some special characters may need to be escaped with a backslash (\) in Windows Command Prompt.

Here is an example of the first command in Windows format:

curl -X POST -H "Content-Type: application/json" -d "{\"username\":\"USERNAME\",\"password\":\"PASSWORD\"}" https://gateway.prod.wekeo2.eu/hda-broker/gettoken

Step by step Guide

💡WEkEO Pro Tip: all commands from this article can be found in the Swagger, a Graphical User Interface (GUI) where you can see and test the REST API functions.

Step 1. Authentication

All calls to the HDA API require an access token, which can be obtained through a call to GET /gettoken. You'll need to provide your credentials, replacing USERNAME and PASSWORD with your own:

$ curl -X POST -H 'Content-Type: application/json' -d '{"username":"USERNAME","password":"PASSWORD"}' https://gateway.prod.wekeo2.eu/hda-broker/gettoken

The HDA API will respond with a token, valid for 1 hour:

{ "access_token": "xxxxxxxx-yyyy-zzzz-xxxx-yyyyyyyyyyyy", ... }

⚠️ All other calls to the HDA API must include the access token. Whenever you see <access_token> in a command, replace it with your token.

Step 2. Accepting the terms and conditions

Before data can be accessed, the Terms and Conditions must be accepted.

This needs to be done only once.

The list of available licenses is the 10 following items:

  • Copernicus_General_License

  • Copernicus_Sentinel_License

  • EUMETSAT_Core_Products_Licence

  • EUMETSAT_Copernicus_Data_Licence

  • Copernicus_DEM_Instance_COP-DEM-GLO-90-F_Global_90m

  • Copernicus_DEM_Instance_COP-DEM-GLO-30-F_Global_30m

  • Copernicus_ECMWF_License

  • Copernicus_Land_Monitoring_Service_Data_Policy

  • Copernicus_Marine_Service_Product_License

  • CNES_Open_2.0_ETALAB_Licence

Here is an example of the command to accept the terms and conditions of Copernicus_Land_Monitoring_Service_Data_Policy:

curl -X 'PUT' 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/termsaccepted/Copernicus_Land_Monitoring_Service_Data_Policy' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <access_token>'

Step 3. Subsetting datasets

Every dataset in the WEkEO catalogue can be subsetted through multiple attributes. You can obtain the full list of subsetting attributes for a particular dataset through a call to GET /querymetadata/{datasetId}.

For instance, to get the attributes for EEA's HPPV dataset on Vegetation Phenology and Productivity:

$ curl -X 'GET'  'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/queryable/EO%3AEEA%3ADAT%3ACLMS_HRVPP_VPP'  -H 'accept: application/json'  -H 'Authorization: Bearer <access_token>'

Old cURL command (click me)

$ curl -X GET --header 'Authorization: <access_token>' 'https://wekeo-broker.prod.wekeo2.eu/databroker/querymetadata/EO:EEA:DAT:CLMS_HRVPP_VPP'

💡WEkEO Pro Tip: if your datasetId contains ":" characters, replace them with the following characters: "%3A". In our case, the datasetId EO:EEA:DAT:CLMS_HRVPP_VPP becomes EO%3AEEA%3ADAT%3ACLMS_HRVPP_VPP.

Result of the command (click me)

{
  "type": "object",
  "title": "Querable",
  "properties": {
    "dataset_id": {
      "title": "dataset_id",
      "type": "string",
      "oneOf": [
        {
          "const": "EO:EEA:DAT:CLMS_HRVPP_VPP",
          "title": "EO:EEA:DAT:CLMS_HRVPP_VPP"
        }
      ]
    },
    "itemsPerPage": {
      "title": "Items PerPage",
      "type": "string",
      "pattern": "^[0-9]*$"
    },
    "startIndex": {
      "title": "Start Index",
      "type": "string",
      "pattern": "^[0-9]*$"
    },
    "uid": {
      "title": "Uid",
      "type": "string",
      "pattern": "[\\w-]+"
    },
    "productType": {
      "title": "Product Type",
      "type": "array",
      "uniqueItems": true,
      "items": {
        "oneOf": [
          {
            "const": "MINV",
            "title": "MINV"
          },
          {
            "const": "MAXD",
            "title": "MAXD"
          },
          {
            "const": "LENGTH",
            "title": "LENGTH"
          },
          {
            "const": "SOSD",
            "title": "SOSD"
          },
          {
            "const": "QFLAG",
            "title": "QFLAG"
          },
          {
            "const": "EOSV",
            "title": "EOSV"
          },
          {
            "const": "TPROD",
            "title": "TPROD"
          },
          {
            "const": "MAXV",
            "title": "MAXV"
          },
          {
            "const": "AMPL",
            "title": "AMPL"
          },
          {
            "const": "SOSV",
            "title": "SOSV"
          },
          {
            "const": "LSLOPE",
            "title": "LSLOPE"
          },
          {
            "const": "EOSD",
            "title": "EOSD"
          },
          {
            "const": "RSLOPE",
            "title": "RSLOPE"
          },
          {
            "const": "SPROD",
            "title": "SPROD"
          }
        ]
      }
    },
    "platformSerialIdentifier": {
      "title": "Platform SerialIdentifier",
      "type": "string",
      "oneOf": [
        {
          "const": "S2A, S2B",
          "title": "S2A, S2B"
        }
      ]
    },
    "tileId": {
      "title": "Tile Id",
      "type": "string",
      "pattern": "[\\w-]+"
    },
    "productVersion": {
      "title": "Product Version",
      "type": "string",
      "pattern": "[\\w-]+"
    },
    "productGroupId": {
      "title": "Product GroupId",
      "type": "string",
      "oneOf": [
        {
          "const": "S2A, S2B",
          "title": "S2A, S2B"
        }
      ]
    },
    "processingDate": {
      "title": "Processing Date",
      "type": "string",
      "format": "date-time",
      "default": ""
    },
    "start": {
      "title": "Start Date",
      "type": "string",
      "format": "date-time",
      "default": ""
    },
    "end": {
      "title": "End Date",
      "type": "string",
      "format": "date-time",
      "default": ""
    },
    "bbox": {
      "title": "Bbox",
      "type": "array",
      "minItems": 4,
      "maxItems": 4,
      "items": [
        {
          "type": "number",
          "maximum": -180,
          "minimum": 180
        },
        {
          "type": "number",
          "maximum": -90,
          "minimum": 90
        },
        {
          "type": "number",
          "maximum": -180,
          "minimum": 180
        },
        {
          "type": "number",
          "maximum": -90,
          "minimum": 90
        }
      ]
    }
  },
  "required": [
    "dataset_id"
  ]
}

The HDA API responds with definitions for multiple variables you can use to subset (e.g. productType). Here's a query based on these variables, which we'll use next:

{
  "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_VPP",
  "productType": [
    "TPROD"
  ],
  "start": "2018-01-01T00:00:00.000Z",
  "end": "2019-01-01T00:00:00.000Z",
  "bbox": [
    -0.58712594,
    45.91075967748305,
    3.25581995,
    48.65066374786067
  ],
  "itemsPerPage": 1,
  "startIndex": 0
}

📌Note: an other way to create an API request is via the WEkEO Data Viewer. You can follow the article How to download WEkEO data? to learn how.

Step 4. Requesting data

Now let's create a command to retrieve the results linked to the query created above.

To do this we use POST /dataaccess/search:

$ curl -X 'POST' \
  'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/search' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <access_token>' \
  -H 'Content-Type: application/json' \
  -d '{
  "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_VPP",
  "productType": "TPROD",
  "start": "2018-01-01T00:00:00.000Z",
  "end": "2019-01-01T00:00:00.000Z",
  "bbox": [
    -0.58712594,
    45.91075968,
    3.25581995,
    48.65066375
  ],
  "itemsPerPage": 1,
  "startIndex": 0
}'

Old cURL command (click me)

$ curl -X POST \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: <access_token>' \
-d '{
    "datasetId": "EO:EEA:DAT:CLMS_HRVPP_VPP",
    "boundingBoxValues": [
      { "name": "bbox", 
        "bbox": [ -0.5871259395913329, 45.91075967748305, 3.255819949392474, 48.65066374786067 ] }
    ],
    "dateRangeSelectValues": [
      {
        "name": "temporal_interval", 
        "start": "2018-01-01T00:00:00.000Z", 
        "end": "2019-01-01T00:00:00.000Z" 
      }
    ],
    "stringChoiceValues": [
      { "name": "productType", "value": "TPROD"}, 
      {"name": "productGroupId", "value": "s1"} 
    ]
  }' 'https://wekeo-broker.prod.wekeo2.eu/databroker/datarequest'

Use the jobId to poll the GET /datarequest/status/{jobId} endpoint:

$ curl -X GET --header 'Authorization: <access_token>' 'https://wekeo-broker.prod.wekeo2.eu/databroker/datarequest/status/FQ6_njWjE9RD7GCeYXyy6ejJ-Kc'

At this point you can retrieve the list of results calling the GET /datarequest/jobs/{jobId}/result endpoint:

$ curl -X GET --header 'Authorization: <access_token>' 'https://wekeo-broker.prod.wekeo2.eu/databroker/datarequest/jobs/FQ6_njWjE9RD7GCeYXyy6ejJ-Kc/result'

This returns an initial result with numerous information about the downloadable item:

{
"properties":{"itemsPerPage":1,"startIndex":0,"totalResults":92},
"features":[{"type":"Feature",
"geometry":{"type":"Polygon",
"coordinates":[[[-1.707468,46.0462655],[-1.7299243,45.0581973],[-0.3364958,45.0341901],[-0.2894559,46.0214217],[-1.707468,46.0462655]]]},
"properties":{
"startdate":"2018-01-01T00:00:00Z",
"enddate":"2018-12-31T23:59:59Z",
"location":"s3://hr-vpp-products-vpp-v01-2018/CLMS/Pan-European/Biophysical/VPP/v01/2018/s1/VPP_2018_S2_T30TXR-010m_V101_s1_TPROD.tif",
"size":119048027,
"dataprovider":"EEA"},
"id":"VPP_2018_S2_T30TXR-010m_V101_s1_TPROD",
"bbox":[-1.7299243,45.0341901,-0.2894559,46.0462655]}]
}

You'll notice that only one of 92 items is displayed, as shown in the first line. To display more, or display a different one, you can modify the last items in the query using the itemsPerPage and startIndex parameters.

🔍 For example, if you want to display 5 items per page and you want to display the 2nd page, then you can set: "itemsPerPage": 5 and "startIndex": 1.

The parameters required for downloading are as follows:

  • dataset_id

  • location (can be in a totally different format than this one)

  • id

Step 5. Get the download ID

To retrieve the download ID for the item returned above, we create a request POST dataaccess/download in which we fill in the fields with the previous mandatory parameters:

$ curl -X 'POST' \
  'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <access_token>' \
  -H 'Content-Type: application/json' \
  -d '{
 "cacheable": true,
 "searchMetadata": "string",
 "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_VPP",
 "product_id": "VPP_2018_S2_T30TXR-010m_V101_s1_TPROD",
 "location": "s3://hr-vpp-products-vpp-v01-2018/CLMS/Pan-European/Biophysical/VPP/v01/2018/s1/VPP_2018_S2_T30TXR-010m_V101_s1_TPROD.tif" 
}'

Old cURL command (click me)

$ curl -X POST \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: <access_token>' \
-d '{
"jobId": "FQ6_njWjE9RD7GCeYXyy6ejJ-Kc",
"uri": "hr-vpp-products-vpp-v01-2018/CLMS/Pan-European/Biophysical/VPP/v01/2018/s1/VPP_2018_S2_T30TXR-010m_V101_s1_TPROD.tif"
}' 'https://wekeo-broker.prod.wekeo2.eu/databroker/dataorder'

You get the following download ID as the download_id parameter:

{"Location":"http://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/6647672a2a44a9b2c0e4e5be",
"download_id":"6647672a2a44a9b2c0e4e5be"}

Step 6. Download data

Finally, we make the download request using the download_id in the request GET /dataacces/download/{download_id}:

$ curl -X 'GET' \
  'http://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/6647672a2a44a9b2c0e4e5be' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <access_token>' \
  --output "VPP_2018_S2_T30TXR-010m_V101_s1_TPROD.tif"

Old cURL command (click me)

$ curl -X GET --header 'Authorization: <access_token>' 'https://wekeo-broker.prod.wekeo2.eu/databroker/dataorder/status/0fW0ZecwsR-kKxRd13vsnjJpXOQ'

The download starts, and the downloaded files appear in your working directory:

  % Total  % Received % Xferd  Average Speed  Time   Time   Time  Current
              Dload Upload   Total            Spent         Left   Speed
100  113M  0  113M    0   0  15.0M    0 --:--:--  0:00:07 --:--:-- 20.0M

💡WEkEO Pro Tip: by default, the file name is the download_id, but you can use --output as we did in the command above to specify another name. For information, the original file name can be found at the end of the location parameter.

After a short while, your files should have been downloaded to your working directory! 😃

⚠️ There is a limitation of Request and Orders. More details in this article.

What's next?

These articles might be of interest for you:

We are user-driven and we implement users' suggestions, so feel free to contact us:

  • through a chat session available in the bottom right corner of the page

  • via our contact webpage

  • via e-mail to our support team (supportATwekeo.eu)

Regardless of how you choose to contact us, you will first be put in touch with our AI Agent Neo. At any time, you can reach a member of the WEkEO User Support team by clicking on "talk to a person" via chat, or by naturally requesting it in reply to Neo's email.

Related Articles
HDA API errors
How to use the HDA API in Python?
How to Use the hdar Package for Accessing the WEkEO HDA API in R?
Improvements brought by the new HDA
What is the Harmonized Data Access (HDA) API?
Did this answer your question?