REST API Overview
The REST API for CMR OPeNDAP aims to accomplish a few core goals:
- Provide a straight forward means of bridging the CMR and OPeNDAP deployments
- Provide operational insights into the CMR OPeNDAP REST service itself
- Do both of these as simply and in as scalable a manner as possible
The REST API is currently focused on generating OPeNDAP data access URLs, allowing users to easily download subsetted files from OPeNDAP servers based on metadata stored in the CMR and presented to end users by EDSC.
About the Docs
This API documentation provides prose in the central column, using the left collumn to highlight usage and code.
Note that, due to some of the long URLs and query examples, the lines in the examples are broken up -- this is done for the sake of visiability on the page. When you use these to make a query you must remove newline breaks.
Releases
The following table provides a mapping between CMR OPeNDAP releases and the API versions supported in each.
| Release | REST API Versions | Status |
|---|---|---|
| 1.1.0 | v1, v2, v2.1 | In development |
| 1.0.0 | v1, v2 | Released |
| 0.1.0 | v1 | Released |
Summary of changes in REST API:
- v2.1: Use of UMM-Var 1.2 new dimensional metadata to remove old hacks
- v2: Breaking change to cache management admin resource
- v1: First version of API (including admin resources)
Versioned API
The CMR OPeNDAP REST API is versioned. By default, the most recent version of
the API is accessed when a request is submitted without specificing the
desired version. Currently, this is v2.1.
To request a specific version of the REST API, you must send an additional header in your request to CMR OPeNDAP.
An
Acceptheader is required to request a specific version of the API:
"Accept: application/vnd.cmr-opendap.v2+json"
The following is also valid:
"Accept: application/vnd.cmr-opendap.v2"
All HTTP responses from the REST API also provide information on the version used when processing the request. This can be useful for clients debugging calls to the service.
Examine the headers with a call to the service:
curl -i ...
(request/get ...)
The response headers will include the version used by the service:
Cmr-Media-Type: cmr-opendap.v2; format=json
{...
:cmr-media-type "cmr-opendap.v2; format=json"
...}
Authorized Access
CMR OPeNDAP requires the use of tokens in order to provide access to potentially protected collections, services, variables, and/or granules.
A token needs to be included in each request, and this is done by sending a special header:
"Echo-Token: <YOUR_TOKEN>"
For example, if you have stored your token in a file:
"Echo-Token: `cat /path/to/tokens/sit`"
Both ECHO as well as URS/Earthdata Login tokens are supported.
Many of the examples in the documentation below show usage of the Echo token.
Responses
Success
CMR OPeNDAP responses have pareticular formats, inherited from CMR response data. A successful request will provide the results, the number of results and, the time taken to process the request.
Successful response body:
{
"hits": ...,
"took": ...,
"items": [...]
"warnings:" null
}
Errors
When CMR OPeNDAP returns errors, it does so in a consistent format. In
particular, the body of the HTTP response is a JSON string (or stream)
representing an object with one key, errors, an associated value being
an array of the errors encountered.
CMR OPeNDAP error format
{
"errors": [...]
}
Warnings
In either a success or error state, CMR OPeNDAP may also return a warning
key containing one or more messages. This is reserved for future use and
will be the primary mechanism used to notify clients of coming changes in the
API that will impact users of the default version.
Warnings sent in a successful response body:
{
"hits": ...,
"took": ...,
"warnings": [...],
"items": [...]
}
Warnings sent in an error response body:
{
"warnings": [...],
"errors": [...]
}
OUS Resources
This is the part of the REST API responsible for creating OPeNDAP-compatible query URLs (intended to be sent to a deployed OPeNDAP service). The name "OUS" (OPeNDAP URL Service) was used in the prototype for this service, and that name has carried through here.
Collection Resources
The CMR OPeNDAP REST API supports two different means of creating a subsetted OPeNDAP URL for granules and/or variables in a collection: one returns a standard JSON document; the other, more effectial means returns a JSON document via HTTP streaming.
The resources are as follows:
GET /opendap/ous/collection/:concept-idGET /opendap/ous/streaming-collection/:concept-id
Example API usage if your client supports streaming:
curl --silent \
-H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/streaming-collection/C1200187767-EDF_OPS" | \
jq .
Example API usage if your client doesn't support streaming:
curl --silent \
-H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS" | \
jq .
Both of which return the same result:
{
"hits": 2,
"took": 0.621,
"items": [
"https://acdisc.gesdisc.eosdis.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.006..."
]
}
Both take the same parameters; the second one takes better, more efficient advantage of system resources (micro-threads and OS threads). In particular, the second one avoids making blocking calls in the request-processing thread; as such, we recommend that this resource be preferred. The only requirement is that your HTTP client supports streaming.
Interaction of Parameters
In the sections below, the various parameters one can pass to the ous resource are described. Each parameter is discussed in isolation, simply addressing it in its own context. However, common usage will involve more than on parameter acting at the same time. As such, here are some points to note:
- Currently CMR OPeNDAP supports only queries with one collection; as such,
the collection query to CMR Search explicitly limits the results to one
with the
page_size=1parameter. - When multiple variable concept ids are passed, these are sent to CMR Search in a single query for efficiency (as opposed to sending one variable request at a time).
- Similarly, multiple granules are queried at once with a single query to CMR Search.
- In the granule query to CMR Search, and spatial and temporal parameters that were passed to CMR OPeNDAP will be used to limit the granule results from CMR Search to just those in which the user is interested.
CMR-Style Parameters
The example above will generate a URL for downloading all the granules and all the variables across the globe for the given collection. To limit these, there are various parameters supported:
bounding-boxexclude-granulesformatgranulestemporalvariables
These may be used alone or in any combination. Details for usage are provided below in separate sections.
Note that both hypens and underscores are supported in parameters that have them; you may use either, per your preference.
bounding-box
This provides clients with the ability to select the spatial data to include in the granule data extracted by the OPeNDAP query to an area of interest.
If not provided, the entire spatial extent of the granule will be used.
The following are examples of accepted bounding-box parameter usage,
note the acceptance of either dash or underscore in the parameter name:
bounding-box=-9.984375,56.109375,19.828125,67.640625bounding_box=-9.984375,56.109375,19.828125,67.640625
Use of
bounding-boxin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
variables=V1200241812-EDF_OPS,V1200241817-EDF_OPS&
bounding-box=-9.984375,56.109375,19.828125,67.640625"
Note that spatial subsetting on non-gridded data is not supported. If
bounding-box is provided against a collection whose granules
are not gridded, the spatial subsetting information will be dropped and one
or more warning messages will be returned in the payload.
exclude-granules
This allows clients to perform the inverse of a granule search. If the
value of this parameter is set to "true", then all granules except
the ones passed will be returned.
If not provided, a regular granule search is performed.
The following are examples of accepted exclude-granules parameter usage,
note the acceptance of either dash or underscore in the parameter name:
exclude-granules=true&granules=G1200187775-EDF_OPSexclude_granules=true&granules=G1200187775-EDF_OPS
Note that the granules parameter here may take any of the accepted forms
shown in the granules section below.
Use of
exclude-granulesin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
exclude-granules=true&
granules=G1200187775-EDF_OPS"
format
This allows the client to ask OPeNDAP to provide files in the given format.
If not provided, the default of nc (NetCDF) is used. Supported formats
depend upon the target OPeNDAP server.
The following is an example of accepted format parameter usage:
format=nc
Use of
formatin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?format=nc"
granules
This allows the client to select the granules to include (or exclude) in their OPeNDAP queries.
If not provided, all granules for the given collection will be used. If used
in conjunction with the exclude-granules parameter, then all granules
except the ones passed will be used in the OPeNDAP query.
The following are examples of accepted granules parameter usage:
granules=G1200187775-EDF_OPSgranules=G1200187775-EDF_OPS,G1200245955-EDF_OPSgranules=G1200187775-EDF_OPS&granules=G1200245955-EDF_OPSgranules[]=G1200187775-EDF_OPS&granules[]=G1200245955-EDF_OPS
Use of
granulesin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
granules=G1200187775-EDF_OPS,G1200245955-EDF_OPS"
temporal
This provides clients with the ability to select a temporal subset of data to include in the granule data extracted by the OPeNDAP query, only providing data for the time of interest.
If not provided, the entire temproal extent of the granule will be used.
The following are examples of accepted temporal parameter usage:
temporal=2002-09-01T00:00:00Z,2016-07-03T00:00:00Ztemporal[]=2002-09-01T00:00:00Z,2016-07-03T00:00:00Ztemporal=2000-01-01T00:00:00Z,2002-10-01T00:00:00Z&temporal=2010-07-01T00:00:00Z,2016-07-03T00:00:00Ztemporal[]=2000-01-01T00:00:00Z,2002-10-01T00:00:00Z&temporal[]=2010-07-01T00:00:00Z,2016-07-03T00:00:00Z
Use of
temporalin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
variables=V1200241812-EDF_OPS,V1200241817-EDF_OPS&
temporal=2002-09-01T00:00:00Z,2016-07-03T00:00:00Z"
variables
This proivdes clients with the ability to limit the granule data extracted by the OPeNDAP query to just the variables in which you are interested.
If not provided, all variables associated in the collection metadata will be used.
The following are examples of accepted variables parameter usage:
variables=V1200241812-EDF_OPSvariables=V1200241812-EDF_OPS,V1200241817-EDF_OPSvariables=V1200241812-EDF_OPS&variables=V1200241817-EDF_OPSvariables[]=V1200241812-EDF_OPS&variables[]=V1200241817-EDF_OPS
Use of
variablesin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
variables=V1200241812-EDF_OPS,V1200241817-EDF_OPS"
WCS-Style Parameters
The original implementation of the OPeNDAP URL translation service was not a part of the CMR, and thus did not take into consideration the internal standards adopted by both the CMR and EDSC code bases. Instead, it was decided to follow the model of WCS. As such, the following parameters are also available, as an alternative to the "classic" CMR/EDSC parameters:
coverageformatrangesubsetsubsettimeposition
These are show in more detail in the sub-sections below.
Note that WCS-style and CMR/EDSC-style parameters may not be mixed in the same request; you must chose one or the other, not both.
coverage
This allows the client to select the granules to include (or exclude) in their OPeNDAP queries.
If not provided, all granules for the given collection will be used.
The following are examples of accepted coverage parameter usage:
coverage=G1200187775-EDF_OPScoverage=G1200187775-EDF_OPS,G1200245955-EDF_OPS
Use of
coveragein a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
coverage=G1200187775-EDF_OPS,G1200245955-EDF_OPS"
format
This allows the client to ask OPeNDAP to provide files in the given format.
If not provided, the default of nc (NetCDF) is used. Supported formats
depend upon the target OPeNDAP server.
The following is an example of accepted format parameter usage:
format=nc
Use of
formatin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?format=nc"
rangesubset
This proivdes clients with the ability to limit the granule data extracted by the OPeNDAP query to just the variables in which you are interested.
If not provided, all variables associated in the collection metadata will be used.
The following are examples of accepted rangesubset parameter usage:
rangesubset=V1200241812-EDF_OPSrangesubset=V1200241812-EDF_OPS,V1200241817-EDF_OPS
Use of
rangesubsetin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
rangesubset=V1200241812-EDF_OPS,V1200241817-EDF_OPS"
subset
This provides clients with the ability to select the spatial data to include in the granule data extracted by the OPeNDAP query to an area of interest.
If not provided, the entire spatial extent of the granule will be used.
The following are examples of accepted subset parameter usage:
subset=lat(56.109375,67.640625)subset=lon(-9.984375,19.828125)
Note that both are needed in order to define a bounding box.
Use of
subsetin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
variables=V1200241812-EDF_OPS,V1200241817-EDF_OPS&
subset=lat(56.109375,67.640625)&subset=lon(-9.984375,19.828125)"
Note that spatial subsetting on non-gridded data is not supported. If
subset is provided against a collection whose granules
are not gridded, the spatial subsetting information will be dropped and one
or more warning messages will be returned in the payload.
timeposition
This provides clients with the ability to select a temporal subset of data to include in the granule data extracted by the OPeNDAP query, only providing data for the time of interest.
If not provided, the entire temproal extent of the granule will be used.
The following is an example of accepted timeposition parameter usage:
timeposition=2002-09-01T00:00:00Z,2016-07-03T00:00:00Ztimeposition=2000-01-01T00:00:00Z,2002-10-01T00:00:00Z&timeposition=2010-07-01T00:00:00Z,2016-07-03T00:00:00Z
Use of
timepositionin a query:
curl -H "Echo-Token: `cat ~/.cmr/tokens/sit`" \
"%%OPENDAP_BASE_URL%%ous/collection/C1200187767-EDF_OPS?
variables=V1200241812-EDF_OPS,V1200241817-EDF_OPS&
timeposition=2002-09-01T00:00:00Z,2016-07-03T00:00:00Z"
Forthcoming
The following are coming soon:
POST /opendap/ous/collection/:concept-idPOST /opendap/ous/collections
Admin Resources
Cache
Administrative users may use a token to manage the authorization cache:
GET /opendap/cache/authDELETE /opendap/cache/authGET /opendap/cache/auth/:item-keyDELETE /opendap/cache/auth/:item-key
Likewise, administrators have the ability to manage the concept cache:
GET /opendap/cache/conceptDELETE /opendap/cache/conceptGET /opendap/cache/concept/:item-keyDELETE /opendap/cache/concept/:item-key
Health
Even though the health resources are classified as "admin", they are
available for use by anyone.
GET /opendap/healthOPTIONS /opendap/health
Ping
Administrative users may use a token to access the following:
GET /opendap/pingPOST /opendap/pingOPTIONS /opendap/ping
Testing Resources
The following resources are provided for use in various tests:
GET /testing/401GET /testing/403GET /testing/404GET /testing/405GET /testing/500GET /testing/503
Errors Messages
Dependent Services
CMR OPeNDAP will not only report its own errors, but errors encountered when making calls to other services upon which it depends. Those errors are not listed below, but will be appended to the array of errors in the response body.
CMR OPeNDAP Warnings
Warning messages will generally be included in a successful reponse, and thus will usually have an HTTP status of 200.
Gridded Data
| HTTP Status | Warning Message |
|---|---|
| 200 | The variables __, __, ... are not gridded. |
| 200 | The bounding information for the query has been removed. |
CMR OPeNDAP Errors
Authorization
| HTTP Status | Error Message |
|---|---|
| 403 | You do not have permissions to access that resource. |
| 403 | An ECHO token is required to access this resource. |
URL Generation
Client-Side
| HTTP Status | Error Message |
|---|---|
| 400 | The requst includes a dataset whose processing level is not supported. |
| 400 | Problematic processing level ___ for collection ___. |
| 400 | The service pattern computed was empty. Is there a service associated with the given collection? Does the UMM-S record in question have values for the pattern fields? |
| 400 | One or more of the parameters provided were invalid. |
| 400 | The values provided for latitude are not within the valid range of -90 degrees through 90 degress. |
| 400 | The values provided for longitude are not within the valid range of -180 degrees through 180 degress. |
Server-Side
| HTTP Status | Error Message |
|---|---|
| 500 | There was a problem extracting a service data file from the granule. |
| 500 | There was a problem creating URLs from granule file data: couldn't match default service pattern %s to service ___. |
| 500 | Problematic granules: [...]. |
| 500 | There was a problem extracting a data URL from the granule's service data file. |
| 500 | There was a problem extracting granule metadata. |
| 500 | There was a problem extracting service metadata. |
| 500 | There was a problem extracting variable metadata. |
General
| HTTP Status | Error Message |
|---|---|
| 400-599 | HTTP Error status code: ___. |
| 400 | This capability is not currently implemented. |
| 400 | This capability is not currently supported. |