ERDDAP   Easier access to scientific data

Brought to you by NOAA NMFS SFSC ERD

ERDDAP > tabledap > Documentation

Using tabledap to Request Data And Graphs from Tabular Datasets

tabledap request URLs must be in the form
   http://coastwatch.pfeg.noaa.gov/erddap/tabledap/ datasetID fileType {?query}
For example,
   http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.htmlTable?longitude,latitude,time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2007-06-01T00:00:00Z

Details:

• Requests must not have any internal spaces.
• Requests are case sensitive.
• {} is notation to denote an optional part of the request.
   
• datasetID identifies the name that ERDDAP assigned to the source web site and dataset (for example, "pmelTao").
  You can see a list of datasetID options available via tabledap.
   
• fileType specifies the type of table data file that you want to download (for example, ".htmlTable").
  The actual extension of the resulting file may be slightly different than the fileType (for example, ".smallPdf" returns a small .pdf file).
  The fileType options for downloading tabular data are:
   

  Data fileTypes	Description	Info	Example
  .asc	View the data as OPeNDAP-style comma-separated ASCII text.	info	example
  .csv	Download the data as comma-separated ASCII text table (missing value = 'NaN'; times are ISO 8601 strings).	info	example
  .das	View the data's metadata via an OPeNDAP Dataset Attribute Structure (DAS).	info	example
  .dds	View the data's structure via an OPeNDAP Dataset Descriptor Structure (DDS).	info	example
  .dods	OPeNDAP clients use this to download the data in the DODS binary format.	info	example
  .geoJson	Download longitude,latitude,otherColumns data as a GeoJSON file.	info	example
  .graph	View a Make A Graph web page.	info	example
  .html	View an OPeNDAP-style HTML data access form.	info	example
  .htmlTable	View an HTML file with the data in a table (times are ISO 8601 strings).	info	example
  .json	Download the data as a JSON table (missing value = 'null'; times are ISO 8601 strings).	info	example
  .mat	Download a MATLAB binary file.	info	example
  .nc	Download a NetCDF binary file with COARDS/CF/THREDDS metadata.	info	example
  .ncHeader	View the header (the metadata) for the NetCDF file.	info	example
  .tsv	Download the data as tab-separated ASCII text table (missing value = 'NaN'; times are ISO 8601 strings).	info	example
  .xhtml	View an XHTML file with the data in a table (times are ISO 8601 strings).	info	example

  
  For example, here is a complete request to download data:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.htmlTable?longitude,latitude,time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2007-06-01T00:00:00Z

  MATLAB
  MATLAB users can download data from within MATLAB. Here is a one line example:

  load(urlwrite('http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.mat?longitude,latitude,time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2007-06-01T00:00:00Z', 'test.mat'));

  The data will be in a MATLAB structure. The structure's name will be the datasetID (for example, "pmelTao").
  The structure's internal variables will be column vectors with the same names as in ERDDAP (for example, use "fieldnames(pmelTao)" ).
  You can then make a scatterplot of any two columns. For example:

  plot(pmelTao.time, pmelTao.sea_surface_temperature)

  Making an Image with a Graph or Map of Tabular Data
  If a tabledap request URL specifies a subset of data which is suitable for making a graph or a map,
  and the fileType is an image fileType, tabledap will return an image with a graph or map.
  tabledap request URLs can include optional graphics commands which let you customize the graph or map.
  As with other tabledap request URLs, you can create these URLs by hand or have a computer program do it.
  Or, you can use the Make A Graph web pages, which simplify creating these URLs (see the "graph" links in the table of tabledap datasets).

  The fileType options for downloading images of graphs and maps of table data are:

  Image fileTypes	Description	Info	Example
  .kml	View a .kml file, suitable for Google Earth, with a reference to the data you selected.	info	example
  .smallPdf	View a small .pdf image file with a graph/map of the data you selected.	info	example
  .pdf	View a standard, medium-sized .pdf image file with a graph/map of the data you selected.	info	example
  .largePdf	View a large .pdf image file with a graph/map of the data you selected.	info	example
  .smallPng	View a small .png image file with a graph/map of the data you selected.	info	example
  .png	View a standard, medium-sized .png image file with a graph/map of the data you selected.	info	example
  .largePng	View a large .png image file with a graph/map of the data you selected.	info	example

  
   
• query is the part of the request after the "?". It specifies the subset of data that you want to receive.
  In tabledap, it is an OPeNDAP DAP constraint query in the form: {resultsVariables}{constraints} .
  For example, "longitude,latitude,time,sea_surface_temperature&longitude>264&latitude>0&time>=2007-08-01&time<=2007-08-08T00:00:00Z".
  • resultsVariables is an optional comma-separated list of variables (for example, "longitude,latitude,time,sea_surface_temperature").
    For each variable in resultsVariables, there will be a column in the results table, in the same order.
    If you don't specify any results variables, the results table will include columns for all of the variables in the dataset.
  • constraints is an optional &-separated list of constraints (for example, "&longitude>264&latitude>0&time>=2007-08-01&time<=2007-08-08T00:00:00Z").
    • The constraints determine which rows of data from the original table are included in the results table.
      The constraints are applied to each row of the original table.
      If all the constraints evaluate to true for a given row, that row is included in the results table.
      Thus, "&" can be roughly interpreted as "and".
    • If you don't specify any constraints, all rows from the original table will be included in the results table.
      For the fileTypes that don't actually return data (notably, .das, .dds, and .html) it is fine not to specify constraints. For example,
      http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.dds
      For the fileTypes that do return data, this may result in a very large results table.
    • tabledap constraints are consistent with OPeNDAP DAP constraint queries, but with a few additional features.
    • Each constraint is in the form <variable><operator><value> (for example, "latitude>45").
    • The valid operators are =, != (not equals), =~ (a regular expression test), <, <=, >, and >= .
    • tabledap extends the DAP standard to allow any operator to be used with any data type.
      (Standard DAP constraints don't allow <, <=, >, or >= to be used with string variables and don't allow =~ to be used with numeric variables.)
      For String data, the <, <=, >, and >= operators act in a case-insensitive manner.
      =~ tests if the value from the variable on the left matches the regular expression on the right.
      • The syntax of regular expressions varies somewhat in different implementations.
        tabledap uses the standard regular expression procedures in Java.
      • For numeric variables, the test is performed on the String representation of the variable's value.
    • tabledap extends the DAP standard to allow constraints to be applied to any variable in the dataset.
      The constraint variables don't have to be included in the resultsVariables.
      (Standard DAP implementations usually only allow constraints to be applied to certain variables. The limitations vary with different implementations.)
    • Although tabledap extends the DAP constraint standard (as noted above), these extensions sometimes aren't practical because tabledap may need to download lots of extra data from the source (which takes time) in order to test the constraint.
    • tabledap always stores date/time values as numeric values in seconds since 1970-01-01T00:00:00Z.
      Here is an example of a query which includes date/time numeric values:
      http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.htmlTable?longitude,latitude,time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>1180656000
      Some fileTypes (notably, .csv, .tsv, .htmlTable, and .xhtml) display date/time values as ISO 8601:2004 "extended" date/time strings (e.g., 2002-08-03T12:30:00Z).
    • tabledap extends the DAP standard to allow you to specify time values in the ISO 8601 date/time format (YYYY-MM-DDThh:mm:ssZ, where Z is 'Z' or a ±hh:mm offset from UTC).
      If you omit Z (or the ±hh:mm offset), :ssZ, :mm:ssZ, or Thh:mm:ssZ from the ISO date/time that you specify, the missing fields are assumed to be 0.
      Here is an example of a query which includes ISO date/time values:
      http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.htmlTable?longitude,latitude,time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2007-06-01T00:00:00Z .
  • Graphics Commands - tabledap extends the DAP standard by allowing graphics commands in the query.
    The Make A Graph web pages simplify the creation of URLs with these graphics commands (see the "graph" links in the table of tabledap datasets).
    So we recommend using the Make A Graph web pages to generate URLs, and then, when needed, using the information here to modify the URLs for special purposes.
    These commands are optional. If present, they must occur after the data request part of the query.
    These commands are used by tabledap if you request an image fileType (e.g., .png) and are ignored if you request a data file (e.g., .asc).
    If relevant commands are not included in your request, tabledap uses the defaults and tries its best to generate a reasonable graph or map.
    All of the commands are in the form &.commandName=value .
    If the value has sub-values, they are separated by the '|' character.
    In most cases, sub-values can be omitted, causing the default value to be used.
    In most cases, if sub-values are omitted from the end of the list, you can just truncate the list (for example, "a||c" can be used instead of "a||c||||").
    The commands are:
    • &.colorBar= specifies the settings for a color bar. The sub-values are:
      • palette - see a Make A Graph web page for options (e.g., "Rainbow").
        The default (no value) varies based on min and max: if -1*min ~= max, the default is "BlueWhiteRed";
        otherwise, the default is "Rainbow".
      • continuous - must be either no value (the default), 'C' (for Continuous), or 'D' (for Discrete). The default is different for different datasets
      • scale - must be either no value (the default), "Linear", or "Log". The default is different for different datasets.
      • min - The minimum value for the color bar. The default is different for different datasets.
      • max - The maximum value for the color bar. The default is different for different datasets.
      • nSections - The preferred number of sections (for Log color bars, this is a minimum value). The default is different for different datasets.
    • &.color= specifies the color for lines, markers, vectors, etc. The value must be specified as an 0xRRGGBB value (e.g., 0xFF0000 is red, 0x00FF00 is green). The default is 0x000000 (black).
    • &.draw= specifies how the data will be drawn, as "lines", "linesAndMarkers", "markers" (default), "sticks", or "vectors".
    • &.font= specifies a scale factor for the font (e.g., 1.5 would make the font 1.5 times as big as normal).
    • &.marker= specifies markerType|markerSize .
      markerType is an integer: 0=None, 1=Plus, 2=X, 3=Dot, 4=Square, 5=Filled Square (default), 6=Circle, 7=Filled Circle, 8=Up Triangle, 9=Filled Up Triangle.
      markerSize is an integer from 3 to 50 (default=5)
    • There is no &.vars= command. Instead, the results variables from the main part of the query are used.
      The meaning associated with each position varies with the &.draw value:
      • for lines: xAxis,yAxis
      • for linesAndMarkers: xAxis,yAxis,Color
      • for markers: xAxis,yAxis,Color
      • for sticks: xAxis,uComponent,vComponent
      • for vectors: xAxis,yAxis,uComponent,vComponent
      If xAxis=longitude and yAxis=latitude, you get a map; otherwise, you get a graph.
    • &.vec= specifies the data vector length (in data units) to be scaled to the size of the sample vector in the legend. The default varies based on the data.
    • &.xRange= specifies the min|max for the X axis. The default varies based on the data.
    • &.yRange= specifies the min|max for the Y axis. The default varies based on the data.
    
    A sample graph URL is
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.png?time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2008-04-01T00:00:00Z&.draw=linesAndMarkers&.marker=1|3&.color=0xFF9900&.colorBar=|C|Linear|0|32|

    Or, if you change the fileType in the URL from .png to .graph, you can see a Make A Graph web page with that request loaded:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.graph?time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2008-04-01T00:00:00Z&.draw=linesAndMarkers&.marker=1|3&.color=0xFF9900&.colorBar=|C|Linear|0|32|
    That makes it easy for humans to modify an image request to make a similar graph or map.

    Or, if you change the fileType in the URL from .png to a data fileType (e.g., .htmlTable), you can download the data that was graphed:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.htmlTable?time,sea_surface_temperature&longitude>180&longitude<180.2&latitude>0&latitude<0.2&time>2008-04-01T00:00:00Z&.draw=linesAndMarkers&.marker=1|3&.color=0xFF9900&.colorBar=|C|Linear|0|32|

    A sample map URL is
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.png?longitude,latitude,sea_surface_temperature,time&time>=2007-06-01T00:00:00Z&time<2007-06-02T00:00:00Z&.draw=markers&.marker=5|5&.color=0x000000&.colorBar=Rainbow|C|Linear|0|32|

    Or, if you change the fileType in the URL from .png to .graph, you can see a Make A Graph web page with that request loaded:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.graph?longitude,latitude,sea_surface_temperature,time&time>=2007-06-01T00:00:00Z&time<2007-06-02T00:00:00Z&.draw=markers&.marker=5|5&.color=0x000000&.colorBar=Rainbow|C|Linear|0|32|

    Or, if you change the fileType in the URL from .png to a data fileType (e.g., .htmlTable), you can download the data that was mapped:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTao.htmlTable?longitude,latitude,sea_surface_temperature,time&time>=2007-06-01T00:00:00Z&time<2007-06-02T00:00:00Z&.draw=markers&.marker=5|5&.color=0x000000&.colorBar=Rainbow|C|Linear|0|32|

• Data Model - Each tabledap dataset can be represented as a a table with one or more rows and one or more columns of data.
  • Each column is also known as a "data variable" (or just a "variable").
    Each data variable has data of one specific type.
    The supported types are (int8, uint16, int16, int32, int64, float32, float64, String of any length).
    Any cell in the table may have a no value (i.e., a missing value).
    Each data variable has a name composed of a letter (A-Z, a-z) and then 0 or more characters (A-Z, a-z, 0-9, _).
    Each data variable has a set of metadata expressed as Key=Value pairs.
  • Each dataset has a set of global metadata expressed as Key=Value pairs.
    Each metadata value is either one or more numeric values (of one type), or one or more Strings
    of any length (using \n as the separator).
• Special Variables
  • In tabledap, a longitude variable (if present) always has the name "longitude" and the units "degrees_east".
  • In tabledap, a latitude variable (if present) always has the name "latitude" and the units "degrees_north".
  • In tabledap, an altitude variable (if present) always has the name "altitude" and the units "m" above sea level. Locations below sea level have negative altitude values.
  • In tabledap, a time variable (if present) always has the name "time" and the units "seconds since 1970-01-01T00:00:00Z".
• Incompatibilities
  • Constraints - Different types of data sources support different types of constraints.
    Most data sources don't support all of the types of constraints that tabledap advertises.
    For example, most data sources can't test StringVariable<StringValue.
    When a data source doesn't support a given type of constraint, tabledap gets extra data from the source,
    then does that constraint test itself. Sometimes, getting the extra data takes a lot of time.
  • File Types - Some results file types have restrictions.
    For example, .kml is only appropriate for results with longitude and latitude values.
    If a given request is incompatible with the requested file type, tabledap throws an error.
• Requesting Compressed Files
  ERDDAP doesn't offer results stored in compressed (e.g., .zip or .gzip) files.
  Instead, ERDDAP looks for accept-encoding in the HTTP GET request header sent by the client.
  If a supported compression type ("gzip", "x-gzip", or "deflate") is found in the accept-encoding list,
  ERDDAP includes "content-encoding" in the HTTP response header and compresses the data as it transmits it.
  It is up to the client program to look for "content-encoding" and decompress the data.
  Browsers and OPeNDAP clients do this by default. They request compressed data and decompress the returned data automatically.
  Other clients (e.g., Java programs) have to do this explicitly.