Go to the previous, next section, or Table of Contents.

EPS File Componenets

The EPIC system library is primarily intended for reading and writing geophysical data, and assumes this data is represented by three spatial axes and one time axis. Each data file has dimensions, or axes, of longitude, latitude, depth and time. One or more of these axes may be collapsed to a single point, but four axes are always present. For example, data files containing oceanographic observational data, such as time series data or CTD profile data, may have latitude and longitude axes each consisting of a single point.

All EPIC data files are self-describing. The EPIC System library reads data and information about the data from the data file, independently of the format of the actual data file on the disk, and returns this information to the calling routine as variables (the data itself), axes (location of the data in space and time), and attributes (information about the data, otherwise known as "header data" or metadata). Standard sets of attributes are supported, as described in the following sections.

The EPIC System is a pointer file system. A pointer file is a list of data file names, with or without path or directory names. We define an EPS file to be a collection of data files and a pointer file which contains the names of these data files. The data files within one pointer file should have the same data file format.

The EPIC System library supports multiple data file formats. At present, supported formats include the Unidata netCDF (Network Common Data Format) format and the Classic EPIC format. Read only support is also provided for generic ASCII file formats and ASCII file formats used internally at PMEL for processing PROTEUS mooring data. The Classic EPIC data files are binary sequential files designed for observational data with three of the four space/time axes represented by a single point. NetCDF data files are multi-dimensional, and are transportable between many different types of hardware platforms, including VAX/VMS, Unix workstations, Cray supercomputers, PCs and MACs. Additional data file formats, such as the HDF format (developed by the National Center for Supercomputing Applications) may be supported in the future. EPS routines can distinguish the data file format when the file is opened for read. For write, the user specifies the output data file format with the open statement. Variables (data), axes, and attributes (metadata) are read and written for each supported data format with the same EPS subroutine calls, so that application programs utilizing EPS library for input/output can be independent of the actual format of the data file on the disk.

Data Files

Classic EPIC File

Classic EPIC data files are binary sequential files which are restricted to data sets in which three of the four dimensions is a single point. For example, CTD data has a depth axis consisting of many points, but the latitude, longitude and time axes are each a single point. Time series data often has a time axis consisting of many points, but the latitude, longitude and depth axes are each a single point. Classic EPIC data files are binary sequential data files with at least one header consisting of eight 80-character lines containing information about the data, followed sequentially by data scans, one per logical record (no packing). The data scans can include any number of real (32-bit float points) variables up to 20. The EPS library returns attribute and variable information to the calling routine independently of the exact format of the data file, which is described in the EPIC manual, EPIC, A system for Management, Display and Analysis of Oceanographic Time Series and Hydrographic Data by Nancy Soreide, NOAA/PMEL. For the Classic EPIC data format, the EPS library returns the "header" information as variable and global attributes, and also as 80-character IMAGE global attributes (see the section in Attributes in this manual). The EPS library will read Classic EPIC files written with VAX/VMS from either VAX/VMS or from Unix, in support of the migration of the EPIC System from VAX/VMS to Unix.

NetCDF File

The netCDF data file format can accommodate full four dimensional data sets, without the restrictions of the Classic EPIC data format. The netCDF file is a binary file containing scientific data in a form that is network-transparent, i.e., it can be accessed by computers with different ways of storing integer, characters and floating-point numbers. The NetCDF User's Guide by The University Corporation for Atmospheric Research (UCAR) describes details about the netCDF format.

For the netCDF format, The EPIC system library is layered on top of the netCDF input/output library, in order to simplify production of a standardized implementation of netCDF, with a focus on oceanographic or geophysical data. It is not intended to provide the complete functionality which is available with the netCDF library. The EPS interface writes netCDF data files with following restrictions:

• Variable dimensions will be 4.
• Variable data type will be 64-bit float-points.
• Axis data type must be either long integer (32-bit integer) or float (32-bit float points).

Refer to Appendix D, EPS Implementation of Unidata NetCDF File Format for details about EPS interface access to netCDF files.

ASCII data files (read only)

Read only support is provided for following ASCII file formats: (1) generic formatted ASCII data files, and (2) format used internally at PMEL for processing PROTEUS mooring data.

Refer to Appendix E, EPS Implementation of ASCII Data File Format for details about EPS interface ascces to ASCII data files.

Variables

An EPS file variable has a variable code, a name, a short name, a long name, a generic name, a format, a unit and a type, all of which must be specified when the variable is defined. A variable has also a shape and value.

A variable code is an integer number which is used to refer to a variable. Some predefined variable codes (positive numbers), together with their short names, long names, generic names, formats and units are provided in a key file called epic.key . (See Appendix C, Epic.key File for details). For Classic EPIC data files, all the variables must have their variable codes predefined in the epic.key file. For netCDF data files, a variable can have its variable code predefined in epic.key file or user-defined with a negative number. When a variable code is predefined, the short name, long name, generic name, format and units of that variable are retrived from epic.key file, otherwise these information are requested from user.

The valid EPS variable data types are EPBYTE (8-bit data), EPSHORT (16-bit integer), EPINT (32-bit integer) and EPREAL (32-bit float-point), which are defined in "epsystem.h" (or "epsystem.fh" for Fortran).

For a variable in Classic EPIC file or a variable whose variable code is predefined in epic.key file, the data type is always assumed to be EPREAL. If missing values of a EPREAL typed data are to be flagged, the value of 1.e35 is recommended to be used for compatibility with EPIC system application programs.

The shape of a variable is described by its axes (dimensions), as described in the next section. The number of variable dimensions must be 4 or less. When reading a netCDF file, once the data file is loaded into EPS internal database, all variables will be converted into 4-D variables with dimension order (x,y,z,t). Dummy axes are inserted if the number of dimension is less than 4.

Axes

The EPS axes are used to specify the shapes and sizes of variables; they must be defined whenever the variable is defined. An EPS axis has an axis code, name, format, unit, type and the variable code of which the axis associates with. Axis has also a length and value.

The axis code is just like a variable code. For a classic EPIC file, it must be predefined in the epic.keyfile since the axis information (its name, format, and units) are obtained by EPS routines from the epic.key file, based on the axis code. For a netCDF file, however, the axis code will be ignored by EPS routines since the axis information are expected to be specified by the user, regardless of axis code used. Therefore, we recommend user to following the PMEL-EPIC convention when defining an axis. For a netCDF file, refer to Spatial Axis Name, Units, and EPIC-code Lookup Table in Appendix D, EPS Implementation of Unidata NetCDF File Format for setting an axis.

The EPS axis types are described by the axis data type (integer or real number) and the axis spacing (even or uneven). The valid axis types are EPEVEN and EPUNEVEN for evenly and unevenly spaced default axis data type; and EPEVEN_INT, EPUNEVEN_INT, EPEVEN_ REAL and EPUNEVEN_REAL for evenly and unevenly spaced integer and real axis data type, respectively. These are defined in "epsystem.h" (or "epsystem.fh" for Fortran).

Since the EPIC system library is intended for geophysical data, it assumes these data are represented by axes of longitude, latitude, depth and time. Although we provide the ability to read netCDF files with fewer than four axes, we do not recommend this practice. Our underlying philosophy is that all geophysical data is located by longitude, latitude, depth, and time; if this information is not included as an axis, it must be included elsewhere within the data file if the file is to be self-describing. The recommended EPS forms for each of the axis types are described in the following sections.

Longitude Axes

Many longitude representations are possible; however, for compatibility with the netCDF calculator function of PPLUS V1.2c (and earlier versions), use of the the West longitude convention is required. The West longitude convention means that values of the longitude axis in the netCDF file are positive for West longitudes. In future releases of the netCDF calculator function of PPLUS and also of the EPS library, both East and West longitude conventions will be supported. The recommended West longitude convention is described in the following paragraphs.

In this document, a West longitude convention means that Western longitudes are positive numbers, for example, 170W is +170.0, and that Eastern longitudes can have either of two conventions:

1. If the data being represented would be best described as continuous across the dateline, then East longitudes are written as (360.-long). EXAMPLE: 170E is written as 190.0.

2. If the data being represented would be best described as continuous across the Greenwich meridian, then East longitudes are written as (-long). EXAMPLE: 170E is written as -170.

Two proceeding representations of East longitudes are both supported by the EPIC system, which uses the PPLUS graphics package. In case 1, it is easy to make a PPLUS plot of the Pacific ocean, continuous across the dateline, from 137E to 95W, with the PPLUS command: XAXIS,223,95,-30. In case 2, it is easy to make a PPLUS plot, continuous across the Greenland Sea, from 15W to 15E, with the PPLUS command: XAXIS, 15, -15, -5.

The units of the longitude axis in the EPIC system library routine call should be selected either from the Unidata netCDF's udunits.dat file, or the EPIC system key file, which is given in the Appendix of this manual. For a netCDF file, the default spatial axis data type is real number. The recommended units are "degree_east", with epic key code 502.

Latitude Axes

The recommended latitude convention is for North latitudes to be represented by positive numbers (e.g., 10N is +10.0), and South latitudes by negative numbers (e.g., 10S is -10.0).

The units of the latitude axis in the EPIC system library routine call should be selected from the Unidata netCDF's udunits.dat file, or the EPIC system key file, which is given in the Appendix of this manual. For a netCDF file, the default spatial axis data type is real number. The recommended units are "degree_north", with epic key code 500.

Depth Axes

We recommend that the depth axis be given with the oceanographic convention of depth as a positive number, increasing downwards from the surface of the water towards the bottom of the ocean. For CTD data in Classic EPIC file, the z-axis (depth) data type must be a real number. For a netCDF file, the default spatial axis data type is real number, and recommended units are Pressure (dbar), with epic key code 1, or Depth (m), with epic key code 3.

At present, only surface atmospheric data is being written to EPIC style netCDF data files. If upper air data is written in future, this axis could be defined as "Height" and any convenient convention could be used to represent it. Different conventions could be used, with EPIC variable codes defined to identify them exactly.

Time Axes

The EPIC system library routines return the time axis from a data file to the calling routine in the form of a two-integer array, in which the first integer is the "true Julian day" with units of days, and the second integer is the number of milliseconds since the start of the day. This two-integer array allows representation of long time periods with high precision. There is a complete set of EPS routines for manipulation, calculation, and character string representation of this standard representation of time. For netCDF data files, the representation of the time axis within the data file can be of several types, for both read and write, including the standard specified by the Unidata style guide, the two-integer array which is used internally by the EPS routines, and some other time representations which are described in Chapter 5, EPS Time Manipulation Routines in this manual. Time axes in a netCDF file can be written or read in either real or integer format.

For time series data in a Classic EPIC file, the time axis written to the data file has a fixed, two-real-number format where the first is hhmm and the second is yymmdd. When writing a Classic EPIC time series file, it is sufficient to specify EPEVEN or EPUNEVEN to specify the axis type. For netCDF files, the default time axis type is the two-integer format.

Note that, regardless of the format of the time axis in the netCDF data file itself, the values used internally by the EPIC system library will be the double integer array.

Attributes

The EPS attributes contain information about an EPS data file (global attributes), variables (variable attributes) and axes (axis attributes) in that data file.

An EPS attribute has a name, data type, length of data array, and value of the attribute. With the EPS library, a global attribute is accessed via the attribute name; a variable attribute is accessed via the attribute name and a variable code, which indicates which variable the attribute applies to; and an axis attribute is accessed via the attribute name and the axis name. The valid attribute types are EPCHAR (string data), EPBYTE (8 bit integer), EPSHORT (16 bit integer), EPINT (32 bit integer), EPREAL (32 bit floating-point) and EPDOUBLE (64 bit floating-point for netCDF file only).

User should refer to the tables in this section for defination of an attribute. Some attribute names are predefined (a Parameter Name for a `` Attribute Name'') in the include files ``epsystem.h'' for C (``epsystem.fh'' for Fortran). User must use `` Attribute Name'' or Parameter Name listed in the tables when writing or reading these attributes. For a classic EPIC file, only predefined variable codes and axis codes from the epic.key file can be used, that is, attributes in classic EPIC data files are restricted to those listed in the attribute tables.

In the following attribute tables, the data Type ``c'' stands for EPCHAR, ``i'' stands for EPINT and ``r'' stands for EPREAL. The Length is the maximum attribute length in bytes used in the file. For classic EPIC, the attribute length in bytes should not exceed those described in the following tables. However, for netCDF file. the attribute length is not limited.

The Table 1 lists the attributes that will be created automatically by the EPS routines based on the information provided by the user or by default. The global attribute ``CREATION_DATE'' is the file creation data and time. When writing, it will be written out to a data file at closing time. The values of variable attributes ``name'', ``long_name'', ``generic_name'', ``FORTRAN_format'', ``units'' and ``epic_code'' are set when function ep_set_var (or epsetvar for Fortran) is called. If the variable code is defined in epic.key file, the values of above variable attributes will be obtained from the epic.key file; otherwise, these values are expected from user's input and ``epic_code'' will not be written out to the data file. Same as the variable attributes, the axis attributes listed in Table 1 are set when function ep_set_axis (or epsetaxis for Fortran) is called. However, regardless whether the axis code is defined in epic.key file or not, these axis attributes values are expected from user's input. In general, we recommend that users use units which are Unidata UDUNITS compatible for all data files.

Table 1. Attributes Automatically Written to the File

Table 2. Attributes Required for Use with EPIC Programs.

Table 3. Program Comment Attributes

Table 4. Profile (CTD) Attributes

Table 5.Time Series Attributes

Table 6. Composite Piece Attributes (optional for Time Series)

Table 7. Classic EPIC File Format Card Image Attributes

Go to the previous, next section, or Table of Contents.