root/cf-conventions/trunk/docbooksrc/introduction.xml

<preface>
    <title>Preface</title>
    <para>
        <variablelist>
            <varlistentry>
                <term>Home page:</term>
                <listitem>
                    <para>
                        Contains links to: previous draft and current
                        working draft documents; applications for processing
                        CF conforming files; email list for discussion
                        about interpretation, clarification, and proposals
                        for changes or extensions to the current conventions.
                        <ulink url="http://www-pcmdi.llnl.gov/cf/">http://www-pcmdi.llnl.gov/cf/</ulink>
                    </para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term>Revision history:</term>
                <listitem>
                    <para>
                        This document will updated as required to correct mistakes
                        or add new material required for completeness or clarity.
                        See <xref linkend="revhistory"/> for the full revision history.
                        Changes in the document use the following
                        mark-up style: <emphasis role="newtext">new text</emphasis>, 
                        <emphasis role="deletedtext">deleted text</emphasis>, and 
                        <emphasis role="commenttext">[a comment]</emphasis>.
                    </para>
                </listitem>
            </varlistentry>
        </variablelist>
    </para>
</preface>


<chapter>
        <title>
                Introduction
        </title>

        <section>
                <title>Goals</title>
                <para>
                        The NetCDF library <biblioref linkend="netcdf"/> is 
            designed to read
                        and write data that has been structured according
                        to well-defined rules and is easily ported across
                        various computer platforms. The netCDF interface
                        enables but does not require the creation of
                        <emphasis>self-describing</emphasis> datasets. 
            The purpose of the CF
                        conventions is to require conforming datasets
                        to contain sufficient metadata that they are
                        self-describing in the sense that each variable
                        in the file has an associated description of
                        what it represents, including physical units if
                        appropriate, and that each value can be located
                        in space (relative to earth-based coordinates)
                        and time.
                </para>
                <para>
                        An important benefit of a convention is that it
                        enables software tools to display data and perform
                        operations on specified subsets of the data
                        with minimal user intervention. It is possible
                        to provide the metadata describing how a field
                        is located in time and space in many different
                        ways that a human would immediately recognize as
                        equivalent. The purpose in restricting how the
                        metadata is represented is to make it practical
                        to write software that allows a machine to parse
                        that metadata and to automatically associate each
                        data value with its location in time and space. It
                        is equally important that the metadata be easy
                        for human users to write and to understand.
                </para>
                <para>
                        This standard is intended for use with climate
                        and forecast data, for atmosphere, surface and
                        ocean, and was designed with model-generated data
                        particularly in mind. We recognise that there are
                        limits to what a standard can practically cover;
                        we restrict ourselves to issues that we believe to
                        be of common and frequent concern in the design of
                        climate and forecast metadata. Our main purpose
                        therefore, is to propose a clear, adequate and
                        flexible definition of the metadata needed for
                        climate and forecast data. Although this is
                        specifically a netCDF standard, we feel that
                        most of the ideas are of wider application. The
                        metadata objects could be contained in file
                        formats other than netCDF. Conversion of the
                        metadata between files of different formats will
                        be facilitated if conventions for all formats
                        are based on similar ideas.
                </para>
                <para>
                        This convention is designed to be backward
                        compatible with the COARDS conventions <biblioref linkend="coards"/>,
                        by which we mean that a conforming COARDS dataset
                        also conforms to the CF standard. Thus new
                        applications that implement the CF conventions
                        will be able to process COARDS datasets.
                </para>
                <para>
                        We have also striven to maximize conformance
                        to the COARDS standard, that is, wherever the
                        COARDS metadata conventions provide an adequate
                        description we require their use. Extensions to
                        COARDS are implemented in a manner such that the
                        content that doesn't depend on the extensions
                        is still accessible to applications that adhere
                        to the COARDS standard.
                </para>

        </section>
    
        <section id="terminology">
                <title>Terminology</title>
                <para>
                        The terms in this document that refer to
                        components of a netCDF file are defined in the
                        NetCDF User's Guide (NUG) 
                        <biblioref linkend="nug"/> NUG.
                        Some of those
                        definitions are repeated below for convenience.
                </para>
                        <variablelist>
                                <varlistentry>
                                        <term>auxiliary coordinate variable</term>
                                        <listitem>
                                                <para>
                                                        Any netCDF variable that contains
                                                        coordinate data, but is not a coordinate
                                                        variable (in the sense of that term defined by the NUG
                                                        and used by this standard - see below). Unlike
                                                        coordinate variables, there is no relationship
                                                        between the name of an auxiliary coordinate
                                                        variable and the name(s) of its dimension(s).
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>boundary variable</term>
                                        <listitem>
                                                <para>
                                                        A boundary variable is associated with a variable that contains
                                                        coordinate data. When a data value provides information about
                                                        conditions in a cell occupying a region of space/time or some
                                                        other dimension, the boundary variable provides a description
                                                        of cell extent.
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>CDL syntax</term>
                                        <listitem>
                                                <para>
                                                        The ascii format used to describe the contents of a netCDF
                                                        file is called CDL (network Common Data form Language). This
                                                        format represents arrays using the indexing conventions of
                                                        the C programming language, i.e., index values start at 0, and in
                                                        multidimensional arrays, when indexing over the elements of
                                                        the array, it is the last declared dimension that is the fastest
                                                        varying in terms of file storage order. The netCDF utilities ncdump
                                                        and ncgen use this format (see 
                                                        <ulink url="http://www.unidata.ucar.edu/packages/netcdf/guidef/guidef-15.html#HEADING15-0">
                                                                chapter 10 of the NUG
                                                        </ulink>
                                                        ). All examples
                                                        in this document use CDL syntax.
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>cell</term>
                                        <listitem>
                                                <para>
                                                        A region in one or more dimensions whose boundary can be described by a set of vertices. The term <emphasis>interval</emphasis> is sometimes used for one-dimensional cells. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>coordinate variable</term>
                                        <listitem>
                                                <para>
                                                        We use this term precisely as it is defined in section 
                                                        <ulink url="http://www.unidata.ucar.edu/packages/netcdf/guidef/guidef-7.html#HEADING7-67">
                                                        2.3.1 of the NUG
                                                        </ulink>
                                                        . It is a one-dimensional variable with the
                                                        same name as its dimension [e.g., <varname>time(time)</varname>], and
                                                        it is defined as a numeric data type with values
                                                        that are ordered monotonically.  Missing values
                                                        are not allowed in coordinate variables.
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                                <term>grid mapping variable</term>
                                        <listitem>
                                                <para>
                                                        A variable used as a container for attributes that define a specific grid mapping. The type of the variable is arbitrary since it contains no data. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                                <term>latitude dimension</term>
                                        <listitem>
                                                <para>
                                                        A dimension of a netCDF variable that has an associated latitude coordinate variable. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>longitude dimension</term>
                                        <listitem>
                                                <para>
                                                        A dimension of a netCDF variable that has an associated longitude coordinate variable. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>multidimensional coordinate variable</term>
                                        <listitem>
                                                <para>
                                                        An auxiliary coordinate variable that is multidimensional. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                                <term>recommendation</term>
                                        <listitem>
                                                <para>
                                                        Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>scalar coordinate variable</term>
                                        <listitem>
                                                <para>
                                                        A scalar variable that contains coordinate data. Functionally equivalent to either a size one coordinate variable or a size one auxiliary coordinate variable. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>spatiotemporal dimension</term>
                                        <listitem>
                                                <para>
                                                        A dimension of a netCDF variable that is used to identify a location in time and/or space. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>time dimension</term>
                                        <listitem>
                                                <para>
                                                        A dimension of a netCDF variable that has an associated time coordinate variable. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                                <varlistentry>
                                        <term>vertical dimension</term>
                                        <listitem>
                                                <para>
                                                        A dimension of a netCDF variable that has an associated vertical coordinate variable. 
                                                </para>
                                        </listitem>
                                </varlistentry>
                        </variablelist>

        </section>
        <section>
                <title>Overview</title>
                <para>
                        No variable or dimension names are standardized
                        by this convention. Instead we follow the lead
                        of the NUG
                        and standardize only the names
                        of attributes and some of the values taken by
                        those attributes. The overview provided in this
                        section will be followed with more complete
                        descriptions in following sections. 
                        <xref linkend="attribute-appendix"/>
                        contains a summary of all the attributes used
                        in this convention.
                </para>
                <para>
                        We recommend that the NUG defined attribute
                        <varname>Conventions</varname>
                        be given the string value "<varname>CF-1.0</varname>"
                        to identify datasets that conform to these
                        conventions.
                </para>
                <para>
                        The general description of a file's contents
                        should be contained in the following attributes:
                        <varname>title</varname>, 
                        <varname>history</varname>, 
                        <varname>institution</varname>, 
                        <varname>source</varname>, 
                        <varname>comment</varname>
                        and 
                        <varname>references</varname>
                        (<xref linkend="description-of-file-contents"/>). 
                        For backwards compatibility with COARDS none
                        of these attributes is required, but their
                        use is recommended to provide human readable
                        documentation of the file contents.
                </para>
                <para>
                        Each variable in a netCDF file has an associated
                        description which is provided by the attributes
                        <varname>units</varname>,
                        <varname>long_name</varname>, and
                        <varname>standard_name</varname>. The
                        <varname>units</varname>,
                        and <varname>long_name</varname>
                        attributes are defined in the NUG and the
                        <varname>standard_name</varname> attribute is
                        defined in this document.
                </para>
                <para>
                        The 
                        <varname>units</varname> 
                        attribute is required for all variables
                        that represent dimensional quantities (except for
                        boundary variables defined in 
                        <xref linkend="cell-boundaries"/>. 
                        The values of the 
                        <varname>units</varname> 
                        attributes are character
                        strings that are recognized by UNIDATA's Udunits
                        package 
                        <citation><link linkend="udunits">UDUNITS</link></citation>,
                        (with exceptions allowed as discussed in
                        <xref linkend="units"/>). 
                </para>
                <para>
                        The 
                        <varname>long_name</varname> 
                        and 
                        <varname>standard_name</varname> 
                        attributes are
                        used to describe the content of each variable. For
                        backwards compatibility with COARDS neither
                        is required, but use of at least one of them
                        is strongly recommended. The use of standard
                        names will facilitate the exchange of climate
                        and forecast data by providing unambiguous
                        identification of variables most commonly
                        analyzed.
                </para>
                <para>
                        Four types of coordinates receive special
                        treatment by these conventions: latitude,
                        longitude, vertical, and time. Every variable
                        must have associated metadata that allows
                        identification of each such coordinate that is
                        relevant. Two independent parts of the convention
                        allow this to be done. There are conventions that
                        identify the variables that contain the coordinate
                        data, and there are conventions that identify
                        the type of coordinate represented by that data.
                </para>
                <para>
                        There are two methods used to identify variables
                        that contain coordinate data. The first is to
                        use the NUG-defined "coordinate variables." <emphasis>The
                        use of coordinate variables is required for all
                        dimensions that correspond to one dimensional
                        space or time coordinates</emphasis>. In cases where
                        coordinate variables are not applicable,
                        the variables containing coordinate data are
                        identified by the 
                        <varname>coordinates</varname> 
                        attribute.
                </para>
                <para>
                        Once the variables containing coordinate data are
                        identified, further conventions are required to
                        determine the type of coordinate represented by
                        each of these variables. Latitude, longitude,
                        and time coordinates are identified solely by
                        the value of their 
                        <varname>units</varname>
                        attribute. Vertical
                        coordinates with units of pressure may also
                        be identified by the 
                        <varname>units</varname> 
                        attribute. Other
                        vertical coordinates must use the attribute
                        <varname>positive</varname>
                        which determines whether the direction of
                        increasing coordinate value is up or down. Because
                        identification of a coordinate type by its units
                        involves the use of an external software package
                        <biblioref linkend="udunits" />,
                        we provide the optional attribute
                        <varname>axis</varname>
                        for a direct identification of coordinates
                        that correspond to latitude, longitude, vertical,
                        or time axes.
                </para>
                <para>
                        Latitude, longitude, and time are defined
                        by internationally recognized standards,
                        and hence, identifying the coordinates of
                        these types is sufficient to locate data
                        values uniquely with respect to time and a
                        point on the earth's surface. On the other
                        hand identifying the vertical coordinate is
                        not necessarily sufficient to locate a data
                        value vertically with respect to the earth's
                        surface. In particular a model may output data
                        on the dimensionless vertical coordinate used
                        in its mathematical formulation. To achieve the
                        goal of being able to spatially locate all data
                        values, this convention includes the definitions
                        of common dimensionless vertical coordinates in
                        <xref linkend="dimensionless-v-coord"/>. 
                        These definitions provide a mapping
                        between the dimensionless coordinate values
                        and dimensional values that can be uniquely
                        located with respect to a point on the earth's
                        surface. The definitions are associated with
                        a coordinate variable via the 
                        <varname>standard_name</varname>
                        and 
                        <varname>formula_terms</varname>
                        attributes. For backwards
                        compatibility with COARDS use of these attributes
                        is not required, but is strongly recommended.
                </para>
                <para>
                        It is often the case that data values are not
                        representative of single points in time and/or
                        space, but rather of intervals or multidimensional
                        cells. This convention defines a 
                        <varname>bounds</varname>
                        attribute
                        to specify the extent of intervals or cells. When
                        data that is representative of cells can be
                        described by simple statistical methods, those
                        methods can be indicated using the 
                        <varname>cell_methods</varname>
                        attribute. An important application of this
                        attribute is to describe climatological and
                        diurnal statistics.
                </para>
                <para>
                        Methods for reducing the total volume of data
                        include both packing and compression. Packing
                        reduces the data volume by reducing the precision
                        of the stored numbers. It is implemented using
                        the attributes 
                        <varname>add_offset</varname> 
                        and 
                        <varname>scale_factor</varname> 
                        which
                        are defined in the NUG. Compression on the other
                        hand loses no precision, but reduces the volume by
                        not storing missing data. The attribute 
                        <varname>compress</varname>
                        is defined for this purpose.
                </para>
        </section>
        <section id="coards-relationship">
                <title>Relationship to the COARDS Conventions</title>
                <para>
                        These conventions generalize and extend the
                        COARDS conventions 
                        <biblioref linkend="coards"/>. 
                        A major design goal
                        has been to maintain <emphasis>backward compatibility</emphasis> with
                        COARDS. Hence applications written to process
                        datasets that conform to these conventions
                        will also be able to process COARDS conforming
                        datasets. We have also striven to maximize
                        <emphasis>conformance</emphasis> to the COARDS standard so that
                        datasets that only require the metadata that was
                        available under COARDS will still be able to be
                        processed by COARDS conforming applications. But
                        because of the extensions that provide new
                        metadata content, and the relaxation of some
                        COARDS requirements, datasets that conform
                        to these conventions will not necessarily
                        be recognized by applications that adhere to
                        the COARDS conventions. The features of these
                        conventions that allow writing netCDF files that
                        are not COARDS conforming are summarized below.
                </para>
                <para>
                        COARDS standardizes the description of grids
                        composed of independent latitude, longitude,
                        vertical, and time axes. In addition
                        to standardizing the metadata required to
                        identify each of these axis types COARDS
                        restricts the axis (equivalently dimension)
                        ordering to be longitude, latitude, vertical,
                        and time (with longitude being the most rapidly
                        varying dimension). Because of I/O performance
                        considerations it may not be possible for models
                        to output their data in conformance with the
                        COARDS requirement. The CF convention places no
                        rigid restrictions on the order of dimensions,
                        however we encourage data producers to make the
                        extra effort to stay within the COARDS standard
                        order. The use of non-COARDS axis ordering will
                        render files inaccessible to some applications
                        and limit interoperability. Often a buffering
                        operation can be used to miminize performance
                        penalties when axis ordering in model code does
                        not match the axis ordering of a COARDS file.
                </para>
                <para>
                        COARDS addresses the issue of identifying
                        dimensionless vertical coordinates, but does
                        not provide any mechanism for mapping the
                        dimensionless values to dimensional ones that
                        can be located with respect to the earth's
                        surface. For backwards compatibility we continue
                        to allow (but do not require) the 
                        <varname>units</varname>
                        attribute of dimensionless vertical coordinates to take the
                        values "level", "layer", or "sigma_level." But we
                        recommend that the 
                        <varname>standard_name</varname> and 
                        <varname>formula_terms</varname>
                        attributes be used to identify the appropriate
                        definition of the dimensionless vertical
                        coordinate (see 
                        <xref linkend="dimensionless-vertical-coordinate"/>).
                </para>
                <para>
                        The CF conventions define attributes which
                        enable the description of data properties
                        that are outside the scope of the COARDS
                        conventions. These new attributes do not violate
                        the COARDS conventions, but applications that
                        only recognize COARDS conforming datasets will not
                        have the capabilities that the new attributes are
                        meant to enable. Briefly the new attributes allow:
                </para>

                <itemizedlist>
                        <listitem>
                                <para> Identification of quantities using standard names. </para>
                        </listitem>
                        <listitem>
                                <para> Description of dimensionless vertical coordinates. </para>
                        </listitem>
                        <listitem>
                                <para> Associating dimensions with auxiliary coordinate variables. </para>
                        </listitem>
                        <listitem>
                                <para> Linking data variables to scalar coordinate variables. </para>
                        </listitem>
                        <listitem>
                                <para> Associating dimensions with labels. </para>
                        </listitem>
                        <listitem>
                                <para>Description of intervals and cells. </para>
                        </listitem>
                        <listitem>
                                <para> Description of properties of data defined on intervals and cells. </para>
                        </listitem>
                        <listitem>
                                <para> Description of climatological statistics. </para>
                        </listitem>
                        <listitem>
                                <para> Data compression for variables with missing values. </para>
                        </listitem>
                </itemizedlist>
        </section>
</chapter>