FEP - Format Use by a Project - International Solar-Terrestrial Physics Project (ISTP) - CDF

1. Format (Format System) Identification

Common Data Format versions 2 and higher using the ISTP/IACG CDF standards. Note that there is a detached SFDU associated with all ISTP/IACG CDF files.

2. Data Model Extensions by Project

[Editors Note: This question was added after form was originally completed.]

3. Original Format Selection Rationale

The original rationale for encapsulating the ISTP higher level products in a standard format is derived from the OPEN Project enabling documentation, "Origin of Plasmas in the Earth's Neighborhood (OPEN), Final Report of the Science Definition Working Group", April 1979. This Report envisioned that a Central Data Handling Facility (CDHF) would compute Level 1B products from the OPEN spacecraft (OPEN was brought under the ISTP umbrella) and ingest Level 1B and higher level products, from a variety of other sources (including space-based, ground-based, and theory-based) encapsulated in a TBD format that would expedite the documentation, merging, display, and analysis of these data. During 1991 the GGS/ISTP Data Committee, after almost 10 years of sometimes acrimonious deliberation, settled on CDF as the standard format for ISTP data, particularly the Key Parameters (relative low time resolution data used primarily for event identification). The first ISTP spacecraft, Geotail, was launched in July 1992, WIND in November 1994, SOHO in December 1995, and POLAR in February of 1996.

It is an important to note that the European Cluster and Equator-S missions have adopted ISTP/IACG CDF as the vehicle for encapsulating the Summary and Prime Parameters. This alone will constitutes a substantial quantity of data as Cluster is 4 spacecraft, 44 instruments total for at least 2 years with a launch in 2000. Cluster has National Data Centers across Europe, Scandinavia, the US, as well as Beijing China.

4. Data Types

All instruments on the ISTP spacecraft along with collaborating ground-based investigations (Canopus, DARN, SESAME, Sondestromfjord) and collaborating space-based investigations [Equator-S, IMP-8, FAST, GOES (series), LANL (series), Interball- Aurora & Tail, SAMPEX, ACE, Triana (when launched), Cluster II (when launched) supply the CDHF with Level 1B products in CDF. In addition a number of the ISTP investigations also supply higher time resolution data to the CDHF in CDF. All of these products are distributed electronically and via CD-ROM and all reside in CDAWeb and are archived at the NSSDC facility. Currently this combined data set is approaching 100 Gbytes in size.

Object types include images, spectra, time-series, and multi- variate distributions.

5. Support

NASA/GSFC Code 630 has developed and supported CDF.

6. Project Software

CDAWeb and the Key Parameter Visualization Tool (KPVT) have been developed to support the documentation, access, and display of CDF data and metadata. KPVT supports a variety of platforms and is available on the CD-ROMs and electronically. CDAWeb is Web based and is highly acclaimed for it utility and user friendliness. Also see 7.) below.

7. Experience

The extensive use of CDF and associated tools, since 1992 in the ISTP Project, has resulted in a mature product that has evolved to meet our needs.

8. Desired Functions

CDF support is in the process of releasing a tool to aid in the making of CDFs which should substantially enhance the ability to readily get data into CDF. This is an important tool that we have needed for sometime.

9. Selection Criteria

I consider CDF and the associated tools to be mature enough that they could be successfully employed in a new project without difficulty. Much of this maturation is the result of the demands made on CDF by ISTP and the IACG since 1992 and are completely "field tested" in this environment.

10. Handling Plans for Format Evolution

Format evolution has to be very carefully coordinated across the ISTP, Cluster, CDAWeb, KPVT environments. We consider CDF to be in a mature state.

11. Other Comments

Detached SFDU's are associated with every CDF file and are used by ISTP for cataloging and indexing the data sets. CD-ROMs carry a cumulative index which is also available on the Web.

Wider Views