|
GEWEX RFA Data File Format and File Naming Convention |
|
The data and documentation supplied for participation in the GEWEX RFA must
adhere to the following format and file naming requirements.
Table of Contents:
For purposes of the Radiative Flux Assessment, a data product is defined
to consist of all of the files sharing the same <ProductName> and
<ProductVersion> tags in their file names. (See
naming convention below.) A single data product may
consist of numerous parameters, but all of these should be related, being
produced by the same algorithm, institution, etc. Selection of the data
product name is left to the discretion of the data provider, however the name
should be clearly related to the name of the original archived data set. If the
data is not a subset of a currently available public/archived data set (e.g., a
new calibration has been applied, etc.), please append -MOD at the end of the
product name. Examples of typical data products and possible RFA-style names
for them are shown below.
Data Product |
<ProductName>_<ProductVersion> |
ISCCP FD TOA and surface data |
ISCCP-FD_Ed000 |
NASA/GEWEX SRB version 2.7, SW fluxes from Pinker algorithm |
GEWEX-SRBGSW_Ed027 |
CERES Monthly TOA/Surface Averages from Terra, incorporating data from
geostationary satellites, Edition 2d including the CERES Rev. 1 correction |
CERES-SRBAVG-Terra-GEO-MOD_Ed02d |
Official BSRN surface measurement data |
BSRN_Ed001 |
A single data description file,
containing data quality information and references to complete documentation,
will be stored for each data product. Each time data is added to, removed
from, or modified in the file set for a product, an updated documentation file
must be submitted. The name of this file must be
<ProductName>_<ProductVersion>.txt, following the convention given
below.
Sample images are to be in GIF, JPG, PNG, or TIFF format with the
appropriate extension.
Data are to be written in ASCII according to the formats specified
below. Data files must have the extension .asc.
Each data file will contain a single parameter as described in
Table 5 below.
Unless otherwise specified, all sampling/averaging times are to be given
in UTC.
The date is defined to change at 0:00 hours whether time is specified in
LST or UTC. Note that, for LST, the date is set relative to the 180°
meridian, not the International Date Line.
"Monthly" data should correspond to actual calendar months,
including 29 days in leap-year Februaries.
Time series files containing 15-minute data should start at the top of
an hour to avoid ambiguity in the file name time tag. Averaging intervals for
these data are 0-14, 15-29, 30-44, and 45-59 minutes in each hour.
While data providers are free to submit data of any or all types defined in
this document, it is recommended that at least the following be supplied for
each satellite data product:
- 2.5° global maps
- Global mean time series
- At least some zonal (or regional) mean time series
All satellite data files are to be written in ASCII format, one F10.3
value per line. In place of missing values, use -9999.000.
Map data must must have global coverage and be provided on a 2.5°
latitude/longitude equal-angle grid. The grid begins at the South Pole and
the International Date Line (-90°, -180°) and increments eastward
and then northward for a total of 10368 global grid cells. The first grid
cell (cell 1) is bounded by latitude -90° to -87.5° and longitude
-180° to -177.5°. The second grid cell (cell 2) is bounded by
latitude -90° to -87.5° and longitude -177.5° to -175°. The
last grid cell (cell 10368) is bounded by latitude 87.5° to 90° and
longitude 177.5° to 180°. The
center points of the grid cells are
listed for reference.
Satellite values at scales greater than one day should represent
averages over the entire interval selected (from Table
3). For scales less than one day (including monthly-diurnal), satellite
values may be averaged or instantaneous, but this should be indicated in the
documentation file.
- Long-term low temporal resolution: These files should cover a minimum
of 5 continuous years.
- Monthly averaged in UTC or LST
- Daily averaged in UTC or LST
- Monthly averaged 15-minute (diurnal) in UTC or LST
- Short-term high temporal resolution: These files should span the year
2004.
- 15-minute averaged in UTC
All ground-based measurement data files must be in ASCII and use the
following format for each line:
F10.3, 1x, F13.4, 1x, I6, 1x, I6, 1x, I6, 1x, F10.3
where the data formats correspond to the following variables:
- mean flux
- time stamp of the form YYYYMMDD.hhmm
- number of measured values used to compute the mean (and standard deviation)
- number of fill values used to compute the mean (and standard deviation)
- total possible number of values for these computations
- standard deviation of the averaged values
Time conventions
Values given in ground site files should be averaged over the period
selected from Table 3, not instantaneous samples.
The chief goal of the LST averages is to have both sunrise and sunset
on the same day. For this reason, data in local solar time will also be
accepted in this category. The time system used should be noted in the data
documentation file.
In the time stamp, hh and mm are not decimal times but just the standard
hour and minute separated from the date by a period. Any unused digits should
be filled with 9s, i.e., 20021099.9999 is a possible time stamp for monthly
data, 20021030.9999 for daily data, 20021099.1415 for monthly mean diurnal
data, and 20041030.1415 for 15-minute data.
For both types of 15-minute data, the time stamp used should correspond
to the BEGINNING of the averaging period. Thus the choices for the minute
portion of the time stamp are 00, 15, 30, and 45. This standard is NOT meant
to imply that data collection for the first quarter hour must cover the
period hh:00:00-hh:14:59. Deviations of up to a minute on either end of
the averaging period will be allowed.
Other items in format
We realize that flux measurements are not made to mW m-2
precision. Please add trailing zeros to fill in unused decimal places.
Time stamps are required. Despite this, no gaps are to occur in files,
i.e., every possible sample time should have a value. If no credible value
is available for a given time, the fill value of -9999.000 should be substituted.
No QC flags will be provided. It is up to the data provider to determine
whether the values they have are credible or not; if not, do not include
this data.
Inclusion of the standard deviation is optional. If standard deviations
are not included, please substitute the standard fill value of -9999.000 in
this position.
Whether standard deviations are provided or not, a breakdown of the number
and type of sample points used to determine the mean as well as the total
possible points for the interval should be provided. If these numbers are
not available, please substitute the fill value -9999 in these positions.
Data files must adhere to the following file naming conventions. Use the
tables below for specific fields within the file name:
<ProductName>_<ProductVersion>_<DataDescriptors>_<TimeTag>_<RFAVersion>.<Extension>
where
- <ProductName>: Unique product name selected by data provider. This
name should be clearly related to the name of the original archived data set.
If the data is not a subset of the currently available public/archived data set
(e.g., a new calibration has been applied, etc.), please append -MOD at the end
of the product name.
- <ProductVersion>: Version of data product, in format Edccc, where
ccc is an alphanumeric string (letters and numbers only), or EdXXX if not
applicable.
- <DataDescriptors>: Concatenation of the following fields separated by
hyphens (-)
- <TimeTag>:
For maps, time of validity in the format YYYYMMDDHH. (For data averaged over a
precession cycle, use the averaging start time.)
For time series, start and end times of data in the format YYYYMMDDHH-YYYYMMDDHH, where
- YYYY - 4 digit year
- MM - 2 digit month, 01 through 12, 99 if not applicable
- DD - 2 digit day, 01 through 31, 99 if not applicable
- HH - 2 digit hour, 00 through 23, 99 if not applicable
(For data averaged over a precession cycle, use the start time of the first
averaging period and the end time of the last period. Time series files
containing 15-minute data should start at the top of an hour to avoid ambiguity
in this tag.)
- <RFAVersion>: Submittal version, in format RFAnn, where nn is a
zero-filled integer. Increment nn by 1 each time a given set of data is
resubmitted (due to formatting or submission errors).
- <Extension>: Allowed values are txt, asc, png, gif, jpg, tiff, tif.
- TOA monthly gridded all-sky SW up data for January 2002 from CERES TISA:
CERESTISA_Ed01a_TOA-MAP-MON-GLOB-ASWUP_2002019999_RFA01.asc
- Sample image for same TOA monthly gridded all-sky SW up data:
CERESTISA_Ed01a_TOA-MAP-MON-GLOB-ASWUP_2002019999_RFA01.png
- ISCCP gridded daily average net surface flux for 1 May 1999 using experimental
calibration values (unreleased data):
ISCCP-MOD_Ed031_SFC-MAP-DAY-GLOB-ATOTNET_1999050101_RFA02.asc
- Surface clear-sky LW up hourly time series for the tropical zone, 1999-2003:
SRBQSW_Ed025_SFC-TS-HRY-20S20N-CLWUP_1999010101-2003123124_RFA01.asc
- Time series of daily-averaged clear-sky surface albedo at Barrow, Alaska,
for January 1994:
CERESPM2_Ed02b_SFC-TS-DAY-BAR-CALB_1994010199-1994013199_RFA02.asc
- Time series of TOA monthly-averaged 3-hourly diurnal clear-sky LW down
for 2000:
ERA40_Ed026_TOA-TS-MOD3-GLOB-CLWDN_2000019999-2000129999_RFA01.asc
- Monthly time series of all-sky SW down for 2003 at Barrow, Alaska, from
BSRN:
BSRN_EdXXX_MEA-TS-MON-BAR-ASWDN_2003019999-2003129999_RFA01.asc
- Hourly time series clear-sky LW up for August 2003 at Boulder SURFRAD
site:
SURFRAD_Ed031_MEA-TS-HRY-BOS-CLWUP_2003080101-2003083124_RFA04.asc
Table 1. Data Categories
Category |
Identifier |
Satellite-based TOA data products |
TOA |
Satellite-based surface data products |
SFC |
Ground-based surface data |
MEA |
Table 3. Temporal Sampling or Averaging Periods
Time Interval |
Identifier |
Yearly |
YR |
Precession cycle of nn days (where nn is a
zero-filled integer) |
PRnn |
Monthly |
MON |
Monthly - averaged on local standard time |
MONLST |
Monthly-diurnal (15 minute) |
MOD15 |
Monthly-diurnal (15 minute) - averaged on local
standard time |
MOD15LST |
Monthly-diurnal (hourly) |
MOD1 |
Monthly-diurnal (hourly) - averaged on local
standard time |
MOD1LST |
Monthly-diurnal (3-hourly) |
MOD3 |
Monthly-diurnal (3-hourly) - averaged on local
standard time |
MOD3LST |
Monthly-diurnal (6-hourly) |
MOD6 |
Monthly-diurnal (6-hourly) - averaged on local
standard time |
MOD6LST |
Daily |
DAY |
Daily - averaged on local standard time |
DAYLST |
Hourly |
HRY |
15-minute |
MIN15 |
Seasonal averages: |
December, January, February | DJF |
March, April, May | MAM |
June, July, August | JJA |
September, October, November | SON |
Table 4. Spatial Sampling or Averaging Areas
Spatial Area |
Identifier |
Global | GLOB |
Global - land only1 | GLOBLND |
Global - ocean only1 | GLOBOCN |
Hemispherical: |
Northern Hemisphere | NH |
Northern Hemisphere - land only1 | NHLND |
Northern Hemisphere - ocean only1 | NHOCN |
Southern Hemisphere | SH |
Southern Hemisphere - land only1 | SHLND |
Southern Hemisphere - ocean only1 | SHOCN |
Zonal: |
See list in Table 6. |
Regional: |
Marine stratus area (0S-30S, 70W-100W) |
MBL |
Tropical convection area (15S-15N, 130E-160E) | TRC |
Site: |
See list in Table 7. |
1
Download land-ocean mask. |
Table 5. Parameters
"Type 2 clear sky" refers to values that are computed whether or not
clear sky is actually detected. Regular or "type 1" clear sky values
are derived from measurements that occurred when clear sky was
detected.
Parameter |
Units |
Identifier |
All-sky shortwave (SW) upward flux |
Wm-2 |
ASWUP |
All-sky shortwave downward total (direct + diffuse) flux |
Wm-2 |
ASWDN |
All-sky shortwave downward diffuse flux |
Wm-2 |
ASWDIF |
All-sky shortwave downward direct horizontal flux |
Wm-2 |
ASWDIR |
All-sky shortwave downward hemispheric1 flux |
Wm-2 |
ASWDHEM |
All-sky shortwave net (up + total down) flux |
Wm-2 |
ASWNET |
All-sky longwave (LW) upward flux |
Wm-2 |
ALWUP |
All-sky longwave downward flux |
Wm-2 |
ALWDN |
All-sky longwave net (up + down) flux |
Wm-2 |
ALWNET |
All-sky total (SW + LW) net (up + down) flux |
Wm-2 |
ATOTNET |
All-sky broadband shortwave albedo |
- |
AALB |
Clear-sky shortwave upward flux |
Wm-2 |
CSWUP |
Clear-sky shortwave downward total (direct + diffuse) flux |
Wm-2 |
CSWDN |
Clear-sky shortwave downward diffuse flux |
Wm-2 |
CSWDIF |
Clear-sky shortwave downward direct flux |
Wm-2 |
CSWDIR |
Clear-sky shortwave downward hemispheric1 flux |
Wm-2 |
CSWDHEM |
Clear-sky shortwave net (up + total down) flux |
Wm-2 |
CSWNET |
Clear-sky longwave upward flux |
Wm-2 |
CLWUP |
Clear-sky longwave downward flux |
Wm-2 |
CLWDN |
Clear-sky longwave net (up + down) flux |
Wm-2 |
CLWNET |
Clear-sky total (SW + LW) net (up + down) flux |
Wm-2 |
CTOTNET |
Clear-sky broadband shortwave albedo |
- |
CALB |
Type 2 clear-sky shortwave upward flux |
Wm-2 |
CSWUP2 |
Type 2 clear-sky shortwave downward total (direct + diffuse) flux |
Wm-2 |
CSWDN2 |
Type 2 clear-sky shortwave downward diffuse flux |
Wm-2 |
CSWDIF2 |
Type 2 clear-sky shortwave downward direct flux |
Wm-2 |
CSWDIR2 |
Type 2 clear-sky shortwave downward hemispheric1 flux |
Wm-2 |
CSWDHEM2 |
Type 2 clear-sky shortwave net (up + total down) flux |
Wm-2 |
CSWNET2 |
Type 2 clear-sky longwave upward flux |
Wm-2 |
CLWUP2 |
Type 2 clear-sky longwave downward flux |
Wm-2 |
CLWDN2 |
Type 2 clear-sky longwave net (up + down) flux |
Wm-2 |
CLWNET2 |
Type 2 clear-sky total (SW + LW) net (up + down) flux |
Wm-2 |
CTOTNET2 |
Type 2 clear-sky broadband shortwave albedo |
- |
CALB2 |
Skin temperature |
K |
TSKIN |
Near surface air temperature |
K |
T2M |
Surface emissivity |
- |
SFCEMISS |
Total column water vapor |
kg m-2 |
TCWV |
1 Single instrument measurement of
broadband SW downward flux including both direct and diffuse components. |
Table 6. Zones
(See Table 4 for additional spatial averaging
regions.)
Spatial Area |
Identifier |
ERBE zone1 |
60°S - 60°N | 60S60N |
60°S - 60°N | 60S60NLND |
60°S - 60°N | 60S60NOCN |
Zone set 1 |
75°N - 90°N | 75N90N |
60°N - 75°N | 60N75N |
40°N - 60°N | 40N60N |
20°N - 40°N | 20N40N |
20°S - 20°N | 20S20N |
20°S - 40°S | 20S40S |
40°S - 60°S | 40S60S |
60°S - 75°S | 60S75S |
75°S - 90°S | 75S90S |
Zone set 1 - land only1 |
75°N - 90°N | 75N90NLND |
60°N - 75°N | 60N75NLND |
40°N - 60°N | 40N60NLND |
20°N - 40°N | 20N40NLND |
20°S - 20°N | 20S20NLND |
20°S - 40°S | 20S40SLND |
40°S - 60°S | 40S60SLND |
60°S - 75°S | 60S75SLND |
75°S - 90°S | 75S90SLND |
Zone set 1 - ocean only1 |
75°N - 90°N | 75N90NOCN |
60°N - 75°N | 60N75NOCN |
40°N - 60°N | 40N60NOCN |
20°N - 40°N | 20N40NOCN |
20°S - 20°N | 20S20NOCN |
20°S - 40°S | 20S40SOCN |
40°S - 60°S | 40S60SOCN |
60°S - 75°S | 60S75SOCN |
75°S - 90°S | 75S90SOCN |
1
Download land-ocean mask. |
Table 7. Measurement sites
Name |
Identifier |
Latitude |
Longitude |
Ahmadabad, India |
AHM |
23° 04' N |
72° 38' E |
Albany, NY |
ALB |
42° 42' N |
73° 50' W |
Alice Springs, Australia |
ASP |
23° 48' S |
133° 53' E |
ARM Southern Great Plains Extended Facility 1 |
E01 |
38.202° N |
99.316° W |
ARM Southern Great Plains Extended Facility 3 |
E03 |
38.201° N |
95.597° W |
ARM Southern Great Plains Extended Facility 13 |
E13 |
36° 36' N |
97° 30' W |
ARM Southern Great Plains Extended Facility 20 |
E20 |
35.564° N |
96.988° W |
ARM Southern Great Plains Extended Facility 22 |
E22 |
35.354° N |
98.977° W |
Atqasuk, AK (ARM) |
ATQ |
70.472° N |
157.408° W |
Balbina, Brazil |
BAL |
03° 10' S |
60° 00' W |
Barrow, AK (ARM) |
BRW |
71.323° N |
156.616° W |
Barrow, AK (CMDL) |
BAR |
71° 19' N |
156° 36' W |
Bermuda |
BER |
32° 18' N |
64° 46' W |
Billings, OK (BSRN site at ARM SGP Central Facility) |
BIL |
36° 36' N |
97° 31' W |
Bondville, IL (SURFRAD) |
BON |
40° 04' N |
88° 22' W |
Bordeaux, France |
BRD |
44° 50' N |
0° 42' W |
Boulder, CO |
BOU |
40° 03' N |
105°00' W |
Boulder, CO (SURFRAD) |
BOS |
40° 08' N |
105° 14' W |
Brindisi, Italy |
BRN |
40° 39' N |
17° 57' E |
Budapest-Lorinc, Hungary |
BUD |
47° 50' N |
19° 05' E |
Burns, OR |
BUO |
43.52° N |
119.02° W |
Calcutta, India |
CLC |
22° 39' N |
88° 27' E |
Camborne, Great Britain |
CAM |
50° 13' N |
05° 19' W |
Cape Matatula, American Samoa (CMDL) |
SMO |
14.232° S |
170.563° W |
Carpentras, France |
CAR |
44° 03' N |
05° 02' E |
Chesapeake Lighthouse, VA |
CLH |
36° 54' N |
75° 43' W |
Cimetta, Switzerland (ASRB) |
CIM |
46° 12' N |
08° 48' E |
Darwin, Australia (ARM) |
DAR |
12.425° S |
130.891° E |
Davos, Switzerland (ASRB) |
DAV |
46° 49' N |
09° 51' E |
Davos, Switzerland (GEBA) |
DVG |
46° 49' N |
09° 51' E |
De Aar, South Africa |
DAA |
30° 40' S |
24° 00' E |
Desert Rock, NV (SURFRAD) |
DRA |
36° 39' N |
116° 01' W |
Eugene, OR |
EUO |
44.05° N |
123.07° W |
Florianopolis, Brazil |
FLO |
27° 32' S |
48° 31' W |
Fort Peck, MT (SURFRAD) |
FPE |
48° 19' N |
105° 06' W |
Fukuoka, Japan |
FUK |
33° 35' N |
130° 23' E |
Georg von Neumayer, Antarctica |
GVN |
70° 39' S |
08° 15' W |
Goodwin Creek, MS (SURFRAD) |
GCR |
34° 15' N |
89° 52' W |
Greenland Summit, Greenland |
GRS |
72° 34' N |
38° 29' W |
Hamburg, Germany |
HAM |
53° 39' N |
10° 07' E |
Hermiston, OR |
HEO |
45.82° N |
119.28° W |
Hradec Kralove, Czech Republic |
HRA |
50° 15' N |
15° 51' E |
Ilorin, Nigeria |
ILO |
08° 32' N |
04° 34' E |
Jokioinen, Finland |
JOK |
60° 49' N |
23° 30' E |
Jungfraujoch, Switzerland |
JUN |
46° 33' N |
07° 59' E |
Jungfraujoch, Switzerland (ASRB) |
JFJ |
46° 33' N |
07° 59' E |
Kagoshima, Japan |
KAG |
31° 34' N |
130° 33' E |
Kiruna, Sweden |
KIR |
67° 51' N |
20° 14' E |
Kwajalein, Marshall Islands |
KWA |
08° 43' N |
167° 44' E |
Lanzhou, China |
LZH |
36° 03' N |
103° 53' E |
Lauder, New Zealand |
LAU |
45° 00' S |
169° 41' E |
Lerwick, Shetland Islands, Great Britain |
LER |
60° 08' N |
01° 11' W |
Lindenberg, Germany |
LIN |
52° 13' N |
14° 07' E |
Locarno-Monti, Switzerland (ASRB) |
LOM |
46° 10' N |
08° 47' E |
Locarno-Monti, Switzerland (GEBA) |
LMG |
46° 10' N |
08° 47' E |
Malin Head, Ireland |
MLH |
55° 22' N |
07° 20' W |
Momote, Manus Island, Papua New Guinea (ARM) |
MAN |
02° 03' S |
147° 26' E |
Naha, Japan |
NAH |
26° 14' N |
127° 41' E |
Nandi, Fiji |
NAN |
17° 45' S |
177° 27' E |
Nauru Island, Republic of Nauru (ARM) |
NAU |
0.521° S |
166.916° E |
Ny Ålesund, Spitsbergen, Norway |
NYA |
78° 56' N |
11° 57' E |
Palaiseau Cedex, France |
PAL |
48° 43' N |
02° 12' E |
Payerne, Switzerland |
PAY |
46° 49' N |
06° 57' E |
Payerne, Switzerland (ASRB) |
PYR |
46° 49' N |
06° 57' E |
Penn State (Rock Springs), PA (SURFRAD) |
PSU |
40° 43' N |
77° 56' W |
Potsdam, Germany |
POT |
52° 23' N |
13° 06' E |
Regina, Saskatchewan, Canada |
REG |
50° 12' N |
104° 43' W |
Reykjavik, Iceland |
RKJ |
64° 8' N |
21° 54' W |
Sapporo, Japan |
SAP |
43° 03' N |
141° 20' E |
Sede Boqer, Israel |
SBO |
30° 52' N |
34° 46' E |
Shenyang, China |
SHN |
41° 44' N |
123° 27' E |
Sion, Switzerland |
SIO |
46° 13' N |
07° 20' E |
Sioux Falls, SD (SURFRAD) |
SXF |
43.730° N |
96.620° W |
Sodankyla, Finland |
SOD |
67° 22' N |
26° 39' E |
Solar Village, Riyadh, Saudi Arabia |
SOV |
24° 55' N |
46° 25' E |
South Pole, Antarctica |
SPO |
90° S |
|
Syowa, Antarctica |
SYO |
69° 00' S |
39° 35' E |
Tamanrasset, Algeria |
TAM |
22° 47' N |
05° 31' E |
Tateno, Japan |
TAT |
36° 03' N |
140° 08' E |
Toravere, Estonia |
TOR |
58° 16' N |
26° 28' E |
Uccle, Belgium |
UCL |
50° 48' N |
04° 21' E |
Ulan-Bator, Mongolia |
ULB |
47° 51' N |
106° 45' E |
Urumqi, China |
URM |
43° 47' N |
87° 37' E |
Valparaiso (USM), Chile |
VAL |
33° 02' S |
71° 29' W |
Wageningen, Netherlands |
WAG |
51° 58' N |
05° 39' E |
Warszawa, Poland |
WAR |
52° 16' N |
20° 59' E |
Weissfluhjoch, Switzerland (ASRB) |
WFJ |
46° 50' N |
09° 49' E |
Wuerzburg, Germany |
WRZ |
49° 46' N |
09° 58' E |
Zakopane, Poland |
ZAK |
49° 17' N |
19° 58' E |
Zvenigorod, Russia |
ZVE |
55° 41' N |
36° 46' E |
|