|
|
| MENA > UDF > UDF DLM Package for IDL |
UDF is a powerful, flexible data storage format. As always, though, with flexibility comes complexity. The UDF-DLM package is intended to provide a simple interface to UDF for IDL programmers.
This document provides a brief introduction to UDF-DLM. Please keep in mind that UDF-DLM is a work in progress, and this page may be out of date by the time you read it.
If you're anxious to get your hands on the goods, go to the download page.
Installation is beyond the scope of this document; please read the INSTALL file provided with the snapshot.
Most programmers are accustomed to the OPEN, READ, CLOSE model of file I/O. The UDF-DLM package provides an abstraction layer on top of the UDF libraries, so IDL programmers can access UDF data with a few seconds' work, instead of a few hours.
UDF-DLM can be used interactively, or in IDL programs. Although your eventual goal is doubtless to access UDF data from within your programs, this example introduces steps in interactive mode. It is assumed that you can take it from there.
IDL> key = udf_key( vinst_to_full_path('ImMStats') ) % Loaded DLM: UDF. IDL> help, key KEY ULONG = 101204098 IDL>
UDF_KEY takes five strings as input: either as five positional parameters, or as one 5-element IDL array (as returned by vinst_to_full_path() above). The returned key can be used by UDF_OPEN, UDF_TIMES, or UDF_PIDF. It is an opaque value, and you really don't need to know anything more about it.
The message "% Loaded DLM: UDF" appears only the first time you access one of the UDF-DLM functions.
NOTE: Try getting a key for your own instrument, not MENA. Even if you get a valid key, UDF_OPEN will fail if you have not installed any UDF data files for that instrument.
IDL> fh = udf_open(key, [2000, 130])
This initiates UDF access to the data set referenced by key, for day 130, 2000, 00:00 through 23:59:59.999. If it is successful, UDF_OPEN silently returns a "filehandle", which can then be passed on to UDF_READ or UDF_EOF.
If there are no data records for the given time period, you might see:
% UDF_OPEN: (file_open): UDF error [NO_DATA]
The probable cause for this is that there are no data files installed for this particular instrument (did you remember to get a key for your own data set, above?), or the data files are for a different time range.
UDF_OPEN can accept two optional keyword parameters: /GROUP and /COLLAPSE. These are described in the following section.
IDL> d = udf_read(fh) IDL> help, d, /struct ** Structure IMMSTATS, 29 tags, length=13360: BTIME STRUCT -> UDF_TIME Array[1] ETIME STRUCT -> UDF_TIME Array[1] D_QUAL DOUBLE 2.0000000 START_AZ FLOAT Array[128] STOP_AZ FLOAT Array[128] SCAN_INDEX FLOAT Array[128] START_HEIGHT FLOAT Array[128] STOP_HEIGHT FLOAT Array[128] START_POSITION FLOAT Array[128] STOP_POSITION FLOAT Array[128] COARSE_AZIMUTH FLOAT Array[128] FINE_AZIMUTH FLOAT Array[128] TOTAL_AZIMUTH FLOAT Array[128] TIME_OF_FLIGHT FLOAT Array[128] HEAD_ID FLOAT Array[128] PACKET_COUNTER FLOAT Array[128] H1_AVG_MCP_P4KV_MON FLOAT = Array[128] H2_AVG_MCP_P4KV_MON FLOAT = Array[128] H3_AVG_MCP_P4KV_MON FLOAT = Array[128] H1_LOW_LEV_PHA_THRES FLOAT = Array[128] H2_LOW_LEV_PHA_THRES FLOAT = Array[128] H3_LOW_LEV_PHA_THRES FLOAT = Array[128] SPIN_COUNTER FLOAT Array[128] GCI_SATELLITE_X FLOAT Array[128] GCI_SATELLITE_Y FLOAT Array[128] GCI_SATELLITE_Z FLOAT Array[128] GCI_S_C_SPIN_AXIS_X FLOAT = Array[128] GCI_S_C_SPIN_AXIS_Y FLOAT = Array[128] GCI_S_C_SPIN_AXIS_Z FLOAT = Array[128] IDL> help, d.btime, /st ** Structure UDF_TIME, 7 tags, length=20: YEAR INT 2000 DOY UINT 130 HOUR BYTE 14 MIN BYTE 16 SEC BYTE 12 MSEC UINT 851 DP DOUBLE 0.0000000 IDL> KEY: [user input] [tags common to all returned STRUCTs]
It doesn't get much simpler than that, does it?
The data structure contains one tag each for beginning and ending time, plus one tag for each sensor defined in the PIDF file. It is generated on-the-fly at run time.
As with any IDL data, you are now free to plot, analyze, bend, fold, spindle or mutilate it. You can even lump structs together... since it's a named structure, you can loop over UDF_READ (calling UDF_EOF to test for end-of-file) and append the structs together into an array of structs:
IDL> while not udf_eof(fh) do d = [d, udf_read(fh)] IDL> help,d D STRUCT = -> IMMSTATS Array[100] IDL>
If you had called UDF_OPEN with /GROUP, this is what your data structure would look like (highlights indicate changes from above):
** Structure IMMSTATS_G, 14 tags, length=13360: BTIME STRUCT -> UDF_TIME Array[1] ETIME STRUCT -> UDF_TIME Array[1] D_QUAL DOUBLE 2.0000000 START_AZ FLOAT Array[128] STOP_AZ FLOAT Array[128] SCAN_INDEX FLOAT Array[128] START_STOP_DATA FLOAT Array[128, 4] ANGLE_DATA FLOAT Array[128, 3] TOF_AND_HEAD_DATA FLOAT = Array[128, 2] PACKET_COUNTER FLOAT Array[128, 1] VOLTAGES_AND_THESHOLDS FLOAT = Array[128, 6] SPIN_COUNTER FLOAT Array[128, 1] GCI_SATELLITE_POSITION FLOAT = Array[128, 3] GCI_SATELLITE_SPIN_AXIS FLOAT = Array[128, 3] KEY: [tags common to all returned STRUCTs] [what has changed since the last one]
Groups are logical groupings of sensors, defined manually in the PIDF.
If you had called UDF_OPEN with /COLLAPSE, your data records would be fully collapsed into one tag, called DATA:
** Structure IMMSTATS_C, 7 tags, length=13360: BTIME STRUCT -> UDF_TIME Array[1] ETIME STRUCT -> UDF_TIME Array[1] D_QUAL DOUBLE 2.0000000 START_AZ FLOAT Array[128] STOP_AZ FLOAT Array[128] SCAN_INDEX FLOAT Array[128] DATA FLOAT Array[128, 23]
Note that the structure length never varies. This is as expected; the bits are the same, only their names have changed.
IDL> UDF_Close, fh
UDF defines ways to extract useful fields from the raw data, via its convert_to_units function. For instance, raw telemetry can be converted to counts, degrees, and so on.
UDF-DLM does this automatically... if there's a sensor defined in the PIDF file. Thus it is in your best interest to make sure that the PIDF is well written!
If you don't want this conversion, use the /RAW flag to UDF_OPEN.
"Sensors" can access multiple units. For instance, ImHimcpl (HENA) has a group of "sensors" that can use any of units 0,4,7,8 (raw, counts/acc, count rate, or Number Flux).
By default, UDF-DLM chooses the "most advanced" unit, that is, the last one defined for each sensor. In the example above, that would be unit 8 for each. If UDF_OPEN is invoked with the /RAW flag, though, the lowest-numbered unit is used, and (more importantly) convert_to_units() is not invoked -- you get the bits exactly as they are in the UDF file.
These two extremes don't always suffice. What if you want to see unit 7 (count rate) in ImHimcpl ?
UDF_OPEN provides a UNITS keyword for this. Its usage is a little complex, though:
For example:
IDL> fh = UDF_OPEN(key, time, UNIT=1)
...will return unit 4 for all sensors in
ImHimcpl.
Similarly, UNIT=0
would return unit 0, UNIT=2 unit 7, and
UNIT=3 unit 8.
IDL> fh = UDF_OPEN(key, time, UNITS=[0,0,0,0,0,0, 0, 0,0,0,0,0, 1,1,1,1,1])
or
IDL> fh = UDF_OPEN(key, time, UNITS=[intarr(12),intarr(5)+1])
Yes, this usage is complicated, and requires intimate knowledge of the dataset definition in the PIDF.
No, I can't really think of any instance where this would be useful.
But if you need it, it's there.
 |
|
|
Operated by the University of California
for the National Nuclear Security Administration,
of the US Department of Energy. Copyright © 2002 UC | Disclaimer/Privacy |
Last Modified $Date: 2004-08-06 18:17:42-06 $