Help for PWSMERGE2
ABBREVIATION
These abbreviations will be encountered in this PDF Help text.
BOP : Boolean OPerator
EDR : Experiment Data Record
GLL : GaLiLeo mission
IM : Instrument Mode (= 1, 2 or 3)
MOD91 : MODular 91 counts (1 count = 2/3 secs in time)
PDS : Planetary Data System
PWS : Plasma Wave System
RIM : Realtime IMage counts (1 count = 60-2/3 seconds in time)
SCLK : Spacecraft CLocK (= RIM*100 + Mod91)
SIS : Software Interface Specification
TM : Telemetry Mode (= "PB" or "RT")
TFMT : Telemtry Format ("XPW", "HCJ", "PW4" ..... etc)
UDR : Unprocessed Data Record
PURPOSE
This process merges a maximum of 20 PWS UDR files into a single UDR file,
which is further processed to generate the EDR file. As a result, PWSMERGE2
outputs both merged UDR and EDR.
REFERENCES
1. Galielo SIS 232-09, "PWS High Rate Experiment Data Record", Revision 2.0,
Issue 4, dated 4/23/90.
2. IOM:384:1989:100, "CDS Processing of PWS Data", dated 7/11/89.
3. JPL D-4683, "Standards for the Preparation & Interchange of Data Sets",
dated Oct. 3, 1988.
BACKGROUND
(A) UDR and EDR
-----------
The UDR/EDR format of the PWS data for the GLL mission has been described
in Ref. 1. Briefly a UDR file consists of 91 records, (also sometimes referred
to as 'lines' ) with each record separated by 1 MOD91 count. The entire
file, therefore, spans a time interval of 1 RIM. The first record in an UDR
must NOT be zero-filled, i.e., it must contain non-vanishing data.
The EDR is simply a more condensed representaion of an UDR. Essentially,
all 0-Filled lines in an UDR are stripped to produce an EDR. As a result, EDR
consists only of lines with 'substance', and is generally smaller in file size.
A major difference between UDR and EDR is that EDR has the PDS label built
in it, but UDR does NOT. The contents of the PDS label are usually defined
by the primary experimental group concurring with the standards described in
Ref. 3.
(B) RECORD LENGTH
-------------
The length of a record depends on TFMT as well as IM, which ranges from
670 Bytes for TFMT = MPW and IM = 1 to 64030 Bytes for TFMT = PW8 and IM =2.
(Ref. 2) Since there can be 10 different TFMT's and 3 different IM's, a total
of 30 TFMT/IM combinations are therefore generally possible. However, some
different TFMT/IM combinations yield the same record length and structure; and
this reduces the total number of concerned record structure to 10 for the
PWSMERGE2 work.
Detailed informatin within each record, i.e., # blocks etc., can be found
in Ref. 1 and will not be repeated here.
EXECUTION
There are 2 modes of execution: the Explicit mode (MODE = 1) and the Catalog
mode (MODE = 2). In the Explicit mode, the names of the UDR files to be merged
are inputted explicitly by the user; while in the Catalog mode the names of the
UDR files to be merged are obtained, based on the user-given criteria, from the
Galileo catalogue.
The command syntaxes are :
(1) Explicit Mode
-------------
VICAR> PWSMERGE2 [CATSRV=servername] [CATDB=catdatabase] CATUSR=xxx CATPW=xxxxxx MODE=1 DISK=disk DIR=dir [EXT=ex] [VER=n] [LOG=name.log] [PHASE=pha] [TARGET=tar] UDRNAME=(udr1, udr2, udr3, ....)
Here MODE = 1 signifies the EXPLICIT mode, where the user must provide
the names of UDR files, (UDRNAME = udr1, udr2, udr3 ....) to be merged.
DISK and DIR specify the output UDR and EDR location. EXT (default
= "E") and VER (default = 1) are the optional extension and version #
for the output EDR, and LOG (default='PWSMERGE2.LOG') is the optional
name of the processing LOG file. PHASE is the mission phase (default
= "VENUS_ENCOUNTER"), and TARGET (default = "VENUS") refers to the
associated plasma environment over which the PWS data is acquisited.
PRINTFLAG specfies whether or not to print out the information
obtained from the catalog. CATSERVER is the name of the server that
the catalog should access. CATDB is the name of the database that
catalog should access. CATUSR is the username for the account that
the catalog should access. CATPASSWD is the password for the username
account that the catalog should access. Note that parameters CATSRV,
CATDB, CATUSR, and CATPW are case sensitive.
The output UDR will be locating in the same DISK and DIR as EDR, and
have the same VER #. However, its file extension is fixed to be 'U'
to differentiate from the EDR.
It should be obvious that the value of UDRNAME, i.e., udr1, udr2, ....,
etc., should contain the compelte information of an UDR file including
the disk and directory name. Besides, these values must be inputted
in monotonically increasing SCLK order, i.e., the SCLK time for udr2
must NOT be earlier than that of udr1 ... etc.
(2) Catalog Mode
------------
VICAR> PWSMERGE2 [CATSRV=servername] [CATDB=catdatabase] CATUSR=xxx CATPW=xxxxxx MODE=2 DISK=disk DIR=dir [EXT=ex] [VER=n] [LOG='name.log'] [PHASE=pha] [TARGET=tar] CRITERIA=(" sclkstrtcnt between 18044700 and 18045300 and", " rawnum between 2 and 5 and ", " filepath=""/gll/pws/udr/""")
Here MODE = 2 indicates the Catalogue-mode nature of execution, where
the UDR files to be merged are determined from a set of user-inputted
CRITERIA. Each criteria specifies a range of value[s] of a field
defined in the RAW domain of the GLL-PWS Catalogue. For a complete
list of fields, refer to the "pws_catalog.h" file. PRINTFLAG specfies
whether or not to print out the information obtained from the catalog.
CATSERVER is the name of the server that the catalog should access.
CATDB is the name of the database that catalog should access. CATUSR
is the username for the account that the catalog should access.
CATPASSWD is the password for the username account that the catalog
should access. Note that parameters CATSRV, CATDB, CATUSR, and CATPW
are case sensitive.
The CRITERIA must be specifed in strict SYBASE syntax. In parti-
cular, all strings sent must be enclosed in single quotes. Also, as
shown above, the continuation from one criteria to the next is made
via the Boolean OPerator, i.e., (AND, OR etc...) which must terminate
the previous criteria. (Refer to the "EXAMPLES" section for more usage
details)
OPERATION
The program PWSMERGE2 first obtains the names of UDR files, either via
explicit inputs in MODE = 1 or via catalogue criteria in MODE = 2, that are
arranged in increasing SCLK time order. The TFMT, IM, and TM of the very
first file, i.e., earliest in SCLK time, and therefore its record structure,
are then identified and stored. All subsequent files latter in time (up to
1 RIM) that have identical TFMT, IM, and TM values are then considered for
MERGE with this earliest file.
MERGE is performed on a line-by-line basis, by sequentially considering
2 files at a time. Lines with the same RIM/Mod91 values but from 2 different
files are first compared for the presence of data. If a line from file #1
has missing data (0-filled) and the same line from file #2 has data present,
then the latter line (file #2) will be selected for the merged output file.
If both lines have data present, then SNR (Signal-Noise-Ratio) is used for
selecting the 'better' line.
When the MERGE of all files is completed, the header as well as the label
information for the merged UDR is then filled. This UDR is further processed
(0-filled lines are stripped, some fields in the header are updated ...etc)
to generate a corresponding EDR. Before both UDR and EDR are outputted, appro-
priate fields in the PWS EDR domain of the GLL catalogue are then updated.
The updated EDR fields, at present, include BEGIN_EDR, END_EDR, EDR_MISSING,
EDR_DISK, EDR_DIR, EDR_FILENAME, EDR_EXT, EDR_VERSION, EDR_STATUS, EDRUPD_DAY,
and EDRUPD_YEAR. Also, the PDS Label file is created, and is, for each EDR,
outputted as a file. (with extesnion = "PDS") Finally, a basic processing
LOG file is also generated for each PWSMERGE2 execution.
EXAMPLES
Note !! The input disk and directory in the following examples are NOT
in existence any more, and serve for the purpose of illustration
only. WPL 8/16/1990
(1) VICAR> PWSMERGE2 CATSRV=MIPSDB1 CATDB=devCat CATUSR=xxx CATPW=xxxxxx MODE=1 DISK=SCX1 DIR=WPL059 EXT='P' VER=3 UDRNAME=( UD4:[WPL059.GLL.SIMU]P0011896619.0 , UD4:[WPL059.GLL.SIMU]P0011896619.0 , UD4:[WPL059.GLL.SIMU]P0011896626.0 )
This is MODE =1 example, i.e, UDR filenames are inputted explicitly via
UDRNAME. As shown, there are 3 files located in UD4:[WPL059.GLL.SIMU]
to be merged, and the output will be directed to SCX1:[WPL059]. The
output merged UDR and EDR from the above process will respectively be:
UDR --> SCX1:[WPL059]P0011896619.U
and
EDR --> SCX1:[WPL059]P0011896619.P
Note the difference in file extension, 'U' (fixed) for UDR and user-
inputted 'P' for EDR. A PWSMERGE2.LOG file and a P0011896619.PDS
label file will also be generated in the local directory.
Also, note the Sybase Catalog server name: MIPSDB1, the Catalog
database: devCat, the Catalog user: xxx and the Catalog
password: xxxxxx.
(2) VICAR> PWSMERGE2 CATUSR=xxx CATPW=xxxxxx MODE=1 DISK=SCX1 DIR=WPL059 UDRNAME=( UD4:[WPL059.GLL.SIMU]P0011896619.0 , UD4:[WPL059.GLL.SIMU]P0011896733.0)
In this example the SCLK values of the 2 input files, (RIM=118966,
MOD91= 19) vs (RIM=118967, MOD91=33), differ by more than 1 RIM.
Since all UDR's can only contain 91 lines and span 1 RIM, there is
no time overlap between the 2 inputs files. Therefore, NO MERGE
action will be taken. However, the first file will be converted
to EDR carrying PDS label.
Also, note the Sybase Catalog server name: MIPSDB1, which is the
default, the Catalog database: devCat, which is the default, the
Catalog user: xxx and the Catalog password: xxxxxx.
(3) VICAR> PWSMERGE2 CATSRV=MIPSDB1 CATDB=devCat CATUSR=xxx CATPW=xxxxxx MODE=2 DISK=GLL3 DIR=PWS.UDR CRITERIA=("sclkstrtcnt between 11896619 and 11896626 and", "filepath=""/gll/nims/udr/""")
This example shows the usage of MODE=2. Only 4 fields in the PWS
RAW domain are accessed but they should be sufficient to illustrate
the SYBASE for CRITERIA input.
Also, note the Sybase Catalog server name: MIPSDB1, the Catalog
database: devCat, the Catalog user: xxx and the Catalog
password: xxxxxx.
(4) VICAR> PWSMERGE2 CATUSR=xxx CATPASSWD=xxxxxx MODE=2 DISK=GLL3 DIR=PWS.UDR EXT='P' VER=5 CRITERIA=("sclkstrtcnt between 11896619 and 11896626 and", "filepath=""/gll/nims/udr/""")
This example is identical to example (3), except the output EDR and
UDR will respectivly be P0011896619.P;5 and P0011896619.U;5 instead
of P0011896619.E;1 and P0011896619.U;1 in example (3).
Also, note the Sybase Catalog server name: MIPSDB1, which is the
default, the Catalog database: devCat, which is the default, the
Catalog user: xxx and the Catalog password: xxxxxx.
HISTORY
Author and Cognizant Programmer: D.D. Knight; September, 1995
Ported to Alpha: D.D. Knight February, 1995
FR Date Description
--------- -------- ------------------------------------------------
87141 9-11-95 Fixed EDR output format. The EDR header was
not being generated correctly.
87130 9-11-95 Generated test data. When this module was
originally ported there was no test data.
DFR 2-18-98 Changed from C to ANSI C compiler (GMY)
AR-100145 6-30-98 Ported to UNIX (RRP).
AR-100459 9-08-98 Corrected pds label output format (RRP).
(I had to deliver this under
pwsmerge2_label_correction package because of
some reasons. Raj. ).
10-2-98 Added LIB_NETWORK to link because of initspice (GMY
PARAMETERS:
MODE
Mode of PWSMERGE2 Execution
DISK
Disk of Output UDR File
DIR
Directory of Output UDR File
EXT
Extension of Output File Name (Optional)
VER
Version of Output File Name (Optional)
EDRCAT
Flag for EDR Catalogue Update
PDSLOG
Flag for PDS-Label LOG File
PROCLOG
Flag for PROC LOG file
LOG
Name of Processing LOG File (Optional)
UDRNAME
Names of Input UDR Files
CRITERIA
Selection Criteria for Input UDR Files
PHASE
Name of Mission Phase
TARGET
Name of TARGET
CATSRV
Optional string - The name of the catalog server
CATDB
Optional string - The name of the catalog database
CATUSR
Required string - The name of the username that the catalog should access
CATPW
Required string - The password for the username that the catalog should access
.VAR PARTITION
Optional integer - SCLK partition
.VAR DEBUG
Optional keyword
Print diagnostic messages
See Examples:
Cognizant Programmer: