Help for PWSMERGE2

ABBREVIATION

   These abbreviations will be encountered in this PDF Help text.

      BOP   :    Boolean OPerator
      EDR   :    Experiment Data Record
      GLL   :    GaLiLeo mission
      IM    :    Instrument Mode (= 1, 2 or 3)
      MOD91 :    MODular 91 counts (1 count = 2/3 secs in time)
      PDS   :    Planetary Data System
      PWS   :    Plasma Wave System
      RIM   :    Realtime IMage counts (1 count = 60-2/3 seconds in time)
      SCLK  :    Spacecraft CLocK (= RIM*100 + Mod91)
      SIS   :    Software Interface Specification
      TM    :    Telemetry Mode (= "PB" or "RT")
      TFMT  :    Telemtry Format ("XPW", "HCJ", "PW4" ..... etc)
      UDR   :    Unprocessed Data Record
 
PURPOSE

   This process merges a maximum of 20 PWS UDR files into a single UDR file,
which is further processed to generate the EDR file.  As a result, PWSMERGE2
outputs both merged UDR and EDR.

REFERENCES

1.  Galielo SIS 232-09, "PWS High Rate Experiment Data Record", Revision 2.0, 
    Issue 4, dated 4/23/90.

2.  IOM:384:1989:100, "CDS Processing of PWS Data", dated 7/11/89.

3.  JPL D-4683, "Standards for the Preparation & Interchange of Data Sets",
    dated Oct. 3, 1988.

BACKGROUND

(A) UDR and EDR
    -----------

    The UDR/EDR format of the PWS data for the GLL mission has been described 
in Ref. 1.  Briefly a UDR file consists of 91 records, (also sometimes referred
to as 'lines' ) with each record separated by 1 MOD91 count.  The entire 
file, therefore, spans a time interval of 1 RIM.  The first record in an UDR 
must NOT be zero-filled, i.e., it must contain non-vanishing data.  

    The EDR is simply a more condensed representaion of an UDR.  Essentially,
all 0-Filled lines in an UDR are stripped to produce an EDR.  As a result, EDR
consists only of lines with 'substance', and is generally smaller in file size. 

    A major difference between UDR and EDR is that EDR has the PDS label built
in it, but UDR does NOT.   The contents of the PDS label are usually defined 
by the primary experimental group concurring with the standards described in
Ref. 3.
 

(B) RECORD LENGTH
    -------------

    The length of a record depends on TFMT as well as IM, which ranges from
670 Bytes for TFMT = MPW and IM = 1 to 64030 Bytes for TFMT = PW8 and IM =2.
(Ref. 2)  Since there can be 10 different TFMT's and 3 different IM's, a total
of 30 TFMT/IM combinations are therefore generally possible.   However, some
different TFMT/IM combinations yield the same record length and structure; and 
this reduces the total number of concerned record structure to 10 for the 
PWSMERGE2 work.   

    Detailed informatin within each record, i.e., # blocks etc., can be found
in Ref. 1 and will not be repeated here.
  
EXECUTION

   There are 2 modes of execution: the Explicit mode (MODE = 1) and the Catalog
mode (MODE = 2).  In the Explicit mode, the names of the UDR files to be merged
are inputted explicitly by the user; while in the Catalog mode the names of the
UDR files to be merged are obtained, based on the user-given criteria, from the
Galileo catalogue.    

   The command syntaxes are :

   (1)  Explicit Mode
        -------------

        VICAR>  PWSMERGE2 [CATSRV=servername]  [CATDB=catdatabase]                             CATUSR=xxx  CATPW=xxxxxx                                             MODE=1  DISK=disk  DIR=dir                                           [EXT=ex]  [VER=n]  [LOG=name.log]                                    [PHASE=pha]  [TARGET=tar]                                            UDRNAME=(udr1, udr2, udr3, ....)      

        Here MODE = 1 signifies the EXPLICIT mode, where the user must provide
        the names of UDR files, (UDRNAME = udr1, udr2, udr3 ....) to be merged. 
        DISK and DIR specify the output UDR and EDR location.  EXT (default 
        = "E") and VER (default = 1) are the optional extension and version # 
        for the output EDR, and LOG (default='PWSMERGE2.LOG') is the optional
        name of the processing LOG file.   PHASE is the mission phase (default
        = "VENUS_ENCOUNTER"), and TARGET (default = "VENUS") refers to the
        associated plasma environment over which the PWS data is acquisited.
        PRINTFLAG specfies whether or not to print out the information 
        obtained from the catalog. CATSERVER is the name of the server that 
        the catalog should access.  CATDB is the name of the database that 
        catalog should access.  CATUSR is the username for the account that
        the catalog should access.  CATPASSWD is the password for the username
        account that the catalog should access.  Note that parameters CATSRV,
        CATDB, CATUSR, and CATPW are case sensitive.

        The output UDR will be locating in the same DISK and DIR as EDR, and
        have the same VER #.  However, its file extension is fixed to be 'U'
        to differentiate from the EDR.  

        It should be obvious that the value of UDRNAME, i.e., udr1, udr2, ...., 
        etc., should contain the compelte information of an UDR file including
        the disk and directory name.  Besides, these values must be inputted 
        in monotonically increasing SCLK order, i.e., the SCLK time for udr2 
        must NOT be earlier than that of udr1 ... etc.
  

   (2)  Catalog Mode
        ------------

        VICAR>  PWSMERGE2 [CATSRV=servername]  [CATDB=catdatabase]                               CATUSR=xxx  CATPW=xxxxxx                                               MODE=2  DISK=disk  DIR=dir                                             [EXT=ex]  [VER=n] [LOG='name.log']                                     [PHASE=pha]  [TARGET=tar]                                       CRITERIA=(" sclkstrtcnt between 18044700 and 18045300 and",                               " rawnum between 2 and 5 and ",                                           " filepath=""/gll/pws/udr/""")   

       Here MODE = 2 indicates the Catalogue-mode nature of execution, where 
       the UDR files to be merged are determined from a set of user-inputted
       CRITERIA.   Each criteria specifies a range of value[s] of a field
       defined in the RAW domain of the GLL-PWS Catalogue.  For a complete 
       list of fields, refer to the "pws_catalog.h" file.  PRINTFLAG specfies
       whether or not to print out the information obtained from the catalog. 
       CATSERVER is the name of the server that the catalog should access.  
       CATDB is the name of the database that catalog should access.  CATUSR 
       is the username for the account that the catalog should access.  
       CATPASSWD is the password for the username account that the catalog 
       should access.  Note that parameters CATSRV, CATDB, CATUSR, and CATPW 
       are case sensitive.


       The CRITERIA must be specifed in strict SYBASE syntax.  In parti-
       cular, all strings sent must be enclosed in single quotes.  Also, as
       shown above, the continuation from one criteria to the next is made 
       via the Boolean OPerator, i.e., (AND, OR etc...) which must terminate 
       the previous criteria.  (Refer to the "EXAMPLES" section for more usage 
       details)

OPERATION

   The program PWSMERGE2 first obtains the names of UDR files, either via 
explicit inputs in MODE = 1 or via catalogue criteria in MODE = 2, that are
arranged in increasing SCLK time order.  The TFMT, IM, and TM of the very
first file, i.e., earliest in SCLK time, and therefore its record structure,
are then identified and stored.  All subsequent files latter in time (up to 
1 RIM) that have identical TFMT, IM, and TM values are then considered for 
MERGE with this earliest file.  

   MERGE is performed on a line-by-line basis, by sequentially considering
2 files at a time.  Lines with the same RIM/Mod91 values but from 2 different 
files are first compared for the presence of data.  If a line from file #1 
has missing data (0-filled) and the same line from file #2 has data present, 
then the latter line (file #2) will be selected for the merged output file.  
If both lines have data present, then SNR (Signal-Noise-Ratio) is used for
selecting the 'better' line.

   When the MERGE of all files is completed, the header as well as the label
information for the merged UDR is then filled.  This UDR is further processed
(0-filled lines are stripped, some fields in the header are updated ...etc) 
to generate a corresponding EDR.  Before both UDR and EDR are outputted, appro-
priate fields in the PWS EDR domain of the GLL catalogue are then updated.
The updated EDR fields, at present, include BEGIN_EDR, END_EDR, EDR_MISSING, 
EDR_DISK, EDR_DIR, EDR_FILENAME, EDR_EXT, EDR_VERSION, EDR_STATUS, EDRUPD_DAY, 
and EDRUPD_YEAR.  Also, the PDS Label file is created, and is, for each EDR,
outputted as a file. (with extesnion = "PDS")  Finally, a basic processing 
LOG file is also generated for each PWSMERGE2 execution. 


EXAMPLES
  
  Note !!   The input disk and directory in the following examples are NOT
            in existence any more, and serve for the purpose of illustration
            only.                     WPL  8/16/1990  

  (1)  VICAR> PWSMERGE2 CATSRV=MIPSDB1  CATDB=devCat                                    CATUSR=xxx  CATPW=xxxxxx                                        MODE=1 DISK=SCX1  DIR=WPL059                                    EXT='P'  VER=3                                                  UDRNAME=( UD4:[WPL059.GLL.SIMU]P0011896619.0 ,                                  UD4:[WPL059.GLL.SIMU]P0011896619.0 ,                                  UD4:[WPL059.GLL.SIMU]P0011896626.0 )     

       This is MODE =1 example, i.e, UDR filenames are inputted explicitly via
       UDRNAME.  As shown, there are 3 files located in UD4:[WPL059.GLL.SIMU] 
       to be merged, and the output will be directed to SCX1:[WPL059].  The 
       output merged UDR and EDR from the above process will respectively be:

          UDR -->    SCX1:[WPL059]P0011896619.U

       and

          EDR -->    SCX1:[WPL059]P0011896619.P          

       Note the difference in file extension, 'U' (fixed) for UDR and user-
       inputted 'P' for EDR.   A PWSMERGE2.LOG file and a P0011896619.PDS 
       label file will also be generated in the local directory.
       Also, note the Sybase Catalog server name: MIPSDB1, the Catalog 
       database: devCat, the Catalog user: xxx and the Catalog 
       password: xxxxxx.

  (2)  VICAR> PWSMERGE2 CATUSR=xxx  CATPW=xxxxxx                                MODE=1 DISK=SCX1  DIR=WPL059                            UDRNAME=( UD4:[WPL059.GLL.SIMU]P0011896619.0 ,                                  UD4:[WPL059.GLL.SIMU]P0011896733.0)

       In this example the SCLK values of the 2 input files, (RIM=118966, 
       MOD91= 19) vs (RIM=118967, MOD91=33), differ by more than 1 RIM.  
       Since all UDR's can only contain 91 lines and span 1 RIM, there is
       no time overlap between the 2 inputs files.  Therefore, NO MERGE
       action will be taken.  However, the first file will be converted
       to EDR carrying PDS label.
       Also, note the Sybase Catalog server name: MIPSDB1, which is the
       default, the Catalog database: devCat, which is the default, the 
       Catalog user: xxx and the Catalog password: xxxxxx.


  (3)  VICAR> PWSMERGE2 CATSRV=MIPSDB1  CATDB=devCat                                             CATUSR=xxx  CATPW=xxxxxx                                                 MODE=2  DISK=GLL3  DIR=PWS.UDR                                       CRITERIA=("sclkstrtcnt between 11896619 and 11896626 and",                               "filepath=""/gll/nims/udr/""")        
                                  
       This example shows the usage of MODE=2.  Only 4 fields in the PWS 
       RAW domain are accessed but they should be sufficient to illustrate
       the SYBASE for CRITERIA input.   
       Also, note the Sybase Catalog server name: MIPSDB1, the Catalog 
       database: devCat, the Catalog user: xxx and the Catalog 
       password: xxxxxx.
       

  (4)  VICAR> PWSMERGE2 CATUSR=xxx   CATPASSWD=xxxxxx                                           MODE=2  DISK=GLL3  DIR=PWS.UDR                                            EXT='P'  VER=5                                                      CRITERIA=("sclkstrtcnt between 11896619 and 11896626 and",                               "filepath=""/gll/nims/udr/""")        
                                  
       This example is identical to example (3), except the output EDR and 
       UDR will respectivly be P0011896619.P;5 and P0011896619.U;5 instead 
       of P0011896619.E;1 and P0011896619.U;1 in example (3).
       Also, note the Sybase Catalog server name: MIPSDB1, which is the
       default, the Catalog database: devCat, which is the default, the 
       Catalog user: xxx and the Catalog password: xxxxxx.


HISTORY

   Author and Cognizant Programmer:  D.D. Knight;  September, 1995 
   Ported to Alpha: D.D. Knight February, 1995


   FR            Date        Description
   ---------     --------    ------------------------------------------------
   87141         9-11-95     Fixed EDR output format.  The EDR header was
                             not being generated correctly.

   87130         9-11-95     Generated test data.  When this module was 
                             originally ported there was no test data.

   DFR           2-18-98     Changed from C to ANSI C compiler (GMY)

   AR-100145     6-30-98     Ported to UNIX (RRP).
   AR-100459     9-08-98     Corrected pds label output format (RRP).
                             (I had to deliver this under
                              pwsmerge2_label_correction package because of
                              some reasons. Raj. ).      
                 10-2-98     Added LIB_NETWORK to link because of initspice (GMY

PARAMETERS: