Help for SSIMERGE2

PURPOSE

SSIMERGE2 will merge the data from several versions of a Galileo SSI image.
The input versions must be in Phase 2 UDR format.  See also SSIMERGE.

EXECUTION

  SSIMERGE2 OUTPATH=outputlocation         CATSRV=MIPS1 CATDB=catalog CATUSR=xxx  CATPW=xxxxxx         CRITERIA=("sclkstrtcnt between 18044700 and 18045300")         other-optional-parameters...
where
 
  OUTPATH is the disk volume name and directory (VMS) or the path name (UNIX)
      for the output merged UDRs.
      VMS:  disk:[directory]    the colon and brackets are required
      UNIX: /pathname/          the beginning and ending slashes are required
  CATSRV is the name of the Sybase server.
  CATDB is the name of the Sybase catalog.
  CATUSR is the user's Sybase user ID.
  CATPW is the user's Sybase password.
  CRITERIA is a search strings in SQL syntax.
 
CRITERIA is used to enter a catalog search string specifying the UDRs to be
merged.  Only fields in the ssiraw table may be used in the search string.
In addition, only UDRs in the correct partition (see PARTITION parameter) and
available on disk (as indicated by the filestatus field in ssiraw) are selected.

CRITERIA must be specified in SQL syntax.  Since SQL strings must be specified
in double quotes, two double quotes must be used to embed a double quote within
the VICAR parameter string:  ""/project/gll/ssi/udr/"".  Also, the continuation
of one criteria to the next is made via the Boolean symbols AND or OR as shown
in the example above.

Note that parameters CATSRV, CATDB, CATUSR, and CATPW are case sensitive.
See also parameters UPDATE/NOUPDATE.

OPERATION

SSIMERGE2 will search the ssiraw table of the SSI catalog and obtain a list of
all UDRs satisfying the specified CRITERIA.

The selected UDRs are sorted into groups where each group consists of UDRs
with the same SCLK (100*RIM+MOD91).  Each group then represents the several
versions of an image as received at different DSN stations.

SSIMERGE2 will merge all the UDRs within a given group into a single (merged)
UDR.  When data for a given image line is available from more than one version
of a UDR, the data from the first UDR (with the smallest version number) is 
selected with the exception where the decompression status in the binary prefix
is not equal to 0.  If the decompression status is not equal to 0, then if
data in the next version exists, that version's data will be selected.

The ORDER parameter may be used to specify the order of preference used in
merging data, as in the following example:

  SSIMERGE2  CRITERIA=("sclkstrtcnt=123456789 and       more criteria... and still more"),       ORDER=(3,1,2,5)       other user parameters...

The ORDER parameter specifies that only versions 1,2,3 and 5 of the image are
to be used, with version 3 being processed first and version 5 being processed 
last.  If ORDER is specified, only one output image is permitted, i.e.
sclkstrtcnt=123456789. If any of the versions specified by ORDER parameter is
missing, SSIMERGE2 will abend.

If the CRITERIA search causes a given group to consist of only one UDR,
a merged UDR is created anyway.  The data in the merged UDR will be identical
to that in the input UDR.

VICAR LABEL PROCESSING:

The VICAR label of the output merged UDR consists of an ASCII label, a binary
header (telemetry header), and a binary prefix (telemetry line record).  The
following paragraphs document how these are created.  The ASCII label and UDR
binary header are identical to that of the input version contributing the most
data to the merge except where indicated below.

The following fields of the ASCII label are computed from the corresponding
fields of the input UDRs using a weighted mean, i.e. weighted by the number
of lines used from each UDR:  COMPRESSION_RATIO, MINIMUM_COMPRESSION_RATIO,
MAXIMUM_COMPRESSION_RATIO, TBPPXL, TPPLNE.  The resulting calculation may
not be completely representative of the merged data.  

The compression ratio for ict compressed images is calculated differently.
The compression ratio is the number of raw bits divided by the number of
compressed bits.  When a UDR of an ICT compressed image is created by the
real time system, separate compression ratios are computed for the truth window
(lossless) and for the image area outside the truth window.  The mean, minumum,
and maximum values of these ratios are stored in the ssiraw table.

When several versions of an image are merged, the mean, minimum, and maximum
compression ratios for the merged image are computed from the corresponding
values of each version.  Because of image overlap, the resulting mean ratios
will in general be only close approximations.

Suppose three versions of an image are merged.  Let 
  n1,n2,n3 = number of pixels in images 1, 2, and 3 outside the truth window
  w1,w2,w3 = number of pixels in images 1, 2, and 3 inside the truth window
  r1,r2,r3 = mean ICT compression ratios for images 1, 2, and 3
  h1,h2,h3 = mean Huffman compression ratios for images 1, 2, and 3
where the ICT and Huffman compression ratios are retrieved from ssiraw.

The mean ICT and Huffman ratios for the merged image are approximated as
follows:
    r = (r1*n1 + r2*n2 + r3*n3)/(n1 + n2 + n3)
    h = (h1*w1 + h2*w2 + h3*w3)/(w1 + w2 + w3)

For the merged image, compute the following:
    n = number of pixels outside the truth window
    w = number of pixels inside the truth window

Let c = number of compressed bytes outside the truth window
    d = number of compressed bytes inside the truth window

Then c and d are approximated as follows:
    c = n/r     d = w/h

The mean ratio for the merged image is:
    t = (n + w)/(c + d)
    
The minimum and maximum ratios are the min and max of the corresponding ratios
from the three inputs.

Binary header fields First SCLK and Last SCLK are set to the corresponding
values from the first and last records of the merged UDR containing valid data.
Binary header fields First ERT and Last ERT refer to the first and last packets
received for the image.

The following fields of the binary header are recalculated based upon all
expected data in the merged UDR: Missing Lines, Partial Lines, Mean DN, and
Histogram.

The following fields of the binary header are computed from the corresponding
fields of the input UDRs using a weighted mean, i.e. weighted by the number
of lines used from each UDR: Truncated bits, Truncated pixels.

The following fields of the binary header cannot be recalculated and the output
values may consequently be inaccurate, i.e. not representative of all of the
merged data: Seq. Breaks, SFDUs.

The following binary header fields will not contain valid data since these
fields are inserted during later processing (by program GALSOS):  I/F,
Entropy Average, Entropies, Scale Factors, Slope-File, Offset-File.

Except for the SEGMENT field (see below), the binary prefix for a given line
is copied unchanged (along with the image data) from the input version
selected.  If no valid input data exists for the line, the binary prefix is
copied from the first version (the prefix should be identical for all versions).

The SEGMENT field in the binary prefix is used to determine whether data for
a given line is present in a UDR.  It is possible for one UDR to have data for
a portion of a line and for another UDR to have data for a different portion of
the same line.  In such cases, all relevant data is used to output as complete
a line as possible.  In this case, the SEGMENT field for the merged UDR is
updated to reflect the location where data is present for that line.

SSI CATALOG INTERFACE (SYBASE):

Each UDR output by the MIPS real-time system is recorded as a record in the
ssiraw table of the Galileo Catalog.  SSIMERGE2 retrieves the record from
ssiraw corresponding to each input UDR.

After the merged UDR has been created, it is catalogued by storing a record in
the ssiraw table.  Except for the following, the contents of the record are
identical to that of the input UDR contributing the most data lines to the
output merged UDR:

  mergenum  constructed from the rawnums of the input images.
  rawnum    current value is incremented by 1.
  filepath  constructed from OUTPATH parameter.
  filename  see section on UDR file naming convention below.
  producttype  = M for merged UDR, = MM for merged on-chip mosaic.
  bestversion set equal to rawnum.

The following fields are recalculated, based upon the data present in the
output merged UDR:

  tml
  maxcml
  startgapline1, startgapline2, startgapline3
  stopgapline1, stopgapline2, stopgapline3
  gaps
  averagedn
  ert
  ertmilli

The following fields are calculated from the corresponding fields of the input
UDRs using a weighted average, i.e. weighted by the number of lines used from
each UDR:  meantruncbits, meantruncsamp, meanictratio, meanhufratio,
minictratio, minhufratio, maxictratio, maxhufratio.  The resulting calculated
values may not be completely representative of the data in the merged UDR.

Note that the fields maxssnr and minssnr are not applicable for Phase 2 data.

After each merged UDR has been completed, the mergedone flag in the ssioverview
record corresponding to the given SCLK is set.

UDR FILE NAMING CONVENTION: The UDR filename is constructed from the starting
RIM and MOD91 counts.  The file extension is constructed from the rawnum
field of the ssiraw table. The file extension for merged UDRs begins with an m,
followed by the rawnum for the merged UDR:

  Ex:	s0000123456.1      Three versions of
	s0000123456.2      a UDR with starting
	s0000123456.3      RIM=1234 and MOD91=56.
	s0000123456.m4	   A merged version of the above 3 UDRs.

The filename extension for an on-chip mosaic UDR begins with MOS followed by
rawnum.  The filename extension for merged on-chip mosaics begins with an mm,
followed by rawnum:

	s0000123456.mos1
	s0000123456.mos2
	s0000123456.mm3	(a merged version of mosaics 1 and 2)


PROGRAM HISTORY

Written by: Jean Lorre, 12 February 1990
Cognizant Programmer: Jan Yoshimizu
Revisions: 
  01 APR 93 JJL updated telemetry header fields from .version to .format_id .
  06 Feb 95 REL Ported to Alpha and replaced Datatrieve interfaces with Sybase.
  07 Feb 95 REL Converted from SSIMERGE for Phase 2.
  05 May 95 REL Fixed the Sybase I/F allowing "UPDATE" & "NOUPDATE" mode. 
  20 May 96 OAM Updated to match new specification of gll_frame_struct.h
	        Deleted catsrv and catdb default values from ssimerge2.pdf.
		Fixed compression values from VICAR label to be average 
	        values. (FR 89308).
  07 Nov 96 JRY Revamped the entire program.  Rewrote it to follow the help.
		and to work on Unix.  Fixed numerous bugs.
		Changed parameters DISK and DIR to OUTPATH.
		Added parameter INPATH.
  06 Dec 96 JRY/ADC Changed MAX_SSIRAW and MAX_OVER_ROWS to 1500.  Updated
		to preserve housekeeping info in binary header regardless
		of which input contained the housekeeping.
  08 Jan 97 HBM Fixed bug with inpath/outpath.
  31 Jan 97 JRY	FR 89927:
		Fixed another bug with inpath on Unix only.
		Commented out check for same truth window per sclk.
		Zeroed out bufferz, bufferb, and newbuf.
  21 Jan 97 JRY FR 89938:
		Fixed the check and Vicar label for truth window per sclk.
		Fix MOFIBE in Vicar label and FIBE (compression flags) in
		binary label when housekeeping is missing.
  10 Mar 97 ADC FR 89944:
		Changed XLGET calls to handle situation where SSIMERGE2 isn't
		first program run after R/T (eg, if GLLMVLN has been run).
  13 Mar 97 ADC FR 87988:
		Sent housekeeping values to ssiraw for appropriate fields.
  15 Apr 97 ADC DFR:
		Corrected AM/PM for ERT stored in ssiraw.
  29 Apr 97 JRY DFR:
		Fixed bug in merge_image
  02 May 97 ADC DFR:
		Corrected AM/PM for ERT stored in ssiraw - again...
  18 Feb 98 GMY Changed from C to ANSI C compiler.
  25 Jun 98 RRP Changed pdf file to better reflect the meaning and useage
                of parameters PARTITION and ORDER. Removed tabs from imake
                file to prevent errors in make file on only SGI. Also
                redefined the decleration for file to have L_FILENAME                 L_DIRECTORY + L_EXTENSION character rather then just
                L_FILENAME extension (AR-9692).
  19 Jul 98 RRP Prevented ssimerge2 from using ert of value less then or
                equal to 0 (AR-100406). Changed criteria to be 1320 characters
                long rather then 10 strings of 132 characters (AR-100471).
                Moreover removed unused variables and update to prevent
                warning messaged. Updated tstssimerge2.pdf to include
                tstssimerge2_old.pdf and also made changes to
                tstssimerge2_old.pdf so it uses concise sql script and
                also cleaned it up.
  07 May 02 AXC Modified test sql script to remove dependency on the OP 
                database during development/testing phases. (AR-106613)

PARAMETERS: