 |
|
|
|
|
|
|
|||||
|
||||||
TABLE OF CONTENTS
LIST OF FIGURES vii
LIST OF TABLES viii
LIST OF ACRONYMS ix
1.0 INTRODUCTION 1
2.0 SYSTEMDESCRIPTION 3
2.1 ON-LINE SYSTEM 3
2.1.1 IMGMAP SYSTEM 3
2.1.1.1 ISOLATION OF TARGET DATA 4
2.1.1.2 CALIBRATION OF SENSOR DATA 4
2.1.1.3 DATA ADJUSTMENTS/CORRECTIONS 5
2.1.1.4 MAPPING 5
2.1.1.5 PRODUCTION OF IMAGE FILES 7
2.1.2 GRAPHICS SUBSYSTEM 7
2.1.3 VEGETATION INDEX SUBSYSTEM 8
2.1.4 COASTWATCH SUBSYSTEM 8
2.1.5 JIC ROTATIONAL DATABASE SUBSYSTEM 10
2.2 OFFLINE SYSTEM 11
2.2.1 COASTWATCH OFF-LINE SUBSYSTEM 12
2.3 DESIGN CHARACTERISTICS 12
2.3.1 KEY INTERFACES 13
2.3.1.1 SYSTEM CONTROL FILES 13
2.3.1.2 INTERMEDIATE FILE 13
2.3.1.3 LOOK-UP TABLES 13
2.3.1.4 DESIGN OF IMAGE FILE 14
3.0 SYSTEM OVERVIEW 15
3.1 IMGMAP SYSTEM 15
3.1.1 FUNCTION 15
3.1.2 SYSTEM CAPABILITIES 15
3.1.3 LIMITATIONS 28
3.2 VEGETATION INDEX SUBSYSTEM 29
3.2.1 FUNCTION 29
3.2.2 SYSTEM CAPABILITIES 29
3.2.3 LIMITATIONS 29
3.3 COASTWATCH SUBSYSTEM 30
3.3.1 FUNCTION 30
3.3.2 SYSTEM CAPABILITIES 30
3.3.3 LIMITATIONS 31
3.4 GRAPHICS SUBSYSTEM 32
3.4.1 FUNCTION 32
3.4.2 SYSTEM CAPABILITIES 32
3.4.3 LIMITATIONS 323.5
3.5 JIC ROTATIONAL DATABASE SUBSYSTEM 33
3.5.1 FUNCTION 33
3.5.2 CAPABILITIES 33
3.6 OFF-LINE SYSTEM 33
3.6.1 #BNDCHK 34
3.6.2 #BREAKUP 35
3.6.3 #HEADER 35
3.6.4 #IMGMAP 35
3.6.5 #LUKUP 36
3.6.6 #MASK 36
3.6.7 #ORBSTAT 36
3.6.8 #SETRES 36
3.6.9 #SPACE 37
3.6.10 #SYSCON 37
3.6.11 #CSTWTCH 38
4.0 SYSTEM INPUTS AND OUTPUTS 39
4.1 IMGMAP SYSTEM 39
4.1.1 INPUT DATA SETS 39
4.1.1.1 SYSTEM CONTROL FILE 39
4.1.1.2 JOINT ICE CENTER LOOK-UP TABLE 41
4.1.1.3 JOINT ICE CENTER SEASONAL
ADJUSTMENT FILE 42
4.1.1.4 LOOK-UP TABLE OF SCAN ANGLE,
SATELLITE ZENITH ANGLE, AND SECANT
OF SATELLITE ZENITH ANGLE 42
4.1.1.5 ORBIT DATA SETS IN LEVEL 1b FORMAT 43
4.1.1.6 INTERMEDIATE FILE 44
4.1.1.7 NONLINEAR LOOK-UP TABLES 44
4.1.1.8 GRAPHICS FILE 45
4.1.2 IMGMAP OUTPUT DATA SETS 45
4.1.2.1 INTERMEDIATE FILE 45
4.1.2.2 IMAGE FILES 45
4.1.2.2.1 GENERIC DATA RECORD FORMAT
FOR IMAGE PRODUCTS 45
4.1.2.2.2 JOINT ICE CENTER FILES 46
4.1.2.2.3 MASTER MAP FILES 46
4.1.2.2.4 OCEAN PRODUCTS FILES 47
4.2 GRAPHICS SUBSYSTEM 48
4.2.1 INPUT DATA SETS 48
4.2.1.1 JCL CARD INPUT 48
4.2.1.2 WORLD DATA BANK RANK FILE 48
4.2.1.3 LAND-SEA TAG FILE 49
4.2.1.4 WORLD DATA BANK II GEOGRAPHY FILES 49
4.2.2 OUTPUT DATA SETS 50
4.2.2.1 GRAPHICS FILE 50
4.3 VEGETATION INDEX SUBSYSTEM 50
4.3.1 VEGETATION INDEX PROCESSOR (VEGINDX) 51
4.3.1.1 VEGINDX INPUT DATA SETS 51
4.3.1.1.1 VEGETATION INDEX SUBSYSTEM
SUBSYSTEM CONTROL FILE 51
4.3.1.1.2 MAPPED IMAGE FILES 52
4.3.1.2 VEGINDX OUTPUT DATA SETS 52
4.3.1.2.1 WEEKLY COMPOSITE FILES 52
4.3.2 WEEKLY COMPOSITE PROCESSOR (WKLYCOMP) 53
4.3.2.1 WKLYCOMP INPUT DATA SETS 53
4.3.2.1.1 VEGETATION INDEX SUBSYSTEM
CONTROL FILE 53
4.3.2.1.2 WEEKLY COMPOSITE FILES 53
4.3.2.2 WKLYCOMP OUTPUT DATA SETS 53
4.3.2.2.1 VEGETATION INDEX FILES 53
4.4 COASTWATCH SUSBSYTEM 54
4.4.1 INPUT DATA SETS 54
4.4.1.1 OCNMAP CONTROL FILES 54
4.4.1.2 IMGMAP OUTPUT IMAGE FILE 54
4.4.1.3 SST FIELD FILES 55
4.4.2 OCNMAP OUTPUT DATA SETS 55
4.4.2.1 THE TRANSFER FILE 55
4.5 JIC ROTATIONAL DATABASE SUBSYSTEM 55
4.5.1 GAC ORBIT DATABASE 56
4.5.1.1 GAC INPUT DATA SETS 56
4.5.1.1.1 GAC ORBIT NAME FILE 56
4.5.1.1.2 GAC STATUS FILE 56
4.5.1.1.3 GAC DATA FILE 57
4.5.1.1.4 GAC HOLD FILES 57
4.5.1.2 GAC OUTPUT DATA SETS 57
4.5.1.2.1 GAC STATUS FILE 57
4.5.2 LAC ORBIT DATABASE 58
4.5.2.1 LAC INPUT DATA SETS 58
4.5.2.1.1 LAC ORBIT NAME FILE 58
4.5.2.1.2 LAC STATUS FILE 58
4.5.2.1.3 LAC DATA FILE 59
4.5.2.1.4 LAC HOLD FILE 59
4.5.2.2 LAC OUTPUT DATA SETS 59
4.5.2.2.1 LAC STATUS FILE 59
4.5.3 HRPT ORBIT DATABASE 59
4.5.3.1 HRPT INPUT DATA SETS 60
4.5.3.1.1 HRPT ORBIT NAME FILE 60
4.5.3.1.2 HRPT STATUS FILE 60
4.5.3.1.3 HRPT DATA FILE 60
4.5.3.1.4 HRPT HOLD FILE 61
4.5.3.2 HRPT OUTPUT DATA SETS 61
4.5.3.2.1 HRPT STATUS FILE 61
5.0 OPERATIONAL SCENARIO 62
5.1 SCHEDULING OF JOBS 62
5.2 JOB INITIATION AND EXECUTION 62
5.3 JOB STRUCTURE 63
5.3.1 OCEAN PRODUCTS 63
5.3.1.1 OPERATIONAL SCHEDULE 63
5.3.2 MASTER MAPS 63
5.3.2.1 OPERATIONAL SCHEDULE 64
5.3.3 JIC PRODUCTS 64
5.3.3.1 OPERATIONAL SCHEDULE 65
5.3.4 COASTWATCH 65
5.3.4.1 PRELIMINARY STEPS 65
5.3.4.2 COASTWATCH REGION PROCESSING 66
5.3.4.3 OPERATIONAL SCHEDULE . 67
6.0 RESOURCE REQUIREMENTS 68
6.1 STORAGE REQUIREMENTS 68
6.1.1 OCEAN PRODUCTS 68
6.1.1.1 COMPUTING STORAGE REQUIREMENTS
FOR A COASTWATCH REGION 69
6.1.2 MASTER MAPS 71
6.1.3 JOINT ICE CENTER PRODUCTS 72
6.1.4 OTHER REQUIREMENTS 73
6.2 COMPUTER RESOURCES 73
6.2.1 DPSS MAINFRAME COMPUTERS 73
6.2.2 NAS MAINFRAME COMPUTERS 73
6.2.3 IBM PC WORKSTATION 74
6.2.4 OPC OPERATIONAL WORKSTATION 74
6.2.5 DIFAS WORKSTATION 75
6.3 INTERCOMPUTER COMMUNICATIONS 75
6.3.1 DPSS AND NAS 75
6.3.2 DPSS AND DIFAS 75
6.3.3 DPSS AND PC WORKSTATIONS 76
6.3.4 DPSS AND OPC OPERATIONAL SYSTEM 76
6.3.5 NAS AND DIFAS 76
6.3.6 NAS AND PC WORKSTATIONS 766.4
IMPACT OF THE IMAGE PRODUCT SYSTEM
ON COMPUTER RESOURCES 76
6.5 PERFORMANCE STATISTICS 77
6.5.1 COASTWATCH PERFORMANCE STATISTICS 77
7.0 MAINTENANCE AND MONITORING 79
7.1 ENVIRONMENT 79
7.2 SCIENCE MAINTENANCE 79
7.3 DATA SIGNAL MONITORING 79
7.4 JOB NETWORK MONITORING 79
7.5 OFF-LINE MONITORING 79
7.6 IMAGE DISPLAY 80
7.7 LIBRARY MAINTENANCE 80
7.8 SPECIAL MAINTENANCE PROCEDURES 80
7.8.1 NEW SATELLITE IMPLEMENTATION 80
7.8.2 ADDING A NEW JOB 81
7.8.3 DATA RECOVERY 81
7.8.4 MODIFYING EXISTING PROGRAMS 81
REFERENCES 82
LIST OF FIGURES
SYSTEM OVERVIEW CHARTS [Figures 1.0-1.5]
Figure 1.0Image Product System Configuration 16
Figure 1.1JIC Rotational Data Base Subsystem 17
Figure 1.2Vegetation Index Subsystem 18
Figure 1.3Coastwatch Subsystem 19
Figure 1.4Graphics Subsystem 20
Figure 1.5Snow/Ice/Precip Subsystems 21
HIGH-LEVEL DATA FLOW DIAGRAMS [Figures 2.0-2.4]
Figure 2.0Image Mapping System (IMGMAP) 22
Figure 2.1JIC Rotational Data Base Subsystem 23
Figure 2.2Vegetation Index Subsystem 24
Figure 2.3Coastwatch Subsystem 25
Figure 2.4Graphics Subsystem 26
Figure 3.0Image Product System Data Flow Diagram 40
LIST OF TABLES
Table 1.0Job Performance Statistics 78
LIST OF ACRONYMS
AMSU Advanced Microwave Sounding Unit
AVHRR Advanced Very High Resolution Radiometer
BDY International Boundaries
BLKSIZE Block Size
BT Brightness Temperature
CIA Central Intelligence Agency
CCF Central Computer Facility
CIL Coastlines, Islands, and Lakes
CLIST Command List
CPSST Cross-product SST
CPU Central Processing Unit
CTC Channel-to-Channel
DACS Data Acquisition and Control Subsystem
DCB Data Control Block
DCDTM DPSS to CCF Data Transfer Monitor
DIFAS Digital Ice Forecast and Analysis system
DPSS Data Processing Services Subsystem
DSN Data Set Name
EBCDICExtended Binary-Coded-Decimal Interchange Code (used for text files)
F Fixed-length Records
FB Fixed-length Blocked Records
FIFO First In First Out
FOC Final Operational Capability
FOV Field of View
FTP File Transfer Program
GAC Global Area Coverage
GHRR GAC AVHRR data 1B data set
GOES Geostationary Operational Environmental Satellite
GORBSTAT GAC Orbit Statistics Processor
HORBSTAT HRPT Orbit Statistics Processor
HRPT High Resolution Picture Transmission
I Image Pixel
I/O Input/Output
IBM International Business Machines Corporation
ID Identification
IDIDASInteractive Digital Image Display and Analysis System
ILABS Image Library and Browse System
IMGMAP Image Mapping System Processor
IOFFOffset Value -- Grid Coordinate of Image Upper Left Corner I
IP Grid Coordinate Corresponding to I
IR Infrared
J Image Row
JCL Job Control Language
JIC Joint Ice Center
JOFFOffset Value -- Grid Coordinate of Image Upper Left Corner J
JP Grid Coordinate Corresponding to J
LAC Local Area Coverage
LHRR LAC AVHRR data 1B data set
LORBSTAT LAC Orbit Statistics Processor
LRECL Logical Record Length
MCSST Multi-channel Sea Surface Temperature
METSAT Meteorological Satellite
MVS Multiple Virtual Storage
MVS/XA MVS/Extended Architecture
NAS National Advanced Systems
NCCF NOAA Central Computing Facility
NESDISNational Environmental Satellite Data and Information Service
NH Northern Hemisphere, NOAA-H
NLSST Nonlinear SST
NOAA National Oceanic and Atmospheric Administration
NREC Number of Records
NWS National Weather Service
OCNMAP Ocean Map
OPC Ocean Product Center
OR Ocean Reflectance
ORBSTAT Orbit Statistics Processor
OSDPD Office of Satellite Data Processing and Distribution
PBY Provincial Internal Boundaries
PC Personal Computer
PL Prime Longitude
RECFM Record Format
R & D Research and Development
RIV Rivers
SH Southern Hemisphere
SPF System Productivity Facility
SSM/I Special Sensor Microwave/Imager
SST Sea Surface Temperature
TBD To Be Decided
TIP TIROS Information Processor
TIROS Television and Infrared Observation System
VEGINDX Vegetation Index Processor
VI Vegetation Index
VS Variable-length Records Spanned Over More Than One Block
WDB World Data Bank
WKLYCOMP Weekly Composite Processor
1.0 INTRODUCTION
The image product system resides on the NOAA Central Computing Facility (NCCF) National Advanced Systems (NAS) mainframe computers. A subset of the system that generates products for the Joint Ice Center is operational on the NESDIS Data Processing Services Subsystem (DPSS) computers. The DPSS version will be upgraded when the final operation capability is implemented. The Image Product System software is written completely in VS FORTRAN. All input and output interfaces are implemented using standard FORTRAN I/O routines thus ensuring portability to other computers. The software meets the OSDPD software standards and guidelines except as follows:
(1) We use RETURN 1 in many of our routines to return control to a dedicated section of the calling routine in the event a very serious error occurs in processing. The use of this construct is very limited in scope and only occurs in the context of error handling. It is intended to make the code more readable and easier to understand.
(2) Through out OCNMAP we use special purpose routines called $-routines to do customized error handling. These routines all expect an integer code to identify the error as the first calling argument. Invariably this code is passed as an integer constant.
(3) We frequently left justify statement labels to separate them as much as possible from the code whose structure (including the end points of loops and other constructs) is made evident through the use of indentation. Right justification sometimes interferes with the information contained by the indentation pattern.
(4) We use multiple entry points in one subprogram, IBMERR, which sets up user supplied exit routines for use with IBM's extended error handling. The entry points correspond to the possible I/O errors that can occur. It would be possible to disperse these related procedures in a large number of separate subroutines; but, since these calls are never made by OCNMAP software but only by the extended error handling system of IBM, doing so would confuse, rather than clarify the issue.
(5) A large number of OCNMAP's subroutines do error checking. Checks are made after every I/O statement and after most calls to lower level subroutines. Generally if an error is detected, we process it with a call to a $-routine, set an error code, and then RETURN to the calling routine directly as opposed to using a GO TO in order to exit through a single RETURN statement at the bottom of the routine. All these exits are clearly documented and inserting extraneous GO TO statements will serve no useful purpose.
This document provides a system level description of the Image Product System. Section 2 provides a description of the components that make up the system. System capabilities and limitations are discussed under system overview in Section 3. Section 4 provides a brief description of system inputs and outputs. Operational scenarios and resource requirements are discussed in Sections 5 and 6. Section 7 summarizes maintenance and monitoring capabilities.
The current configuration of the Image Product System consists of the Image Mapping system (IMGMAP) as the main system, and Graphics, Vegetation Index (VI),CoastWatch, and JIC Rotational database as subsystems. The subsystems either supply the necessary input data to the main system or make use of the data produced by the main system to generate derived products. The Image Product system has both an on-line and an off-line system. The on-line system is responsible for the production of operational products. The off-line system functions as a support system to meet non-operational needs such as Research and Development (R&D) activities, to serve as a backup system in the event of on-line system failures, and to serve as an off-line maintenance and monitoring system. A description of the on-line and off-line systems follows.
The IMGMAP system consists of IMGMAP as the main driver. Using the AVHRR 1b data as input, the system produces three major environmental image products: Ice maps for the NAVY/NOAA Joint Ice Center, Master Maps for derivation of Vegetation Index , snow cover, and other climatological products, and Ocean Maps for CoastWatch and the Ocean Products Center (OPC). The system also produces ancillary maps such as solar zenith angle, satellite zenith angle, scan angle, relative azimuth angle, and scan time. The ancillary data are used in data corrections and in multichannel algorithms. The system provides a number of processing options. The products generated by the system can be tailor-made to suit the application of the end user by activating appropriate options. These include options for satellite selection (morning or afternoon), data type (GAC, LAC, HRPT), selection of channels and ancillary maps, data corrections, type of calibration, graphics, type of map projection, map resolution, image pixel length, compositing with a previously generated map, and fill-up of unoccupied pixels. These options are set up interactively through a system control file.
Functionally, the system can be broken up into five major processes:
1. Isolation of target area (from 1b data set)
2. Calibration of sensor data
3. Data adjustments/corrections
4. Mapping
5. Production of image files
2.1.1.1 ISOLATION OF TARGET AREA
A target area is a geographical area of interest selected for processing and is usually defined by specifying the beginning and ending lat-longs. Taking into account that the polar orbits are sun synchronous, the system initially makes a gross level check to see if a given orbit covers the target area selected for processing. The equator crossing times of the satellite in question are taken into consideration to make this determination. Since the 1b orbit data set is also ordered by latitudes, a conclusive check is made by applying the binary search algorithm to locate the target latitudes. This is followed by a check for target longitudes (only for ocean maps) to complete the search. Longitude check is not necessary for JIC maps or master maps as the JIC product requires the entire coverage over the polar caps (36N-90N,50S-90S. The lower latitude limit is adjusted seasonally and retrieved from a seasonal adjustment file as a function of month) and master maps require global coverage. However, for both JIC product and master maps, the processed data need to be broken up based on northern or southern hemisphere and daytime or nighttime coverage. To facilitate this, the system breaks up the 1b orbit data set into four segments with each segment corresponding to daytime or nighttime coverage over northern or southern hemisphere. Each of these segments is processed independently. No processing gets done if the system is unable to locate the target area from 1b data set.
2.1.1.2 CALIBRATION OF SENSOR DATA
When the AVHRR data are ingested into a 1b format, the sensor data are stored into a number of scan lines. Each scan line contains a number of spots (also known as Field Of Views), 409 for GAC and 2048 for LAC/HRPT, and each spot contains observations from each of the five spectral channels. Each scan line contains two calibration coefficients, the slope and intercept, for each channel. Once scan lines over the target area are determined, the system invokes a procedure to unpack and calibrate channel data at each spot from each scan line. When the data are unpacked, each channel produces a 10-bit data value which is known as raw count. Spectral radiances are obtained by applying slope and intercept on raw counts. Albedos (from visible and reflected IR, channel 1 and 2), and brightness temperatures (from thermal channels, channels 3, 4, and 5) are derived from radiances. The system provides an option to output data at any stage of the processing in units of raw counts, radiances, and albedos and brightness temperatures. Calibration process is carried out only for selected channels.
Currently, for JIC products, the channel data are calibrated to albedos and brightness temperatures and output as 1-byte pixels; for master maps, raw counts are truncated to 8-bits from visible channels, and brightness temperatures computed from thermal channels are scaled to 8-bit pixels using the GOES look-up table; and for ocean products, channel data will be calibrated to albedos and brightness temperatures and output as 1-byte or 2- byte pixels.
2.1.1.3 DATA ADJUSTMENTS/CORRECTIONS
Separate scaling schemes are available to output channel data to fit into a 1- byte or 2-byte image pixel. Two byte image pixels retain the full radiometric precision of the AVHRR instrument. The precision can also be retained with 1-byte image pixels but only for a limited dynamic range. Prior to scaling, the channel data are optionally corrected for limb, non-linearity, and Sun angle. Both limb correction and non-linearity correction are additive corrections to brightness temperatures and are applied over a limited range of temperatures. The limb correction corrects brightness temperatures for atmospheric attenuation; and the non-linearity correction corrects temperatures derived from channel 4 and 5 for non-linearity of calibration. The limb correction terms are computed based on satellite zenith angle. The non-linearity correction terms are retrieved from a look-up table that is specific to a satellite. The Sun normalization corrects visible counts or albedos for Sun angle (see Section 6.7 of Ref. 4).
Currently, no corrections are applied for JIC maps and master maps. The non-linearity correction will be applied to ocean maps.
2.1.1.4 MAPPING
The mapping process involves converting earth locations ( the lat-long points) to image coordinates [ the (I,J) coordinates ] using standard projections such as polar stereograph, mercator, and linear lat-long, and transferring data at each earth location to its corresponding image coordinate. Since only 51 spots of an AVHRR scan have earth locations in the 1b data, image coordinates for the missing spots are derived by interpolating the known ones. A parameter called resolution comes into play in converting earth locations to image coordinates. Resolution is defined as distance between two consecutive pixels on a mapped image. For polar projection, the resolution is defined at 60-degree latitude and for mercator projection it is defined at the equator. Resolution is measured in units of Kilometers for both polar and mercator projections and in units of degrees/pixel for linear lat-long projection. With unmapped images, no conversion is made from lat-longs to image coordinates. Instead, each scan line is made into an image row and each spot on the scan, or Field Of View (FOV), is made into an image column. Data at each FOV is then transferred to its corresponding image coordinate.
Once the image coordinates are determined and channel data at each image coordinate are identified, the system computes selected ancillary data. The scan angle and satellite zenith angle are directly retrieved from a look-up table as a function of spot number. Solar zenith angle and relative azimuth angle are computed at each of the 51 spots of each scan where the earth locations are available and interpolated to cover the entire scan. The scan time is directly retrieved from each scan line and the same time is used for all image coordinates derived from the same scan line. Mapped (I,J) values, selected channel data, and ancillary data at each image coordinate are stored to an intermediate file to complete the mapping process.
Prior to mapping, for JIC target areas, an appropriate prime longitude is determined based on orbit swath. Mapping for JIC is currently done in polar stereograph and the prime longitude used is one of four longitudes designated for JIC target areas, 80W, 170W, 10E, or 100E. The polar grid size used is 1/128th mesh for LAC/HRPT, and 1/64th mesh for GAC which corresponds to a map resolution of 2.977 Km and 5.953 Km respectively. The images are produced in 2048X2048 for LAC/HRPT and in 1024X1024 for GAC.
Master maps are also produced in polar stereograph with the prime longitude fixed at 80W. The polar grid size used is 1/16th mesh which corresponds to a map resolution of 23.812 Km. The images are produced in 1024X1024. No master maps are produced from LAC/HRPT.
Ocean maps use both GAC and LAC/HRPT data types for processing. For operational use, the mapping is done mostly in mercator projection. There are five operational target areas considered for processing. The image sizes for these areas are given in Section 6.1.1.
2.1.1.5 PRODUCTION OF IMAGE FILES
Once the intermediate file is stacked up with the data processed over the target area, the system invokes a procedure to separate out the data and disburse it to individual image files. Before the data are transferred to individual image files, three major processes take place. If the composite option is activated, data from previously processed images are brought into program memory. Based on the criterion defined for composite, at each image pixel, either the latest pixel value or the previous pixel value or an average of the two will be retained. Holes (unoccupied pixels) in each image are filled using the average or adjacent fill-up method (see Section 6.9 of Ref. 4). If the graphics option is activated, a static graphics file is brought into memory and graphics are overlaid over channel images. Due to limitations on run-time core and the image sizes being too large to handle, only a portion of the image is loaded into program memory each time.
Currently, no graphics are produced for JIC maps and master maps. Both Fill-up options are used for all products. No composite option is used for JIC maps; master maps are composited based on minimum nadir angle; and ocean maps will most likely use one of five composite options: no composite, composite based on minimum nadir angle, average pixel, later pixel, warmer pixel, or colder pixel.
2.1.2 GRAPHICS SUBSYSTEM
The graphics subsystem consists of IMGGRP as the main driver. The system provides the necessary geographic reference to identify the target area. The graphics produced are lat-long grids, coastlines, and land mask. To produce coastlines the system retrieves geographical features from the World Data Bank II geography database. The World Data Bank has features organized by continents and ranks. The level of detail required for geography is controlled by ranks. The graphics file is also known as static file as the graphics produced remain unchanged once the target area and the mapping parameters are fixed. Since the contents of the file are static, the file need not be produced time and again in an operational environment. Taking this into account, the graphics processing is detached from IMGMAP and made into a subsystem. Another justification for this is that the retrieval of geography from World Data Bank II is a CPU intensive process. Separating the graphics system from the main processing will help reduce the CPU time and avoid redundant processing. The graphics system can generate lattitude/longitude grids, coastlines obtained from the World Data Bank II, and a land mask (derived from a land/sea tag file on a 1/16th degree resolution). The graphics subsystem runs off-line and uses the same target area and mapping options that are used by the IMGMAP system to ensure proper matching of graphics with image. The default grid interval used for lat-long grid lines is 1-degree or 5-degrees depending on the map resolution. The pixel size used for graphics is 1-byte. The least significant bit is unused and reserved for flagging filled-up spots. IMGMAP optionally flags this bit while merging the graphics produced by the graphics system. The higher three bits are used for lat-long grids, coastlines, and land mask.
2.1.3 VEGETATION INDEX SUBSYSTEM
The Vegetation Index subsystem consists of two load modules, VEGINDX, and WKLYCOMP. The daytime master maps produced by the IMGMAP system are routed to the Vegetation Index subsystem at the end of each day for a weekly composite. Once the composite period has elapsed, the Vegetation Index subsystem computes the Vegetation Index from weekly composite files and produces a Vegetation Index file. VEGINDX and WKLYCOMP modules will be executed manually when the system goes operational. VEGINDX is responsible for the daily composite. The module loads previously composited master maps and the daily master maps into program memory. The green values are computed at each pixel from both sets and the greener of the two is taken as a criterion to replace the composite pixel with the daily pixel or to retain the composite pixel as it is. When the composite pixel is replaced, all channel and ancillary pixels are also replaced simultaneously to ensure that the pixels are coincident in time. The difference of channel 2 and channel 1 pixel value determines the green value. The load module WKLYCOMP is invoked at the end of the composite period to produce a Vegetation Index file. The processing options for the Vegetation Index subsystem are controlled through a Vegetation Index subsystem control file. Since the operational schedule of the Vegetation Index subsystem is different from the schedule of the IMGMAP system, the Vegetation Index system is detached from IMGMAP and made into a subsystem.
2.1.4 COASTWATCH SUBSYSTEM
The CoastWatch subsystem is implemented operationally by the program OCNMAP, which accesses image files of user-defined regions that are produced by the IMGMAP system. OCNMAP breaks up these regions into sectors for which it then produces a variety of images. The actions that OCNMAP takes during a run are, to a high degree, user-specifiable. A user communicates with the program through a file in which a large number of program controls, including those that define sectors and their images, can be specified. These control files are quite complex and, in order to simplify their construction, an IBM System Productivity Facility (SPF)-panel driven utility is provided. In addition to the many panels that elicit information from the user, the subsystem also provides an extensive set of help panels that explain the various options available to the user, and a suite of routines that check for consistency among the many user-supplied controls and between what OCNMAP expects and what IMGMAP has or is about to produce.
CoastWatch sectors are generally subsets of the larger regions imaged by IMGMAP, but they can also contain the region or lie partially outside of it. The resolution (i.e., the distance between pixels) of a sector is the same or -- in the case of subsampling -- is less than that of the parent region.
CoastWatch can produce sector images of channel data and ancillary data that are essentially the same as those produced by IMGMAP. The channel data files can be compressed and can have graphics data merged with them. Ancillary data can neither be compressed nor have merged graphics.
The old function of the MCSST subsystem of IMGMAP has been assumed and extended by OCNMAP which can produce images of CPSST and NLSST as well as MCSST. These images can be optionally compressed and can have graphics data merged with them. Also the capability for producing cloud masks that used to be provided by the MCSST subsystem has been improved and extended in OCNMAP. New tests have been added, and the processing algorithms that provide pixel data to the routines have been improved.
A capability for computing Ocean Reflectance (OR) has been provided. OR images can be compressed and have graphics information added.
OCNMAP can read region graphics files produced by the Graphics Subsystem and use the data to create stand-alone sector graphics files. It can also append this graphics information to any sector images of channel, sea-surface temperature, and ocean reflectance data that have been compressed. Once a compressed stand-alone graphics file for a sector has been created it can be used as the source of graphics information for any future, compressed sector images of channel, sea-surface temperature, or ocean reflectance.
A brief description of OCNMAP processing follows:
OCNMAP first reads a control file and sets up a common block (referred to as the control structure) that the program uses to control its processing.
Next it reads the region file and extracts small "segments" from each of the channel and ancillary images stored therein. These data, which are stored in the segment buffer, are then processed image-row by image-row sequentially by a group of image processors.
There are image processors for each of the various images OCNMAP can produce -- channels, ancillaries, SSTs, OR, cloud masks & graphics. The image processors use the control structure to determine what processing has been requested by the user. Each image processor checks each of the defined sectors to see if the data currently in the segment buffer applies. If it does, the processing is carried out. If not, the next sector is checked.
As each row of image data in the segment buffer is processed, pointers into the buffer are adjusted. When they indicate that there is sufficient room in the buffer to hold a complete physical record of channel or ancillary data, additional data is read from the region file and control is returned to the top of the processing loop. The program continues in this fashion until all the pertinent data have been exhausted.
As data are created for each of the rows in a sector image, they are passed to the output module which either outputs them directly or causes them to be compressed and stored in a buffer until enough data for a full record has accumulated. At that point, the I/O is performed.
This process continues until all the data in the region file has been read and processed.
The JIC rotational database consists of three modules, GORBSTAT, LORBSTAT, and HORBSTAT. Currently, the production of JIC maps is operational on the DPSS mainframe computers. As there is no communication link between the DPSS computers and the Digital Ice Forecast and Analysis System (DIFAS) workstation, the images produced on the DPSS computers are file transferred to the NAS mainframe computers, and from the NAS they are transferred to the DIFAS workstation via Ethernet. Since the JIC does not have a real-time need for the availability of maps, the JIC maps are stacked up to a rotational database that resides on the NAS environment. The JIC user queries the database through a status file and transfers maps of interest to the workstation. Up to eight orbits each of GAC, LAC, and HRPT are stored to the rotational database. The database files are revolved using First In First Out (FIFO) method. Each GAC orbit produces four files with each one corresponding to northern or southern hemisphere and daytime or nighttime, resulting into 32 database entries. Each LAC or HRPT orbit produces only one file, resulting into 8 database entries.
The Image Product Off-line system provides a set of interactive utilities to submit jobs from foreground, and to rapidly check the quality of processed images. The off-line system also provides a set of information and file management utilities. The following are some of the salient features offered by the off-line system:
1. Through the off-line system, the user can create/update system control files, select orbits of interest, and submit jobs from foreground.
2. Image files produced by the image product system can be broken up into individual images for further processing.
3. Elementary quality checks can be performed on processed images. These include displaying pixel values from user-specified locations, checking for pixel bounds, cloud percentage, etc.
4. Image header information, and history of composite can be displayed.
5. Amount of disk space required for image files can be estimated.
6. Look-up tables for 11-bit to 8-bit conversion can be created/updated.
7. Given the image size, beginning and ending lat-longs, and type of projection, mapped resolution can be computed.
or
Given the image size, central lat-long, mapped resolution, and type of projection, beginning and ending lat-longs can be computed.
8. An Ocean Map control file can be created or modified.
Each utility is driven by a Command list (Clist) or a set of Command lists. The utilities are organized into a panel. The off-line system can be initiated with a start up command:
EX 'NSS.PSASAIMP.IMAGV.CLIST(#S)' 'T'
after logon and
#PANEL
The control structure of OCNMAP is of necessity quite complex. A user has a large number of options from which to choose and a high degree of control over the various algorithms used to create the sector images. In order to aid a user to create a control file that will in turn cause OCNMAP to perform the desired computations, an interactive, panel-driven utility has been created that will solicit information from the user, check it for consistency -- not only with other of the user's specifications, but also with what is known about the processing that IMGMAP will carry out during its production of the region file.
The Image Product System features a generic system that is easily maintainable and expandable, and an optimized code that takes less CPU and I/O time. The system is designed in such a way that no software changes are necessary in the wake of changing satellites, algorithms, and processing conditions. Two design aspects that are given the highest priority are the top down design and modular approach.
IMGMAP system, the heart of the Image Product System, offers a variety of mapping options. The features offered by the system are not just limited to the imaging requirements of AVHRR and AMSU. Image products from other sensors, and from any kind of earth located data can be generated effortlessly and economically by making a few peripheral changes to the system. This concept is illustrated in the overview charts presented in Section 3. It is expected that the IMGMAP system will not only satisfy research and operational needs of the NOAA/NESDIS but also serve as a 'Consolidated Mapping System'. Efforts are already underway to generate SSM/I master maps from IMGMAP.
This section identifies and describes the role of some key interfaces in the overall design of the Image Product system
IMGMAP and its subsystems with the exception of JIC Rotational Database Subsystem are controlled externally by their individual System Control Files. The purpose of control files is many fold, to provide a powerful interface to the processing system, to provide a user-friendly interface to the user, and to facilitate a data driven mode of operation. The control files are designed in such a way that changing operational requirements, processing conditions, product attributes, and algorithms will not force software changes. The only changes that are required under these events are changes to the contents of these files. The contents themselves are in a readily readable text format. To make it even simpler, each data record has a literal description associated with it indicating what it stands for. Through these files, the user can exploit to the fullest extent the capabilities offered by the system.
The Intermediate File is a temporary file that gets allocated at the submission of the IMGMAP job and deallocated at the completion of the job. The file grows dynamically as more space becomes necessary. The file stores mapped image coordinates and the corresponding channel and ancillary data selected for processing. At the end of the processing the contents of the file are transferred to individual image files. This design has greatly optimized the run-time memory requirements of IMGMAP and awarded the system a unique capability of producing images of virtually any size.
2.3.1.3 LOOK-UP TABLES
Wherever possible, the IMGMAP system uses look-up tables to help reduce the CPU time and to avoid redundant computations. The system uses look- up tables for 11-bit to 8-bit conversion, for scan angle and satellite zenith angles, and for non-linearity correction factors.
2.3.1.4 DESIGN OF IMAGE FILE
Images of selected channels and ancillary data are stored one after the other to a single file. This design has simplified data set maintenance and monitoring efforts and eliminated the need of changing processing JCLs each time an additional channel or ancillary image is added or dropped.
This section presents the capabilities and limitations of the Image Product System. System overview charts presented in Figures 1 through 1.5 identify the primary source of input, the processing system, and the product generated by each system. Data flow diagrams given in Figures 2 through 2.4 describe the flow of data and identify major input files, the processing system, and major output files produced by the Image Product System. Hierarchical relationship of IMGMAP system with its subsystems is also evident from these figures. Each component of the Image Product System has a system control file; however, the JIC rotational data base subsystem is an exception to this. Through the system control file, the user chooses the attributes of the image product desired.
3.1.1 FUNCTION
The IMGMAP system produces three major environmental image products using sensor data processed from the AVHRR and (future) AMSU instruments.
3.1.2 SYSTEM CAPABILITIES
Various capabilities offered by the IMGMAP system are discussed under the following list of categories:
Satellites:
Morning or afternoon or both satellites can be selected for processing.
Products:
IMGMAP system produces the Joint Ice Center product, Master Maps, and Ocean products.
Instruments/data types:
Data from all data types of AVHRR and (future) AMSU can be processed.
Day/night option:
Selected set of images can be produced either from a daytime or nighttime overpass or from both.
Figure 1.0 Figure 1.1 Figure 1.2 Figure 1.3 Figure 1.4 Figure 1.5 Figure 2.0 Figure 2.1 Figure 2.2 Figure 2.3 Figure 2.4Channels:
One or more channels can be selected in any combination.
Ancillary information:
Images of selected ancillary data can be produced. These are: scan angle, satellite zenith angle, solar zenith angle, relative azimuth angle, and scan time.
Data corrections:
Computed albedos can be optionally corrected for sun angle. Likewise, limb correction or nonlinearity calibration correction can be optionally applied to computed brightness temperatures.
Calibration options:
Four calibration options are available: raw counts, radiances, albedos and brightness temperatures, and albedos and brightness temperatures using GOES look-up table. Each selected channel can be calibrated independently using any one of these four options.
Graphics options:
Graphics can be selected from lat-long grids, coastlines, land-sea tags, and filled-up spots.
Projection:
Four different types of projections are available: Unmapped, Mercator, Polar, and Linear lat-long.
Target area:
Both global and regional options are available for all projections. Global maps cover an entire longitude window between selected latitudes (one or more latitude zones) such as the entire globe from the North pole to the South pole, either of the northern or southern hemispheres, or equatorial regions.
Image resolution:
Image size is completely variable. Mapping can be performed at a user- specified resolution. If the user does not specify resolution, the program determines one that is appropriate for a given set of input parameters.
Image pixel length:
Image pixel length can be 1-byte or 2-bytes. However, images of ancillary data are always produced in 2-bytes.
Composite flag:
Images can be optionally composited with previously generated images either spatially or temporally using a 5-way composite flag. The options available are: composite based on minimum nadir angle, average, later, warmer, or colder composites.
Fill-up option:
Holes in a mapped image can be optionally filled using an average method or an adjacent method.
Satellite zenith angle cut-off option:
A cut-off option is available to process only those spots that are within a user defined threshold zenith angle.
3.1.3 LIMITATIONS
1. Master maps from AMSU will not be available until after the launch of NOAA-K. Additional software will have to be written for the Image Mapping System to accommodate the AMSU 1b format which has not yet been defined.
2. Limb correction and nonlinearity correction are implemented as being mutually exclusive. Only one can be selected. For sea surface temperature applications, limb correction must not be selected as a correction to that effect has been taken into account in the Multi- Channel Sea Surface Temperature (MCSST) algorithms.
3. No limits are imposed on the number of image rows. However, there is a limit on the number of image columns. Image columns may not exceed 23476 if using a 1-byte option for image pixel or 11738 if using a 2-byte option. This restriction comes from the fact that the number of image columns times the image pixel length constitutes a logical record length; and a record length of 23476 ( half a track on 3380 disk packs) is optimal to minimize disk storage and access time. Some minimum limits apply on the number of image columns as well; typically, 160 bytes to accommodate the image header record. If the minimum limit is not followed, the header record may overflow into data records. This limitation is, generally, applicable to all image products discussed in this manual.
3.2 VEGETATION INDEX SUBSYSTEM
3.2.1 FUNCTION
The Vegetation Index (VI) subsystem produces a vegetation index from a weekly composite of AVHRR Global Area Coverage (GAC) master maps.
3.2.2 SYSTEM CAPABILITIES
The Vegetation Index subsystem offers the following capabilities:
1. Number of days for composite is user-selectable.
2. The Vegetation Index computed is always a normalized index. However, there is an option to do the Vegetation Index either from raw counts or from albedos. If the albedo option is selected, there is a provision to do it either from pre-launch coefficients ( as retrieved from a level 1b orbit data set) or from user-defined calibration coefficients.
3.2.3 LIMITATIONS
Number of days selected for composite may not exceed 10 days. If it does exceed 10 days, header records run out of room to store additional information.
3.3.1 FUNCTION
The CoastWatch Subsystem extends and supplements the capabilities of IMGMAP, allowing region images generated by IMGMAP to be broken into one or more sectors for which a variety of images can be generated.
3.3.2 SYSTEM CAPABILITIES
* Sectorization: A user may specify geographical sectors with respect to a region for which images are to be generated by IMGMAP. These sectors may lay wholly or partially within the region, and sectors may overlap. For each sector, a variety of images can be generated, as can stand-alone sector graphics files. The user may request sectors at the same resolution as the region images generated by IMGMAP or that the region be subsampled.
* Channel & Ancillary Images: Channel and ancillary images for any sector, similar to those generated by IMGMAP for a region, can be generated.
* SST Images: Split, Dual & Triple Window algorithms for each of the MCSST, CPSST, and NLSST techniques for computing SSTs, in any combination, can be generated for user-defined sectors. The user may alter any or all of the algorithms by providing values for the various coefficients and constants that comprise the algorithm. The user can supply two sets of coefficients for each of the algorithms -- one set for use with pixels that are in darkness and another for pixels in daylight. SST calculations can be restricted to be performed only over water and can also be restricted to cloud-free areas.
* Ocean Reflectance: OR calculations can be performed for user-defined sectors. The coefficients and constants that comprise the algorithm can be modified by the user. OR calculations can be restricted to be performed only over water and can also be restricted to cloud-free areas. The user may specify totally different sets of coefficients for each sector.
* Cloud Masks: The user may select from among 20 user-modifiable cloud tests. The results of the cloud tests can be preserved in a cloud mask image for any sector and the user can designate particular bits in the mask to be set (or, for restoral tests, reset) depending upon the outcome of particular tests. A different bit can be specified for pixels in daylight from those in darkness. Cloud tests can be performed with or without a unit array and in a variety of modes and conditions -- all user specifiable.
* Sector Graphics: For any sector, the user can request graphics be either merged with various images generated, or that graphics be placed in a stand-alone file for that sector. In the event that a stand-alone graphics file for the sector has been created sometime in the past, the user can cause that data to be merged with various images for that sector.
* Data Compression: For any sector the user can specify that the appropriate images for that sector be compressed. (Not all images -- e.g., cloud masks and ancillary data -- are compressible.)
3.3.3 LIMITATIONS
* The limitations listed for the IMGMAP System generally apply.
* Logical Record Length: There is a minimum length allowed for the logical record of any region file and for any sector file produced by OCNMAP of 180 bytes because the logical record length must be large enough to contain the image header.
* Region Graphics: IMGMAP map should produce images without merged graphics. Graphics information should be supplied to OCNMAP in either a single stand-alone region file or in a set of stand-alone sector files, one for each sector.
* File Limit: OCNMAP will produce at most 50 output files for any single run.
* Single-byte Pixel Images: OCNMAP can not currently process single-byte pixel images from IMGMAP, nor output single-byte images except on cloud and graphics images.
* Mixing Cloud Type Modes: Do not mix single-pixel cloud tests with restoral tests performed over a unit array.
3.4.1 FUNCTION
The graphics subsystem produces selected graphics from: lat-long grids, coastlines, and land-sea tags and down-loads them to a static graphics file IMGMAP system uses the static graphics file to optionally merge graphics with image data.
3.4.2 SYSTEM CAPABILITIES
1. The graphics subsystem offers the same mapping options as offered by the IMGMAP system.
2. Griding interval used for lat-long grids is user-selectable. If the user does not specify one, the program uses either a 1- degree interval or a 5-degree interval depending on the mapped resolution.
3. The user can select Geography from one or more continents.
4. The level of detail required for Geography can be selected through a WDB rank file. The current configuration selects all ranks (see Sections 4.2.1.2 and 4.2.1.4 ).
3.4.3 LIMITATIONS
1. In order to successfully merge graphics with image data, the image pixel length has to be 2-bytes. No graphics can be merged with 1-byte pixel data. Also, the static graphics file must have the same blocksize as that of the corresponding image file.
2. The land-sea tag data base, from which the land-sea tags are retrieved, has a resolution of 1/16th of a degree or about 7 km; as a result, the land-sea tag registration may be off by a few pixels if the resolution of the target image is finer than 7 km. This can be rectified by choosing a data base that has a better resolution than 7 km (currently under development).
3. The WDB II access routine performs an extensive search to retrieve appropriate Geography from the data base. This operation takes considerable time for large target areas/images. Time can be saved by selecting only those continents that are required for processing.
3.5.1 FUNCTION The JIC rotational data base system receives JIC image files produced on the DPSS computers and uploads them to a rotating data base resident on the NAS computers.
3.5.2 CAPABILITIES
1. The system operates in a real-time mode. The features offered by the system are generally applicable to other applications.
2. The data base can stack up to eight orbits each of HRPT, LHRR, and GHRR data types.
3. The system maintains a dialogue with the user through a status file. The status file contains relevant information about various files resident on the data base at any given time. Through the status file, the user can select files of interest for further transmission or processing.
The off-line system is invoked by a command: #PANEL after starting up the operational library (see Section 2.2). The following appears on the terminal once the command is entered.
THIS IS A MASTER CLIST FOR THE IMGMAP SYSTEM
SELECT A UTILITY OF INTEREST
1 = #BNDCHK - DETERMINES TEMPERATURE AND VALUE OF A
PIXEL GIVEN POSITION OF THE PIXEL
2 = #BREAKUP - CREATES INDIVIDUAL CHANNEL AND ANCILLARY
FILES FROM OUTPUT IMAGE FILE OF
IMGMAP SYSTEM
3 = #HEADER - DISPLAYS HEADER INFORMATION
4 = #IMGMAP - EXECUTES THE 'IMGMAP' SYSTEM
5 = #LUKUP - CREATES 8 BIT LOOKUP TABLES
6 = #MASK - DETERMINES PERCENTAGE
CLOUDINESS IN A CLOUD MASK FILE
7 = #ORBSTAT - DISPLAYS COMPOSITE INFORMATION
8 = #SETRES - DETERMINES LAT/LONG BOUNDARIES AND
RESOLUTION
9 = #SPACE - DETERMINES AMOUNT OF SPACE REQUIRED
FOR OUTPUT IMAGE FILES
10 = #SYSCON - CREATES CONTROL FILE
11. = #CSTWTCH - CREATES AN OCEAN MAP CONTROL FILE
ENTER Q IF YOU WISH TO QUIT
A description of each utility follows.
3.6.1 #BNDCHK
Given an image file, this utility can be used to display the minimum and maximum pixel values or to display pixel values at specific pixel locations as selected by the user. Displayed parameter is in counts, albedos, temperatures or degrees depending on which image file is selected. The utility displays in numeric order a list of channels and ancillary information that was written to the file. The user can select an image of interest by entering a number corresponding to that of the image desired. Once the image is selected, the user is prompted again to select one of the options, minimum and maximum pixel values or pixel values at user-selected locations.
3.6.2 #BREAKUP
This utility breaks up image files into individual images. The utility can be used to separate channel images, ancillary images, and images of SST. Separated images will be saved into the user's catalog with the user's ID as a prefix. The user will be prompted to enter an image file name that has more than one image written to it. The utility looks into image headers and displays a list of images that were written to the file. The user will be prompted again to select images of interest. Based on the user's selection, appropriate files are allocated, the information is separated, and written to those files.
3.6.3 #HEADER
Given an image file, the utility #HEADER displays the contents of the image header records. Options are available to display the entire header record, to display a selected range of words, or to display a specific word. In addition to display of contents, word positions and a literal description of contents are also provided.
3.6.4 #IMGMAP
Through the utility #IMGMAP, the user can create/update the IMGMAP system control file, select an orbit of interest or have the utility select a suitable one, build a production JCL stream, and submit it from foreground. As a first step, #IMGMAP builds a system control file. Once the control file is ready, orbit selection is initiated. The orbit selection is done using one of two options: Program choice, and User's choice. The User's choice is normally opted when the user wants to select a known 1b orbit for processing. To facilitate this, the utility displays an up-to-date catalog of available 1b orbits for the chosen data type and spacecraft. The Program choice determines a suitable orbit from a catalog of available 1b orbits. Once the orbit selection is done, the orbit Data Set Name (DSN) is implanted in the template JCL and a production JCL is made ready for submission.
The production JCL uses the jobname SMPGACZ1 which can be changed at any time. #IMGMAP also makes a provision to process 1b orbits that are stored to a static 1b data set. Orbits that can not be processed right away are normally stored to a static 1b data set for later processing. #IMGMAP has two key word operands that are satellite dependent. These key words are set to NG and NH, the spacecrafts that are currently in orbit. New satellites can be added to the system by changing the spacecraft IDs.
#IMGMAP does not interact with subsystem control files. Any changes to subsystem control files are to be carried out manually. The MCSST subsystem has four control files, DAMCSST, NAMCSST, DMMCSST, NMMCSST, where the first letter stands for daytime or nighttime and the second one stands for afternoon or morning satellite. The Vegetation Index subsystem has VEGCNTL as a control file and the graphics system has WDBRNK as a control file.
3.6.5 #LUKUP
Through #LUKUP, the user can define interactively a 11-bit to 8-bit look-up table. The look-up table can be created/updated using this utility. Either albedos or temperatures or both can be defined/updated. The table has entries for 5 channels X 2047 values. When the table is created for the first time, look-up values for all 5 channels have to be specified. For successive updates, entries for any channel of interest may be updated without affecting the entries of the rest of the channels. Only the linear relationship is valid. There is no limit on the number of steps. For each step, the user defines an albedo/ temperature value followed by a corresponding 8-bit count value. At the end of the session, the look-up table can be saved to a permanent file.
3.6.6 #MASK
The utility #MASK accesses a cloud mask file supplied by the user and reports the percentage of clouds.
3.6.7 #ORBSTAT
Given an image file as input, the utility #ORBSTAT reports a history of the composite. A list of orbits used to composite the data is reported. #ORBSTAT looks into image headers and retrieves orbital information.
3.6.8 #SETRES
Given the beginning and ending lat-longs, image size, and type of projection, the utility #SETRES computes and displays an optimum resolution (Program choice). Alternatively, given a central lat-long, image size, mapped resolution, and type of projection, #SETRES displays the beginning and ending lat-longs of the calculated target area (User's choice). The utility prompts the user to select one of two options: Program choice, and User's choice. The Program choice is to be opted to obtain a computed resolution. The User's choice returns lat-longs of the calculated target area. Since the same routines and methodology are also used within the IMGMAP system, this utility serves as a guideline to select image size and target area appropriately.
3.6.9 #SPACE
Given the number of channels and ancillary information, and the image size, the utility #SPACE estimates amount of disk space required to allocate an image file.
3.6.10 #SYSCON
Through the utility #SYSCON, the user can create/update the IMGMAP system control file and optionally store it to a permanent file. A temporary file CNTROL will be created under the user's catalog with the user's ID as a prefix. At the end of the session, the utility displays the control file and lets the user make any corrections.
#SYSCON has a few key word operands that are satellite dependent. These are the central wave numbers of channels 3,4, and 5 and equatorial crossing times for both morning and afternoon satellites. New satellites can be added to the system by resetting these parameters
In addition to the panel described above, the off-line system has two information utilities. These are #SHOW and PLOT.
#SHOW displays an up-to-date catalog of available 1b data sets of AVHRR for a chosen data type, HRPT,LHRR, or GHRR. If no catalog is available for a chosen spacecraft or if a valid data type is not selected, the Clist exits with an error message. The catalog defaults to the NOAA-H spacecraft. Catalogs of other spacecrafts can be displayed by specifying the spacecraft ID via the key word 'SCID'. For example, the HRPT catalog of NOAA-G can be displayed with the command: #SHOW HRPT SCID(NG)
The Clist PLOT, when executed after #SHOW, retrieves earth locations from a selected level 1b data set of AVHRR and displays lat-longs of the left most FOV, nadir, and the right most FOV of every few scan lines. The Clist #SHOW displays a catalog of available 1b orbits at any given time in numerical order. The user picks a number displayed for an orbit of interest and enters the same number after the command PLOT to activate the Clist. The data set name of the selected orbit and the number of scan records in that orbit are also made available to the user. This Clist is normally used to verify if an orbit covers a user-selected target area. PLOT is also useful to verify the accuracy of earth locations provided with the 1b data set. The interval used for scan line sampling is 250. No display will be available if #SHOW is not executed prior to the execution of PLOT.
3.6.11 # CSTWTCH
#CSTWTCH initiates the Ocean Map Off-line subsystem. This, unlike the CLIST driven off-line system of Image Map, is an ISPF panel driven application. Through its use, a user can create or modify an Ocean Map control file - the file which controls the Ocean Map program's processing. If the user is familiar with ISPF/PDF, then there should be no problems using this subsystem: ISPF consistently was a major design objective. If a problem does arise, a complete hierarchical structure of Help panels is available at anytime when the HELP command (PF1) is issued. From there, the user has complete access to the remaining HELP panels. The CoastWatch Off-line Tutorials design and function are consistent with other ISPF/PDF Help panels, as well.
This section provides a brief description of each data set used and/or produced by the Image Product System. Ref. 3 provides a detailed byte level description of data formats. Figure 3.0 depicts the overall data flow of the Image Product System. Interconnectivity of the IMGMAP system with its subsystems is also evident from the figure. Data Set Names (DSN) and Data Control Blocks (DCB) are provided for all permanent data sets. All permanent data sets described in this manual reside on the NAS mainframe computers. The JIC image files produced on the DPSS environment are stored to temporary files and file transferred to respective hold files in the NAS environment.
4.1.1 INPUT DATA SETS
4.1.1.1 SYSTEM CONTROL FILE
DSNs:
JIC-GHRR: NSS.PSASAIMP.IMAGV.DATA(JICGCNTL)
JIC-LHRR: NSS.PSASAIMP.IMAGV.DATA(JICLCNTL)
JIC-HRPT: NSS.PSASAIMP.IMAGV.DATA(JICLCNTL)
MASTER MAPS: NSS.PSASAIMP.IMAGV.DATA(MASGCNTL)
OCEAN PROD: TBD
DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=22
File Format: EBCDIC
Through the system control file the user chooses the attributes of the image
product produced by IMGMAP. The file's flags specify data types and
channels, data adjustments, etc., and provide relevant details about the
projection, image resolution, and desirable geographical coverage of the
image product.
The following are the contents of the IMGMAP system control file: Job Processing flag Satellite Products Data types Day/night option Channels (day) FIGURE 3.0 Channels (night) Ancillary images (day) Ancillary images (night) Data corrections Calibration options Graphics Projection Target area Image resolution Image pixel length Composite flag Fill-up option Central wave numbers and equ. crossing times of morning satellite Central wave numbers and equ. crossing times of afternoon satellite Block size of image file Satellite zenith cut off
Section 2.1.1 of Ref. 3 provides a detailed description of these contents. A separate system control file is set up for each operational job. The file is updated to change image product attributes and to change satellite- dependent constants when a new satellite is added to the system.
The system control file contains twenty-two 80-byte data records. The file has no header record. Data items contained in each record begin in the first character of that record and may be followed by a descriptive comment which is prefaced by one or more blanks. Multiple data items in a single record typically contain no embedded spaces (i.e. they appear as a character string). The only exceptions are multiple data items in a record with at least one floating point data item: each of these data items is separated by one or more spaces. The data items in the file can be integers as well as real.
4.1.1.2 JOINT ICE CENTER LOOK-UP TABLE
DSN:NSS.PSASAIMP.IMAGV.JICLUT DCBs: LRECL=10235 BLKSIZE=10235 RECFM=F NREC=1 File Format: Binary
IMGMAP provides four options for calibration (refer to Tables 2.1.1.1 and 2.2.2.2 of Ref. 3). Out of these, options 2 and 3 are used to produce albedos and brightness temperatures. Option 2 maps computed parameters to 11-bit data quantities and option 3 maps them to 8-bit data quantities. Option 2 is typically used with 2-byte image pixels and option 3 with 1-byte pixels. However, the accuracy offered by option 3 may not be adequate for some applications. In order to take advantage of better accuracy offered by option 2 and, at the same time, produce pixels in 1-byte, the user may define a look-up table to reduce 11-bit quantities over a desired temperature range to corresponding 8-bits . The JIC look-up table is used in this context to reduce 11-bit quantities to 8-bits. When the user selects option 2 for calibration and further specifies pixel length as 1, the system automatically looks for a look-up table to convert 11-bit quantities in 8-bits. This table is specific to JIC applications; however, similar tables may be used in place of the JIC look-up table to handle other applications.
The JIC look-up table has 5-partitions, one for each channel. Each partition has a corresponding 8-bit value for each 11-bit value in the table. The table can be created/updated interactively using the Clist #LUKUP (refer to section 5.2.5 of Ref. 4). The table contains no header record and only one data record. Each 2047 words of data correspond to the 2047 unique 11-bit pixel values for a particular channel (there is no pixel value of zero).
4.1.1.3 JOINT ICE CENTER SEASONAL ADJUSTMENT FILE
DSN:NSS.PSASAIMP.IMAGV.DATA(JICSAF) DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=12 File Format: EBCDIC
The Joint Ice Center (JIC) seasonal adjustment file contains seasonally adjusted latitude boundaries of JIC target areas for each month of the year. IMGMAP uses this file only as it creates JIC Products. The file contains no header record, but has one data record for each month of the year. Data in the first record correspond to the month of January. Subsequent records correspond to the following months, ending with December.
4.1.1.4 LOOK-UP TABLE OF SCAN ANGLE, SATELLITE ZENITH ANGLE, AND SECANT OF SATELLITE ZENITH ANGLE DSN:NSS.PSANAVMP.LUKUP.ANGLES DCBs: LRECL=10,000 BLKSIZE=10,000 RECFM=F NREC=3 File Format: Binary
The look-up table for scan angle, satellite zenith angle, and the secant of satellite zenith angle contains angles for GHRR and LHRR/HRPT scans and is used by IMGMAP whenever such ancillary data are necessary. The file contains no header record and only three data records: the first contains scan angles, the second contains satellite zenith angles, and the third contains the secants of the satellite zenith angles.
4.1.1.5 ORBIT DATA SETS IN LEVEL 1b FORMAT
Level 1b data consists of instrument raw data , calibration, and earth location information. For AVHRR, three types of 1b data sets are available: the GHRR, LHRR, and HRPT. The LHRR and HRPT, which are commonly known as Local Area Coverage (LAC), provide an average data resolution of 1.1 km. The GHRR data give an average resolution of 4.0 km. The AVHRR instrument makes observations in five different spectral ranges. The channelization of the AVHRR is as follows:
Channel 1 0.58-0.68 mm Visible
Channel 2 0.725-0.11 mm Reflected IR
Channel 3 3.55-3.93 mm Thermal IR
Channel 4 10.3-11.3 mm Thermal IR
Channel 5 11.5-12.0 mm Thermal IR
With the advent of NOAA-K, one additional infrared channel will be added to operate at 1.6 mm. When this channel becomes available, it will be known as channel 3a and the current channel 3 will be known as 3b. These two channels operate in a time-shared mode with channel 3a providing daytime coverage and channel 3b providing nighttime coverage.
Sensor data from the Advanced Microwave Sounding Unit (AMSU) will be available after the launch of NOAA-K. The sounding unit consists of two instruments: AMSU-A and AMSU-B. The primary purpose of AMSU is to provide microwave sounding capability. However, both AMSU-A and AMSU- B have some channels that operate at window frequencies. These channels will be used to generate master maps of AMSU data. The following table provides a list of AMSU window channels and their central frequencies.
INSTRUMENT Channels Central Frequency (GHz)
AMSU-A 1, 2, 3, 15 23.8, 31.4, 50.3, 89.0
AMSU-B 16,17 89.0,157.0
The level 1b orbit data sets are integral to IMGMAP's production of all image products. The level 1b orbit data set from AVHRR is described in Reference 5. The level 1b orbit data set from AMSU will be described in this document with the advent of NOAA-K.
4.1.1.6 INTERMEDIATE FILE
DSN: Temporary SYSDA file DCBs: LRECL=23476 BLKSIZE=23476 RECFM=FB File Format: Binary
The intermediate file is used internally by IMGMAP as both input and output. It has no utility to the user. For each spot in every scan, the file contains mapped (I,J) values and corresponding ancillary information for all channels and ancillaries selected by the user. IMGMAP dynamically allocates intermediate file records only as they are needed. There is no header record. The file contains as many data records as are needed to accommodate its information.
4.1.1.7 NONLINEAR LOOK-UP TABLES
The nonlinear look-up tables contain correction factors to be applied to
brightness temperatures computed from channels 4 and 5 of AVHRR. There
are separate tables for channels 4 and 5 and for morning and afternoon
satellites. IMGMAP refers to these tables when correction for nonlinearity is
selected through the system control file. Currently, IMGMAP accesses
tables that are set up for NOAA-11 and NOAA-10. New tables must be
introduced as new satellites are added to the system.
4.1.1.8 GRAPHICS FILE
DSN & DCBs: Described under the GRAPHICS Subsystem (Section 4.2.2.1).
The graphics file is produced by the graphics subsystem. If any graphics are
selected through the system control file, IMGMAP overlays the graphics
present in the graphics file on the appropriate images. No graphic overlays
are produced for ancillary images.
4.1.2 IMGMAP OUTPUT DATA SETS
4.1.2.1 INTERMEDIATE FILE
The intermediate file has no utility to the user, but is used internally by
IMGMAP as both input and output. It is described under input files in
Section 4.1.1.6.
4.1.2.2 IMAGE FILES
Image size, pixel length, BLKSIZE, number of channels, and ancillary data are
defined by the user. Image size is determined by the number of image rows
and columns: there is one pixel in each row for every image column and one
data record in each file for every image row. Pixel length is either one or
two bytes. The largest possible BLKSIZE is optimal to minimize I/O. The size
of each image file depends on the selected image size, pixel length, number
of channels, and ancillary information.
4.1.2.2.1 GENERIC DATA RECORD FORMAT FOR IMAGE PRODUCTS
Data for all image products, Joint Ice Center, Master Maps, and Ocean
Products, is structured in such a way that each logical record corresponds to
an image row and each word within a logical record corresponds to a pixel
within that row. Each pixel is either 1 or 2 bytes in length, as defined by
the user. The first word in the first logical record of the image data
corresponds to image coordinate (1,1) and indicates the origin of the image.
The origin is fixed at the upper left corner of the image. Image columns
increase from left to right and rows increase from top to bottom.
Ancillary data for Master Maps and Ocean Products is structured in the same
manner as image data (JIC Products have no ancillary information). Each
ancillary data word is two bytes.
Pixel(word) values are produced from the data values using appropriate
scaling schemes.
4.1.2.2.2 JOINT ICE CENTER FILES
The DSNs and DCBs of the JIC files are described under the JIC Rotational
Data Base Subsystem. This section provides additional information which is
specific to JIC Products.
Joint Ice Center (JIC) images are typically produced in 1024 X 1024 (from
GAC) and 2048 X 2048 (from LAC/HRPT) polar projection. Although the
user can select other projection types and array sizes, these are optimal
given the AVHRR instrument resolution, typical JIC target areas, and JIC
requirements.
For each orbit processed over JIC target areas, images of channels 1 and 4
are produced from daytime overpass and images of channel 4 are produced
from nighttime overpass. Four JIC image files are produced from GAC data.
These represent each combination of day or night and North or South.
Typically only one image file is produced from LAC/HRPT data. This file
represents either day or night and either Northern or Southern hemisphere.
No ancillary data is present for JIC images.
4.1.2.2.3 MASTER MAPS FILES
This section provides additional information which is specific to Master
Maps.
Master Maps are typically produced in 1024 X 1024 polar projection.
Although the user can select other projections and image sizes, this choice is
optimal given the AVHRR instrument resolution, Master Maps target areas,
and Master Maps requirements.
Two Master Maps are produced for all channels of GAC from daytime
overpasses and two for GAC channels 3, 4, and 5 from nighttime
overpasses. Thus four separate image files are produced: one for each
combination of day or night and Northern or Southern hemisphere. At the
end of each file are selected maps for coincident spots: images of scan
time, solar zenith angle, scan angle, and relative azimuth angle. No Master
Maps are created from LAC/HRPT data. The Master Maps described above
are revolving files. At any given time the files consist of mapped AVHRR
data from one or more orbits (typically up to 14). At the end of the day,
these files are copied to daily Master Maps. The daily daytime Master Maps
are routed to the Vegetation Index subsystem for a weekly composite and a
Vegetation Index product.
4.1.2.2.4 OCEAN PRODUCTS FILES
This section provides additional information which is specific to Ocean
Products.
Ocean Products images are typically produced in two formats, mapped and
unmapped. Mapped images are produced in any size, as selected by the
user. Unmapped image size is dependent on the extent of the mapped
region and the sampling mode (scan line or spot).
Ocean Products images are produced from all selected channels and data
types. One image file is created for each target area. The image represents
day, night, Northern hemisphere, Southern hemisphere, or any combination
thereof. At the end of the file are selected maps for coincident spots:
images of solar zenith angle, satellite zenith angle, relative azimuth angle,
and scan time.
4.2.1 INPUT DATA SETS
4.2.1.1 JCL CARD INPUT
The JCL Card input provides necessary input parameters for the graphics
subsystem to generate overlays. The Clist #IMGMAP [see Section 5.2.4.1
of Ref. 4] builds the JCL card input and implants into a template JCL-
NSS.PSASAIMP.IMAGV.JCL(SYSJCL).
The JCL card input is an EBCDIC file which contains seven 80-byte data
records. Data items contained in each record begin in the first character of
that record and may be followed by a descriptive comment which is
prefaced by one or more blanks. Multiple data items in a single record
typically contain no embedded spaces (i.e. they appear as a character
string). The only exceptions are multiple data items in a record with at least
one floating point data item: each of these data items is separated by one
or more spaces. Data items in the file can either be integers or real.
4.2.1.2 WORLD DATA BANK RANK FILE
The World Data Bank (WDB) rank file governs which ranks (geographical
features) of the World Data Bank II geography files are to be included in the
graphics file. If all ranks were selected from all categories, the graphics
generated might be too crowded or produce too much detail. The WDB rank
file is an EBCDIC file which contains twenty-seven 80-byte data records,
one for each rank in each WDB geography data file. The WDB rank file has
no header record. Data items contained in each record begin in the first
character of that record and are followed by a literal description of the rank
which is prefaced by one or more blanks.
4.2.1.3 LAND-SEA TAG FILE
The graphics subsystem uses the land-sea tag database, which consists of
global land-sea flags at 1/16th degree resolution, to determine if a particular
lat-long is land or sea. Each grid point in the database is 1 (for land) or 0
(for sea). Reference 6 may be consulted for a detailed format description of
land-sea tag data base.
4.2.1.4 WORLD DATA BANK II GEOGRAPHY FILES
The graphics subsystem accesses the WDB II geography data files when
geography is selected as one of the graphics parameters in the JCL card
input (Section 4.2.1.1). This database is organized by continents and ranks
(geographical features), resulting in the sixteen data sets whose DSNs are
listed above. The data are first divided by continent: North America, South
America, Europe, Africa, and Asia. Each continent is then divided by
category: CIL=coastlines, islands, and lakes; RIV=rivers;
BDY=international boundaries; and PBY=provincial internal boundaries.
The PBY category is only present for North America.
Within each file, data are organized on the basis of rank. Each rank
corresponds to a geographical feature. The CIL category has up to 12
ranks, the RIV up to 11, the BDY up to 3, and the PBY has 1 rank.
The graphics subsystem retrieves features from this data base with the help
of an access routine, GETGE1 [Ref. 2].
4.2.2 OUTPUT DATA SETS
4.2.2.1 GRAPHICS FILE
The graphics file has no header record. Each logical record in the graphics
file corresponds to an image row and each word in a logical record
corresponds to an image column: the pixels are one byte in length. The
number of data records is the same as the number of image rows and the
length of the records is the same as the number of image columns. Each of
the eight bits in a pixel represents an individual graphics plane.
The daytime AVHRR Master Maps generated by IMGMAP (section
4.1.2.2.3) are further processed by the Vegetation Index subsystem to
produce the Vegetation Index (VI) Product. The Vegetation Index Product is
typically produced after a 7-day composite period, but the user may select
other time periods. The Vegetation Index subsystem consists of two
processors: VEGINDX is executed daily to composite the daytime Master
Maps; WKLYCOMP uses the composite made by VEGINDX to produce the
Vegetation Index Product after the number of days in the compositing period
has elapsed.
4.3.1 VEGETATION INDEX PROCESSOR (VEGINDX)
VEGINDX is executed daily to composite the Master Maps for use by
WKLYCOMP. The length of the compositing period is defined by the user in
the Vegetation Index control file (section 4.3.1.1.1).
4.3.1.1 VEGINDX INPUT DATA SETS
4.3.1.1.1 VEGETATION INDEX SUBSYSTEM CONTROL FILE
Through the Vegetation Index subsystem control file the user chooses the
attributes of the Vegetation Index Product. The file's flags specify the
number of days in a composite period and provide relevant details used in
the creation of the product. The Vegetation Index subsystem control file is
shared by VEGINDX and WKLYCOMP and remains unchanged while each
Vegetation Index Product is created (a period of 7 days or so, as determined
by the user).
Through the Vegetation Index subsystem control file, the following options
can be selected:
The Vegetation Index subsystem control file can be updated manually.
The Vegetation Index subsystem control file is an EBCDIC file which
contains seven 80-byte data records. The file has no header record. Data
items contained in each record begin in the first character of that record and
may be followed by a descriptive comment which is prefaced by one or
more blanks. Multiple data items in a single record are separated from one
another by one or more spaces.
4.3.1.1.2 MAPPED IMAGE FILES
The two mapped image files are the two daytime AVHRR Master Maps
generated by IMGMAP (one file for each hemisphere). DCBs and header and
data record formats are identical to the Master Maps files and are defined in
Section 4.1.2.2.3. Each day VEGINDX composites the mapped image files
into two weekly composite files (Section 4.3.1.2.1).
4.3.1.2 VEGINDX OUTPUT DATA SETS
4.3.1.2.1 WEEKLY COMPOSITE FILES
The two weekly composite files, one for each hemisphere, contain the
composites of the two daytime AVHRR Master Maps files over the number
of days specified in the Vegetation Index subsystem control file. Once these
days have elapsed, WKLYCOMP uses the weekly composite files to produce
the Vegetation Index Product. Once the Vegetation Index Product has been
produced, the JCL which controls WKLYCOMP initializes the two files by
writing zeroes to them.
The two weekly composite files contain the same number of records as the
Master Maps files from which they originated. They contain ancillary data
composites only if ancillary data were present in the Master Maps.
The headers of the weekly composite files each span two records to
accommodate the overflow of all the orbital information. The header
records are the same size as the data records. Data record formats are
identical to the Master Maps files and are defined in Section 4.1.2.2.3
4.3.2 WEEKLY COMPOSITE PROCESSOR (WKLYCOMP)
WKLYCOMP is executed whenever the number of days specified in the
Vegetation Index subsystem control file (section 4.3.1.1.1) has elapsed.
Typically this is seven days. WKLYCOMP uses the composite made by
VEGINDX to produce the Vegetation Index Product.
4.3.2.1 WKLYCOMP INPUT DATA SETS
4.3.2.1.1 VEGETATION INDEX SUBSYSTEM CONTROL FILE
The Vegetation Index subsystem control file is shared by VEGINDX and
WKLYCOMP and is described in section 4.3.1.1.1
4.3.2.1.2 WEEKLY COMPOSITE FILES
The two weekly composite files are produced by the VEGINDX for use by
WKLYCOMP. They contain the composites of daytime Master Maps over
the number of days specified in the Vegetation Index subsystem control file.
The JCL which controls WKLYCOMP initializes these two files by writing
zeroes to them once the Vegetation Index Product has been produced.
4.3.2.2 WKLYCOMP OUTPUT DATA SETS
4.3.2.2.1 VEGETATION INDEX FILES
There are two Vegetation Index Product files, one for each hemisphere.
WKLYCOMP produces these files using the two weekly composite files
produced by VEGINDX when the number of days specified in the Vegetation
Index subsystem control file has elapsed.
The header and data record formats are identical to the weekly composite
files. However, the image pixel takes up four bytes to accommodate the
index which is a floating point number.
The AVHRR images generated by IMGMAP (section 4.1.2.2) are further
processed by the CoastWatch Subsystem to produce sector images of
channel and ancillary data, sea-surface temperatures, ocean reflectance,
cloud masks, and stand-alone graphics.
4.4.1 OCNMAP INPUT DATA SETS
4.4.1.1 OCNMAP CONTROL FILES
The OCNMAP control files are satellite specific. Two are maintained for the
operation: one for the morning satellite and one for the afternoon satellite.
Control files are text files, and when created by the off-line subsystem of
CoastWatch, contain comments that indicate what is meant by the data on
any given line. As such, they can be read and understood by a user and, if
necessary, modified. However, we recommend against such practice and
intend rather that the off-line subsystem be used to create and update all
control files. In this way range and consistency checking can be performed
on all the user options prior to job submission.
4.4.1.2 IMGMAP OUTPUT IMAGE FILE
Any Ocean Products file generated by IMGMAP can be read and, assuming
all the requisite data is present, processed by OCNMAP. The DCBs, and the
header and data record formats for these files are defined in section 4.1.2.2.
4.4.1.3 SST FIElD FILES
The SST Field file capability has not yet been implemented in OCNMAP.
4.4.2 OCNMAP OUTPUT DATA SETS
OCNMAP output image files look just like the output files from IMGMAP
except that: (1) there is never more that one image in a file, and (2) certain
of the images may be compressed and have compressed graphics
information attached. Stand-alone sector graphics files look like those
produced by the graphics subsystem except that they may be compressed
and, if they are, they contain a header that is similar to an image file header.
Channel, SST, OR and graphics images may be compressed, ancillaries may
not, nor may any 1-byte/pixel image (except the sector graphics images).
Cloud mask files have the properties of uncompressed graphics files; i.e.,
they have 1 byte per pixel, logical record lengths in bytes equal to the
number of columns in the image, and block sizes that are equal to the block
size of corresponding, uncompressed channel image of the same sector.
4.4.2.1 The Transfer File
DCB LRECL=80 BLKSIZE=3120 RECFM=FB FILE FORMAT: EBCDIC
The Transfer File is output by OCNMAP as an aid to subsequent processing
which might be altered dependent upon the percentage of good (that is,
non-zero) pixels there are in any particular sector image.
The most recent eight orbits of JIC image files are saved by the ORBSTAT
processors in rotational databases. Images derived from each orbit data
type (GAC, LAC, and HRPT) are saved in separate databases. The images in
each database are referenced by database position. Each position
represents one orbit: orbit 1 is the first position and orbit 8 is the last. This
referencing scheme is not tied to the actual orbit which each image
represents. Into each database, the ORBSTAT processors transfer images
from one orbit into orbit 1, from the next orbit into orbit 2, ..., from the next
orbit into orbit 8, and then back to orbit 1. As images are transferred into
orbit n, the images which previously occupied orbit n are overwritten.
4.5.1 GAC ORBIT DATABASE
The GORBSTAT processor transfers GAC orbit JIC images into the GAC
orbit database. Four JIC images are produced from GAC data for each orbit
(one for each quadrant), so each of the eight orbits in the GAC orbit
database contains four image files. Files 1 to 4 are orbit 1, 5 to 8 are orbit
2, and so on, to files 29 to 32, which are orbit 8.
4.5.1.1 GAC INPUT DATA SETS
4.5.1.1.1 GAC ORBIT NAME FILE
This file contains the name of the level 1b data set most recently made
available to the image mapping system from GAC data. The file is an
EBCDIC file with no header record and only one 80-byte data record. The
level 1b data set name begins in the first character of the record and
contains no embedded blanks.
4.5.1.1.2 GAC STATUS FILE
The GAC status file contains housekeeping information and status
information about the eight most recent orbits that have been processed by
the IMGMAP system. The user can browse this file to display status
information on the terminal. The JIC user interacts with this file to store
information about the orbits that have already been accessed.
This file is an EBCDIC file with no header record and forty-one 80-byte data
records. The first data record contains housekeeping information and the
next 40 contain status information.
4.5.1.1.3 GAC DATA FILE
The GAC data file contains the Data Set Names (DSN) of JIC image files in
the rotating data base. The data file has 32 entries. The DSNs of GAC
rotational data base files are: COM.PSANAVMP.ORBITm.GJICQn (m=1 to
8; n=1 to 4). The DCBs are LRECL=1024 BLKSIZE=22528 RECFM=FB.
This file is an EBCDIC file with no header record and thirty-two 80-byte data
records.
4.5.1.1.4 GAC HOLD FILES
Hold files are copies of JIC image files produced by the IMGMAP system on
the DPSS computers. Hold files reside on the NAS computers. GORBSTAT
processor uploads hold files to appropriate locations on the GAC data base.
4.5.1.2 GAC OUTPUT DATA SETS
4.5.1.2.1 GAC STATUS FILE
This is the same file used by this process as an input file. It is described in
section 4.5.1.1.2.
4.5.2 LAC ORBIT DATABASE
The LORBSTAT processor transfers LAC orbit JIC images into the LAC orbit
database. One JIC image is produced from LAC data for each orbit, so each
of the eight orbits in the LAC orbit database contains one image file. File 1
is orbit 1, 2 is orbit 2, and so on.
4.5.2.1 LAC INPUT DATA SETS
4.5.2.1.1 LAC ORBIT NAME FILE
This file contains the name of the level 1b data set most recently made
available to the image mapping system from LAC data. The file is an
EBCDIC file with no header record and only one 80-byte data record. The
level 1b data set name begins in the first character of the record and
contains no embedded blanks.
4.5.2.1.2 LAC STATUS FILE
The LAC status file contains housekeeping information and status
information about the eight most recent orbits that have been processed by
the IMGMAP system. The user can browse this file to display status
information on the terminal. The JIC user interacts with this file to store
information about the orbits that have already been accessed.
This file is an EBCDIC file with no header record and seventeen 80-byte data
records. The first data record contains housekeeping information and the
next 16 contain status information.
4.5.2.1.3 LAC DATA FILE
The LAC data file contains the Data Set Names (DSN) of JIC image files in the
rotating data base. The data file has 8 entries. The DSNs of LAC rotational
data base files are: COM.PSANAVMP.ORBITm.LJIC (m=1 to 8). The DCBs
are LRECL=2048 BLKSIZE=22528 RECFM=FB.
This file is an EBCDIC file with no header record and eight 80-byte data
records.
4.5.2.1.4 LAC HOLD FILE
Hold files are copies of JIC image files produced by the IMGMAP system on the
DPSS computers. Hold files reside on the NAS computers. LORBSTAT
processor uploads hold files to appropriate locations on the LAC data base.
4.5.2.2 LAC OUTPUT DATA SETS
4.5.2.2.1 LAC STATUS FILE
This is the same file used by this process as an input file. It is described in
section 4.5.2.1.2.
4.5.3 HRPT ORBIT DATABASE
The HORBSTAT processor transfers HRPT orbit JIC images into the HRPT orbit
database. One JIC image is produced from HRPT data for each orbit, so each
of the eight orbits in the HRPT orbit database contains one image file. Files 1
is orbit 1, 2 is orbit 2, and so on.
4.5.3.1 HRPT INPUT DATA SETS
4.5.3.1.1 HRPT ORBIT NAME FILE
This file contains the name of the level 1b data set most recently made
available to the image mapping system from HRPT data. The file is an EBCDIC
file with no header record and only one 80-byte data record. The level 1b data
set name begins in the first character of the record and contains no embedded
blanks.
4.5.3.1.2 HRPT STATUS FILE
The HRPT status file contains housekeeping information and status information
about the eight most recent orbits that have been processed by the IMGMAP
system. The user can browse this file to display status information on the
terminal. The JIC user interacts with this file to store information about the
orbits that have already been accessed.
This file is an EBCDIC file with no header record and seventeen 80-byte data
records. The first data record contains housekeeping information and the next
16 contain status information.
4.5.3.1.3 HRPT DATA FILE
The HRPT data file contains the Data Set Names (DSN) of JIC image files in the
rotating data base. The data file has 8 entries. The DSNs of HRPT rotational
data base files are: COM.PSANAVMP.ORBITm.HJIC (m=1 to 8). The DCBs
are LRECL=2048 BLKSIZE=22528 RECFM=FB.
This file is an EBCDIC file with no header record and eight 80-byte data
records.
4.5.3.1.4 HRPT HOLD FILE
Hold files are copies of JIC image files produced by the IMGMAP system on the
DPSS computers. Hold files reside on the NAS computers. HORBSTAT
processor uploads hold files to appropriate locations on the HRPT data base.
4.5.3.2 HRPT OUTPUT DATA SETS
4.5.3.2.1 HRPT STATUS FILE
This is the same file used by this process as an input file and is described in
Section 4.5.3.1.2.
The IMGMAP system requires a dedicated JCL for each product area (JIC maps,
master maps, or ocean maps), for each data type (low resolution GHRR, or high
resolution LHRR or HRPT), and for each target area. Resultantly, there are two
dedicated jobs for JIC products, one to produce low resolution GHRR maps and
the other to produce high resolution LHRR/HRPT maps; one dedicated job for
master maps to produce GHRR master maps; and as many jobs as target areas
for ocean products. However, if target areas are contiguous or likely to be
covered by the same orbit, individual jobs may be combined into one job. The
Vegetation Index subsystem has two jobs, a weekly composite job and a
Vegetation Index job. The Rotational Database subsystem has three jobs, one
each for GHRR, LHRR, and HRPT. No separate JCL is required for the Graphics
subsystem, which either runs off-line or on-line with the IMGMAP JCL. The
CoastWatch subsystem is considerably more complicated and requires a
separate JCL deck for each of the operational regions. See 5.3.4.3 below.
5.2 JOB INITIATION AND EXECUTION MECHANISM
The job initiation and execution is triggered by a special software system, the
DPSS to CCF Data Transfer Monitor (DCDTM) system. The NOAA/NESDIS
Data Processing and Services Subsystem (DPSS) performs ground data
handling functions for data collected by NOAA environmental satellites, and
stores the ingested data on a 24 hour revolving METSAT DPSS 1b Database.
Once the process is complete, the 1b data sets are transferred to the NAS
environment using the File Transfer Program (FTP). At the same time an array
of jobs are initiated and set into execution mode. The DCDTM system is also
responsible for monitoring 1b data sets that have already been processed. In
the NAS environment, the Image Product System is likely to take advantage of
the features offered by the DCDTM system to submit jobs in a timely and
orderly fashion.
In the DPSS environment, currently, the job initiation and execution is
controlled by the ingest network. When the system is completely operational,
the DCDTM software may be used in the DPSS environment as well to monitor
job execution.
5.3.1.1 OPERATIONAL SCHEDULE
Ocean maps require high resolution AVHRR data from LAC and HRPT. The low
resolution GAC data is also required for some operational areas. The
operational schedule for ocean products is being finalized. At least one daytime
and a nighttime orbit for each operational area is likely to be considered for
processing. All steps are executed in near real-time as and when a suitable
orbit is available for processing.
5.3.2 MASTER MAPS
5.3.2.1 OPERATIONAL SCHEDULE
Currently, master maps are produced from AVHRR GAC data. The map
database contains a spatial composite of all GAC orbits processed over a period
of 24-hours. The database files are revolving files and at any given time they
contain data processed from one or more orbits. At the end of the day, the
database files are copied to daily master maps and the daily master maps are
used to produce weekly composite maps. The weekly composite maps contain
a temporal composite of daily master maps. At the end of the week,
Vegetation Index is computed from weekly composite maps and Vegetation
Index maps are produced. Step 1 is executed automatically in real-time as a
1b GAC orbit is available for processing. Steps 2 through 5 are executed
manually once a day. Step 6 is executed manually once a week.
5.3.3 JIC PRODUCTS
5.3.3.1 OPERATIONAL SCHEDULE
The JIC products are produced operationally on the DPSS computers.
Currently, all orbits of GAC and selected orbits of LAC and HRPT are
processed. Steps 1 through 4 are executed automatically in real-time as and
when a 1b orbit is ready for processing. Step 5 is executed manually as
frequently as necessary. The JIC rotational database contains mapped data
from eight most recent orbits of each data type.
5.3.4 COASTWATCH
The CoastWatch operational job stream for any particular region requires some
special preparation to be done in the interest of efficiency. Specifically, it is
advantageous to prepare stand-alone graphics files for each of the region's
sectors so that no additional graphics information need be generated during the
day-to-day operational runs. This one-time-only preparation will result in
significant savings of CPU time in subsequent OCNMAP runs.
5.3.4.1 PRELIMINARY STEPS
STEP 1: Use the Graphics subsystem to produce a stand-alone graphics file for
the region. The control file should request all those graphics options that will
be necessary for any of the sectors. The control file should also specify the
desired grid spacing, and if different grid spacings are required for different
sectors, this step will have to be repeated for each of the different spacings.
For example, if there are several sectors that are to have a 2-degree grid
spacing and a synoptic sector that is to have a 5-degree spacing, 2 runs of the
graphics will be required: one to produce a graphics file with 2-degree
latitude/longitude boxes and another to produce the 5-degree spacing. Once
the sector images have been created, the region graphics files can be deleted.
It would probably be wise, however, to retain the control files that were used
in their creation.
STEP 2: Use OCNMAP to produce the stand-alone sector graphics files for
each of the region's sectors. This step must be done once for each of the
stand-alone graphics files produced as described in step 1. Thus, following the
example cited in step 1, one run would be required to produce the stand-alone
graphics file for the synoptic sector and another to produce the graphics files
for all the other sectors.
STEP 3: Allocate data sets to contain the sector images that are to be
generated by OCNMAP.
STEP 4: Produce control files for IMGMAP and OCNMAP using the Image
Products Off-line Subsystem.
5.3.4.3 OPERATIONAL SCHEDULE
As HRPT or, in some cases GAC, data sets are received from SOCC, a suite of
jobs (one for each of the CoastWatch Regions) is submitted. The first step of
each of these jobs determine whether the 1B data actually covers the
associated region. The job proceeds only in cases where there is sufficient
coverage. IMGMAP is then run to produce a region file and then OCNMAP to
perform the sectorization and to produce any additional derived products (SST,
OR, cloud masks). The final step in a CoastWatch job reads the transfer file, a
file that gives the percentage of nonzero pixels in each of the sectors, and
constructs a jcl to transfer those sectors with a sufficiently high percentage of
nonzero pixels to the end users. The completed jcl is submitted through the
internal reader.
Storage requirements for each of the three major products are projected in this
section. Operational requirements could vary from these estimates depending
on how the Final Operational Capability (FOC) is implemented.
6.1.1 OCEAN PRODUCTS
Several regions have been defined for the CoastWatch Final Operational
Capability, however, the details of the implementation are likely to change, so
we will explain how the storage requirements can be calculated. The regions
currently defined are:
6.1.1.1 COMPUTING STORAGE REQUIREMENTS FOR A COASTWATCH
REGION
The data sets required for a typical CoastWatch region are as follows:
Notes:
1. Under the operational scenario, graphics
information is provided via stand-alone sector graphics
files, and, hence, the region graphics file need
only exist during the production of the sector
graphics files and may, thereafter, be deleted.
Notes:
1. Channel images, SSTs and Ocean Reflectances
can be compressed and, invariably, are under
the operational scenario.
2. The compression factor accounts for the savings
due to compression but must allow for a margin
of error since the actual size of a compressed file
may vary from run to run. Currently a value of 0.70 is
in use.
Notes:
1. Ancillary data cannot be compressed and are
always 2-bytes/pixel.
2. Cloud masks cannot be compressed and are
always 1-byte/pixel
6.1.2 MASTER MAPS
6.2.1 DPSS MAINFRAME COMPUTERS
The NOAA/NESDIS Data Processing and Services Subsystem performs the
ground data handling functions for data collected by NOAA environmental
satellites. The computer system will be the central facility for the ingest,
storage and distribution of both polar orbital and geostationary satellite digital
sensor data. Many products derived from this data, as well as data and
products from the Department of Defense will also be generated and/or stored
on the DPSS. In addition to the current projects many future projects are
planned for DPSS. Resultantly, a simultaneous upgrading of DPSS is also being
undertaken. Plans are under way to replace the DPSS computers with more
powerful mainframe computers which are expected to be an order of magnitude
faster than the current ones. Currently, the DPSS equipment includes an IBM
4300 multiprocessor system. The central processors are made up of one 4341
and four 4381 mainframes for DPSS operational processing.
6.2.2 NAS MAINFRAME COMPUTERS
The Central Computer Facility (CCF) at NOAA has three mainframe computers,
the NAS series, to provide necessary software and product support to National
Weather Service (NWS) and NESDIS.
Initially the CCF consisted of IBM 360/195 computers. Since the year 1984 it
has undergone a major upgrade. The CCF IBM 360/195 computers were
replaced by NAS 9000 series computers. The NAS 9000 series have an IBM
compatible processor. These are continuously being upgraded with additional
central processors, central memory, channels, and channel-to-channel (CTC)
adapters. This continuous upgrading has also resulted in awarding the NAS the
ability to support an up-to-date version of IBM's operating system, the
MVS/XA. The NAS 9000 series have 9040, 9050 and 9060 as uniprocessors
and 9070 and 9080 as dual processors. The present configuration of CCF
consists of one NAS 9050 and two 9070s. The existing 9050 is also in the
process of being upgraded to 9070.
The NAS computers are primarily used to perform post 1b processing of polar
orbital data and processing of geostationary and conventional data.
6.2.3 IBM PC WORKSTATION
The IBM PC-AT with Number Nine Revolution graphics card was chosen as a
workstation for the development NOAA-K,L&M Interactive System. The
objectives of the system are many-fold, to help study the performance of cloud
tests on high resolution HRPT/LAC data, to aid in the study of appropriate
methods for data composite, and to serve as a support tool for various research
& developmental (R & D) activities. The current configuration of the
workstation is as follows:
6.2.4 OPC OPERATIONAL WORKSTATION
The operational system proposed for the Ocean Product Center (OPC) consists
of two IBM PC-Compatible computers, one for operation, and the other for
backup. The current configuration of the workstation is as follows:
6.2.5 DIFAS WORKSTATION
The (DIFAS) workstation runs on a dual processor with GPX MICROVAX II as
one processor and the RAMTEK as the other. The RAMTEK processor is
equipped with display monitors and support software for image processing.
6.3.1 DPSS AND NAS
Communication between the NAS and the DPSS computers is achieved through
a special privileged software, the File Transfer Program (FTP). Currently, this
feature is utilized mostly for internal use and front-end processing. The JIC
products produced on the DPSS computers are file transferred to the NAS
computer using FTP. This was necessary as there is no link between the DPSS
computers and the DIFAS workstation.
6.3.2 DPSS AND DIFAS
There is no communication link between the DPSS computers and the DIFAS
workstation.
6.3.3 DPSS AND PC WORKSTATIONS
The low speed communication link that exists between the DPSS and the PC
has a very limited usage as far as its utility for NOAA-K Image Products is
concerned. This link is likely to be upgraded to a T1 link (1.64 mb). If a
compatible communication link does not exist during the operational phase,
data transfer from the DPSS to the PCs will be done via the NAS computers.
6.3.4 DPSS AND OPC OPERATIONAL SYSTEM
Currently, there is a 9600 baud communication link between the DPSS and the
OPC. A 56kb line has been procured; an Ethernet node off the T1 link is also
a possibility. If a compatible communication link does not exist during the
operational time frame, data transfer from the DPSS to the OPC will be done
via the NAS computers.
6.3.5 NAS AND DIFAS
Communication between the NAS mainframe computers and the JIC DIFAS
workstation is accomplished using Ethernet.
6.3.6 NAS AND PC WORKSTATIONS
Communication between the NAS computers and PC workstations is
accomplished using FORTE, and IRMA communication boards which are
connected to controller boxes.
A subset of the Image Product System that produces Ice maps for the Joint Ice
Center is already operational on the DPSS computers. Currently, the system
produces maps from all orbits of GHRR and selected orbits of LHRR and HRPT.
The DPSS version is being upgraded to produce Master Maps, Ocean Products,
and the derived products. However, for want of resources, the production of
Ocean maps will not start until after the DPSS computers are replaced with the
new ones. With the current version of DPSS computers, it is expected that the
production of Master maps and the Vegetation Index will be initiated by
replacing the Global Vegetation Index (GVI) system with the upgraded
software. To meet the current operational needs, production of Ocean maps
will be initiated on the NAS mainframe computers through the CoastWatch
program. Because of this distributed processing, the impact on computer
resources will not be significant.
The job performance statistics are as given in Table 1.0. The figures are valid
for the current configuration of NAS computers.
6.5.1 CoastWatch Performance Statistics
Operational CoastWatch jobs can vary considerably in what they are asked to
do and as a result no one set of statistics can accurately describe all of them.
Here we provide a description of the job currently being used to process data
for the Great Lakes Region:
Job Name: SCWGTLK9
Characteristics: The region is to be divided into 4 sectors, one of which is
synoptic. Channel data and SSTs are output. No cloud masks are produced and
Ocean Reflectance is not computed. Graphics information is to be merged onto
the images from pre-existing sector graphics files, and all the images are
compressed.
Statistics: In the following table times are given as minutes:seconds.
There is an additional step to automatically transfer the products to the end
user but it does not cause a significant increase in CPU or clock time.
Table 1.0
This section provides an overview of maintenance and monitoring capabilities.
Reference 7 provides a more detailed description of these capabilities.
7.1 ENVIRONMENT
The maintenance and monitoring environment consists of the DPSS mainframe
computers, the NAS mainframe computers and display workstations.
7.2 SCIENCE MAINTENANCE
No algorithm changes are expected in IMGMAP, Graphics, Vegetation Index,
and Rotational Database systems. Algorithms in the CoastWatch subsystem
are likely to be modified from time to time in the wake of changing satellites,
cloud tests, and SST algorithms. However, no software changes are necessary
as these algorithms are mapped into a generic scheme of equations and
integrated into CoastWatch control files (see Section 3.3.3 of Ref. 4). By
making changes to regression coefficients and thresholds in the control files,
new algorithms can be introduced into the system.
7.3 DATA SIGNAL MONITORING
Data dropouts and noise related problems in 1b data are reported in the hard
copy output produced by IMGMAP. IMGMAP provides these statistics for each
orbit processed by the system. These statistics indicate the quality of 1b data.
7.4 JOB NETWORK MONITORING
The DCDTM system is used to monitor the job network and to prevent the
same orbit from getting reprocessed.
7.5 OFF-LINE MONITORING
Product completion, accuracy, and quality are monitored using the utilities
developed for the Image Product Off-line System. These include #BREAKUP,
#HEADER, #BNDCHK, and #ORBSTAT. The off-line system is also equipped
with file management utilities such as #LUKUP, #SYSCON, and #SPACE, and
information utilities such as #SETRES, #MASK, #SHOW, and PLOT (see
Section 3.6 for more details) .
7.6 IMAGE DISPLAY
Image display is a powerful means of monitoring the quality and accuracy of
products produced by the Image Product System. The Interactive Digital
Image Display and Analysis System (IDIDAS) is widely used within the NESDIS
for image display and analysis. The system is being used for both research and
operational use. Routine monitoring will be done using the capabilities provided
by IDIDAS . Other capabilities such as the VICOM display system [Ref. 8] and
Image Library And Browse System (ILABS) [Ref. 9] are also available. The JIC
personnel monitor JIC Ice products on the DIFAS workstation. Most of the
display workstations work with a standard 512 X 512 images.
7.7 LIBRARY MAINTENANCE
The operational JCLs and control files are exclusively meant for operational use
and should not be used in any other context unless the purpose is to recover
data that are lost due to system failure. A limited privileged access is
recommended for operational libraries to prevent members from being deleted
or destroyed. In addition to the system backup performed by the systems
administrator, routine backup is recommended at least once a week. The JCL
that performs the backup can be placed on an auto scheduler. The Clists
#BACKUP and #RETRIEVE are also available for backup and recovery. Any
changes to software or data files must be made in developmental libraries first
and later transferred to operational libraries only after they are completely
tested and found to be working right. Configuration control procedures should
be enforced to document any changes made to the software.
7.8 SPECIAL MAINTENANCE PROCEDURES
7.8.1 NEW SATELLITE IMPLEMENTATION
No software changes to IMGMAP are necessary to introduce new satellites.
(Minor ones are necessary to OCNMAP and are described in the next
paragraph.) The IMGMAP system control files contain satellite dependent
variables for both morning and afternoon satellites. These are the central wave
numbers of channels 3, 4 and 5 of the AVHRR instrument and the equator
crossing times. The CLIST #SYSCON can be used to create a control file with
the new wave numbers and equator crossing times. The data set names (DSNs)
of the non-linear look-up tables of the new satellite must also replace the
existing ones in the operational JCLs.
OCNMAP of the CoastWatch subsystem of Image Products has a subroutine,
CLDSET, that must be modified to include the new value of the AVHRR
instrument's central wave number, and it must be modified to access any data
for the new satellite from the visible cloud threshold file. Of course, CLDSET
will then have to be recompiled and linked to the OCNMAP load module.
It may be desirable to produce new OCNMAP control files, but that must be
decided by the user community. If so, #CSTWTCH will provide an easy means
of producing the new control files or updating the old ones.
7.8.2 ADDING A NEW JOB
To add a new job, system control files and processing JCLs have to be
developed. Unique Job names for each product area are yet to be developed.
7.8.3 DATA RECOVERY
Data lost due to system failure can be recovered by processing the same orbit
off-line using the Clist #IMGMAP (See section 3.6.4).
7.8.4 MODIFYING EXISTING PROGRAMS
If the IMGMAP software is modified for any reason, all three products produced
by the systems, the JIC maps, master maps, and ocean maps must be tested
independently to ensure that changes made to one product do not affect the
production of other products.
REFERENCES
1. Critical Design of NOAA-K, L & M Environmental Image Products. SMSRC
Report (Tadepalli, K. and Jandhyala, A., 1988) .
2. Geography Routines for World Data Bank II (Kidwell, D.) 763-8133.
3. Interface Control Document for NOAA-K, L & M Environmental Image
Products (Tadepalli, K. 1991).
4. USER'S MANUAL for the NOAA-K, L & M Environmental Image Products
(Tadepalli, K., 1990).
5. NOAA Polar Orbiter Data (TIROS-N,NOAA-6,NOAA-7,NOAA-8) User's Guide
(Kidwell,K.,1984).
6. Sapper,J., PSB-NOAA/NESDIS, 763-4310.
7. MAINTENANCE MANUAL for the NOAA-K,L,&M Environmental Image
Products (Jandhyala, A.,1990).
8. Image Processing of Interactive SST- User's Guide (Tadepalli,K.,1987).
9. ILABS User's Manual (Jandhyala,V.,1991).
10. Program Description Document for the NOAA-K, l & M
Environmental Image Products (Jarva, V., Aug. 1992).
11. Maintenance Manual for the CoastWatch Subsystems of the
Environmental Image Products (Jarva, V., Aug. 1992).
Last Modified December 10, 2003 (jw)
DSNs: NSS.PSANAVMP.COMPST.PRIMARY1 Northern Hemisphere
(Daytime)
NSS.PSANAVMP.COMPST.PRIMARY2 Southern Hemisphere
(Daytime)
NSS.PSANAVMP.COMPST.PRIMARY3 Northern Hemisphere
(Nighttime)
NSS.PSANAVMP.COMPST.PRIMARY4 Southern Hemisphere
(Nighttime)
DCBs: LRECL=1024 BLKSIZE=22528 RECFM=FB
File Format: Binary
DSN & DCBs: TBD
4.2 GRAPHICS SUBSYSTEM
DSN: NSS.PSASAIMP.IMAGV.JCL (SYSJCL)
DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=7
File Format: EBCDIC
DSN:NSS.PSASAIMP.IMAGV.DATA(WDBRNK)
DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=27
File Format: EBCDIC
DSN:NSS.PSATAVST.LSTAGS.TEMPHIGH
DCBs: LRECL=3200 BLKSIZE=3200 RECFM=F
FILE FORMAT: BINARY
DSNs: IMS.GETGEO.NORTH.AMERICA.CIL
IMS.GETGEO.NORTH.AMERICA.RIV
IMS.GETGEO.NORTH.AMERICA.BDY
IMS.GETGEO.NORTH.AMERICA.PBY
IMS.GETGEO.SOUTH.AMERICA.CIL
IMS.GETGEO.SOUTH.AMERICA.RIV
IMS.GETGEO.SOUTH.AMERICA.BDY
IMS.GETGEO.EUROPE.CIL
IMS.GETGEO.EUROPE.RIV
IMS.GETGEO.EUROPE.BDY
IMS.GETGEO.AFRICA.CIL
IMS.GETGEO.AFRICA.RIV
IMS.GETGEO.AFRICA.BDY
IMS.GETGEO.ASIA.CIL
IMS.GETGEO.ASIA.RIV
IMS.GETGEO.ASIA.BDY
DCBs: LRECL=2052 BLKSIZE=2056 RECFM=VS
File Format: Binary
DSN: TBD
DCBs: LRECL=#image columns
BLKSIZE=BLKSIZE of IMGMAP image file
RECFM=FB/F
NREC=#image rows
File Format: Binary
4.3 VEGETATION INDEX (VI) SUBSYSTEM
DSN:NSS.PSASAIMP.IMAGV.DATA(VEGCNTL)
DCBs:LRECL=80 BLKSIZE=3120 RECFM=FB NREC=7
File Format: EBCDIC
Number of days for composite
Calibration flag
Calibration coefficients
DCBs
Vegetation Index computation flag
DSNs: NSS.PSANAVMP.COMPST.MASMAP1
NSS.PSANAVMP.COMPST.MASMAP2
DCBs : LRECL=1024 BLKSIZE=22528 RECFM=FB
File Format: Binary
DSNs: NSS.PSANAVMP.COMPST.WEEKLY1
NSS.PSANAVMP.COMPST.WEEKLY2
DCBs : LRECL=1024 BLKSIZE=22528 RECFM=FB
File Format: Binary
DSNs: NSS.PSANAVMP.COMPST.VEGINDX1
NSS.PSANAVMP.COMPST.VEGINDX2
DCBs:LRECL=4096 BLKSIZE=20480 RECFM=FB
File Format: Binary
4.4 COASTWATCH SUBSYSTEM
DSNs: TBD
DCBs: LRECL=80
BLKSIZE=80 * N, where N is an integer. RECFM=FB
Uncompressed Compressed
------------------- -----------------------
DCB: See section 4.1.2.2 LRECL=1024 BLKSIZE=5120
FILE FORMAT: See section 4.1.2.2 BINARY
4.5 JIC ROTATIONAL DATABASE SUBSYSTEM
DSN: NSS.PSANAVMP.ONAME.GJIC
DCBs: LRECL=80 BLKSIZE=80 RECFM=F NREC=1
File Format: EBCDIC
DSN:COM.PSANAVMP.RSTATUS.GJIC
DCBs:LRECL=80 BLKSIZE=80 RECFM=F NREC=41
File Format: EBCDIC
DSN:NSS.PSASAIMP.IMAGV.DATA(ROTGAC)
DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=32
File Format: EBCDIC
DSN:NSS.PSANAVMP.HOLD.GJICQ1-GJICQ4
DCBs:LRECL=22528 BLKSIZE=22528 RECFM=F
File Format: Binary
DSN: NSS.PSANAVMP.ONAME.LJIC
DCBs: LRECL=80 BLKSIZE=80 RECFM=F NREC=1
File Format: EBCDIC
DSN: COM.PSANAVMP.RSTATUS.LJIC
DCBs: LRECL=80 BLKSIZE=80 RECFM=F NREC=17
File Format: EBCDIC
DSN: NSS.PSASAIMP.IMAGV.DATA(ROTLAC)
DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=8
File Format: EBCDIC
DSN:NSS.PSANAVMP.HOLD.LJIC
DCBs:LRECL=22528 BLKSIZE=22528 RECFM=F
File Format: Binary
DSN: NSS.PSANAVMP.ONAME.HJIC
DCBs: LRECL=80 BLKSIZE=80 RECFM=F NREC=1
File Format: EBCDIC
DSN: COM.PSANAVMP.RSTATUS.HJIC
DCBs: LRECL=80 BLKSIZE=80 RECFM=F NREC=17
File Format: EBCDIC
DSN: NSS.PSASAIMP.IMAGV.DATA(ROTHRP)
DCBs: LRECL=80 BLKSIZE=3120 RECFM=FB NREC=8
File Format: EBCDIC
DSN:NSS.PSANAVMP.HOLD.HJIC
DCBs : LRECL=22528 BLKSIZE=22528 RECFM=F
File Format: Binary
5.0 OPERATIONAL SCENARIO
5.3 JOB STRUCTURE
5.3.1 OCEAN PRODUCTS
STEP 1 : INITIALIZE MAP DATABASE IF NO COMPOSITE OR END OF
COMPOSITE
STEP 2 : EXECUTE IMAGE MAPPER (IMGMAP)
2.1 If the orbit does not cover the target area then
skip all subsequent steps and exit.
2.2 Process data over target area.
2.3 Implant Graphics (optional)
2.4 Save processed data to Map Database.
STEP 3 : INITIALIZE SST IMAGE FILE
STEP 1 : EXECUTE IMAGE MAPPER (IMGMAP)
1.1 Process data from each orbit and save processed
data to map database.
STEP 2 : INITIALIZE DAILY MASTER MAPS
STEP 3: COPY THE CONTENTS OF MAP DATABASE TO DAILY MASTER
MAPS
STEP 3.1 : ARCHIVE DAILY MASTER MAPS (to be developed)
STEP 4: INITIALIZE MAP DATABASE
STEP 5: PRODUCE WEEKLY COMPOSITE MAPS (VEGINDX)
5.1 : Composite daily master maps and save the composite to weekly
composite maps.
STEP 6: PRODUCE VEGETATION INDEX MAPS (WKLYCOMP)
6.1: Initialize Vegetation Index maps
6.2: Compute Vegetation Index from weekly composite and
store to Vegetation Index maps
6.3: Initialize weekly composite maps
STEP 1: INITIALIZE MAP DATABASE
STEP 2: EXECUTE IMAGE MAPPER (IMGMAP)
1.1 Save mapped data to Map database.
STEP 3: FILE TRANSFER MAP DATABASE FILES TO HOLD FILES (FTP FROM
DPSS TO NAS)
STEP 4: UPLOAD HOLD FILES TO JIC ROTATIONAL DATABASE
(GORBSTAT,LORBSTAT,HORBSTAT)
STEP 5: TRANSFER ROTATIONAL DATABASE FILES TO DIFAS
WORKSTATION (ETHERNET)
5.3.4.2 COASTWATCH REGION PROCESSING
STEP 1: CHECK THE ORBIT TO ENSURE IT COVERS THE REGION
STEP 2: INITIALIZE MAP DATABASE
STEP 3: EXECUTE IMAGE MAPPER (IMGMAP)
* Controls are to be provided in the control file produced in
preliminary step 4. No graphic processing should be requested
and hence no region graphics file need be supplied.
* Save processed data to temporary Map Database initialized in
step 1.
STEP 4: EXECUTE THE COASTWATCH PROGRAM (OCNMAP)
* Process the data passed from step 2 in the temporary Map
Database. Control parameters are to be provided in the control file
produced in preliminary step 4.
* The stand-alone graphics files for each sector must be defined
on the appropriate DD cards.
* A file (referred to as the transfer file) containing the percentage
of non-zero pixels in each of the sector images is created and
passed to the next step.
STEP 5: INITIATE THE TRANSFER PROCESS
* The transfer file is read and the information therein is used to
control the construction of a JCL deck which will cause the transfer
of the sector images to various users.
* Write the final transfer JCL to the internal reader to spawn a
new job to cause the actual data transfer to take place.
6.0 RESOURCE REQUIREMENTS
The following topics are discussed in this section:
1. Storage requirements
2. Computer resources
3. Intercomputer Communications
4. Impact of the Image Product System on computer resources.
5. Performance statistics
6.1 STORAGE REQUIREMENTS
1. SOUTHEAST US COAST
2. GREAT LAKES
3. NORTHEAST
4. GULF OF MEXICO
5. WEST COAST
6. CARIBBEAN
7. ALASKA
8. HAWAII
1 Region File
1 Region Graphics File
n Sector Graphics Files
m Compressed Sector Image Files
p Uncompressed Sector Image Files
q Cloud Mask Images
where n is the total number of sectors and where m, p & q are
totals over all the sectors of the region.
In the following sizes are given in bytes:
Size of Region File =
(rows+1) x columns x 2 bytes/pixel x
(number_of_channels + number_of_ancillaries)
Notes:
1. Generally the region file is temporary, i.e., it
only need be retained for the duration of the job.
2. We add 1 to rows to account for the header.
3. 2-byte pixels are standard for CoastWatch
purposes.
Size of Region Graphics File = rows x columns
Size of Compressed Sector Images =
(rows+1) x columns x 2 bytes/pixel x compression_factor
Size of Compressed Stand-alone Sector Graphics =
(rows+1) x columns x compression_factor
Size of Uncompressed Sector Images =
(rows+1) x columns x pixel_size
GAC:
Number of daytime maps
(Channels 1-5,scan time, scan angle,
solar zenith angle, relative azimuth angle) 9
Number of nighttime maps
(Channels 3,4,5, scan time, scan angle,
solar zenith angle, relative azimuth angle) 7
Total maps per hemisphere 16
Total maps per operational satellite system 32
Map Resolution 1024 x 1024
Data precision: 1-byte for
channel data
and two bytes for
ancillary data
Map database requirement 48M
Daily Master Maps requirement 48M
Weekly Composite files 26M
Vegetation Index requirement 8M
Total requirement for GAC 130M
LAC/HRPT: No requirement
AMSU-A : TBD
AMSU-B: TBD
6.1.3 JOINT ICE CENTER PRODUCTS
GAC:
Number of daytime maps
(Channels 1,4) 2
Number of nighttime maps
(Channel 4) 1
Total maps per hemisphere 3
Total maps per operational satellite system 6
Map resolution 1024 x 1024
Data precision: 1-byte
Total requirements for GAC 6M
LAC/HRPT:
Maximum Number of maps per hemisphere 2
Total per operational satellite system 4
Map resolution 2048 x 2048
Data precision: 1-byte
Total requirement for LAC/HRPT 16M
AMSU-A : No requirement
AMSU-B: No requirement
Total requirement for JIC (DPSS) 22M
Hold files (NAS) 22M
Rotational Data Base (NAS) 176M
(Eight orbits each of GHRR,LHRR, and HRPT)
6.1.4 OTHER REQUIREMENTS
Software libraries and data files 25M
6.2 COMPUTER RESOURCES
RAM 640 KB
Harddisk 40MB
Floppy disk drives 1.2 MB
NUMBER NINE REVOLUTION 512 X 512 X 32 Graphics board
A high resolution color monitor for image display
A color monitor for text display
Microsoft mouse
Epson dot matrix printer/hp 7475a plotter
Hayes 1200/2400 bps modem
IRMA 3278 Emulator board
High resolution color monitor for image display.
Number Nine Corporation Revolution 512 X 512 X 32 graphics board.
A color monitor for text display.
Microsoft mouse.
Hayes compatible modem.
IRMA 3278 Emulator board.
Group III facsimile machine.
Hard copy device(s).
6.3 INTERCOMPUTER COMMUNICATIONS
6.4 IMPACT OF THE IMAGE PRODUCT SYSTEM ON COMPUTER
RESOURCES
6.5 PERFORMANCE STATISTICS
STEP CPU CLOCK EXCPS
-------- ------- ------- ---------
PREF1 00:01 0:28 655
IMGMAP 11:52 26:58 13,261
OCNMAP 11:25 16:22 1,839
7.0 MAINTENANCE AND MONITORING
Privacy Policy
| Disclaimer