Use the LDAD Install Manual for initial system installation and setup. The installation includes some default configuration files that require site-specific information. This section provides the information necessary to localize the configuration files for the following modules:
Environment variables are defined in ~ldad/.environs, to wit:
# here's the original set of environment definitions
FXA_DATA /data/fxa
FXA_NATL_CONFIG_DATA /awips/fxa/data/localization
# required Informix variables
SQLEXEC sqlrm
INFORMIXDIR /opt/informix
#INFORMIXDIR /opt/informix
ONCONFIG onconfig.ds1
INFORMIXSERVER ONLINE
INFORMIXC '/opt/ansic/bin/cc -Aa'
# path to LDAD "root" directrories
LDAD_INTERNAL_HOME /awips/ldad
LDAD_EXTERNAL_HOME /ldad
# path to "Raw" data directory
LDAD_RAWDATA ${FXA_DATA}/LDAD/Raw
# path to log file directories
LDAD_INTERNAL_LOGDIR ${LDAD_HOME}/logs
LDAD_EXTERNAL_LOGDIR /data/logs/ldad
# Set the name of the LDAD Internal Database name
NEWGATEWAY_DB ldadgateway
# path to the External Incoming Directory
LDAD_INCOMING /data/Incoming
# Hostnames
LDAD_INTERNAL_HOST ds
LDAD_EXTERNAL_HOST ls1
# begin the additional set of environment variables
# FXA_HOME needs to be defined
# FXA_MASTER_BIN needs to be defined
# path to LDAD data directrories
# Note: LDAD_INTERNAL_DATA is also defined in ~fxa/.environs
# make sure you keep 'em in sync!
LDAD_INTERNAL_DATA ${LDAD_HOME}/data
LDAD_EXTERNAL_DATA /data/ldad
# extends WFO default path
FILE_SERVER_DEFAULT_PATHS +${LDAD_HOME}/data/
# path to LDAD bin directories
LDAD_INTERNAL_BIN ${LDAD_INTERNAL_HOME}/bin
LDAD_EXTERNAL_BIN ${LDAD_EXTERNAL_HOME}/bin
# path to LDAD source code directories
LDAD_INTERNAL_SOURCE_DIR ${LDAD_HOME}/src
LDAD_DECODER_SOURCE_DIR ${LDAD_INTERNAL_SOURCE_DIR}/decoder
LDAD_STORAGE_SOURCE_DIR ${LDAD_DECODER_SOURCE_DIR}/storage
LDAD_GATEWAY_SOURCE_DIR ${LDAD_INTERNAL_SOURCE_DIR}/gateway
# path to the external ldad data repository
LDAD_EXTERNAL_DATA_REPOSITORY ${LDAD_EXTERNAL_DATA}/hmIngest
# LDAD decoded Data path
LDAD_DECODED_DATA ${FXA_DATA}/LDAD/decoded
# path to the LDAD crontab files
LDAD_INTERNAL_CRONTABS ${LDAD_INTERNAL_BIN}/crontabs
LDAD_INTERNAL_SYSTEM_CRONTABFILE /usr/spool/cron/crontabs/ldad
LDAD_INTERNAL_CRONTABFILE ${LDAD_INTERNAL_BIN}/ldad.crontab
# path to the LDAD external public files
LDAD_EXTERNAL_PUBLIC ${LDAD_EXTERNAL_DATA}/public
# path to LDAD data files
LDAD_DATA ${FXA_DATA}/LDAD
# path to LDAD Java Class files
CLASSPATH ${LDAD_HOME}/classes:.:${LDAD_EXTERNAL_DATA}/emwww/htdocs/Base64Convert.jar
# local timezone
LDAD_LOCAL_TZ MST7MDT
# Udunits Path
# Note: This is also defined in ~fxa/.environs
UDUNITS_PATH ${LDAD_INTERNAL_DATA}
# needed for SHEF encoder
# Note: This is also defined in ~fxa/.environs
APPS_DEFAULTS_USER /awips/hydroapps/.Apps_defaults
#LOG Prefs etc
LOG_DIR /data/logs/ldad
LOG_PREF ${LDAD_HOME}/.logPref
# The following 4 environment variables will need to be localized for each WFO.
# FXA_LOCAL_SITE might conceivably be the same as DEFAULT_SITE. If there is no
# DEFAULT RFC then this environment variable must be omitted altogether. All
# environment variables must have a translation. Otherwise, the code will
# crash.
# Local Site ID
FXA_LOCAL_SITE BOU
# Default State
DEFAULT_STATE CO
# Default WFO
DEFAULT_SITE BOU
# Default RFC
DEFAULT_RFC MKC
|
The Informix triggers mechanism is used by LDAD to forward text products to the external server (ls1) for dissemination.
LDAD Triggers are set during localization, using $FXA_HOME/data/localization/LLL/LLL-ldadSiteConfig.txt in conjunction with $FXA_HOME/data/localization/nationalData/ldadTrigger.template. (LLL is the local site ID.) Site backup triggers are also set, using LLL-ldadSiteBackupConfig.txt with ldadSiteBackupTrigger.template. (In most cases, you must create the backup config file yourself - it's not delivered with the AWIPS software.)
See the AWIPS triggers write-up for more information on configuring and installing triggers.
The required ls1:/ldad/bin directory files are: tmain, CO_serv,
sendLDADnotification.pl, startLDADexternal.csh, rb, rc, rx, rz, sb,
sx, and sz. The latter seven files are the XYZ/Modem software commands
for file transfers. It is assumed that the following "standard" UNIX
commands are installed in /usr/bin: vi, mailx, and kermit. BBS menu
and help files tmenu.mb, tmenuhelpmain.txt, and tmenuhelpvi.txt are
found in /ldad/data. The BBS executable and configuration files must
have at a minimum the following permission settings:
| Executable | Permissions | Configuration/Help File | Permissions | |
|---|---|---|---|---|
| tmain | -rwxr-xr-x | tmenu.mb | -rw-r-r-- | |
| rb, rc, rx, rz | -rwxr-xr-x | tmenuhelpmain.txt | -rw-r-r-- | |
| sb, sx, sz | -rwxr-xr-x | tmenuhelpvi.txt | -rw-r-r-- |
World execute permission is needed on the executables listed above because tmain is invoked when an external user logs into the LDAD BBS through his or her assigned UNIX account. Since tmain is not started from a shell, it does not use environment variables. Thus, all the directory pathnames that it uses are hard coded. Any Interprocess Communication (IPC) performed by tmain is to CO_serv, which is assumed always to be running on the LDAD Server.
The standard allowed login time for the external user is 15 minutes. If needed, the LDAD system administrator can create a file named .timeout in the external user's login home directory to override this default. This file contains a single integer representing the number of minutes the user may stay on the system.
The menus shown to the external users are configurable through a menu definition file named ls1:/ldad/data/tmenu.mb. A sample portion of this file is shown below.
###############################################################
# Menu Configuration File
# Each menu item has the following struct:
#
# <Type> - PULLDOWN, BUTTON (no difference here), or SEPARATOR,LABEL
# <Name> - text for menu option
# <Mnemonics> - key needed to get to this selection on the menu
# <Action> - product filename or directory containing displayable files
# or ERROR for a "not implemented" message to user
#
#BEGIN MENU DESCRIPTION
Begin
PULLDOWN
Watches/Warnings/Advisories/Statements
W
NULL
Begin
PULLDOWN
Aviation User's Menu
A
/data/ldad/public/nwswwas
Begin
# NOTE: this next type applies to any item that may be downloaded with
# the d command. The file itself is on the last line for this menu item
# e.g. wwwAWWddd1.txt
#
# NOTE: The filenames follow the AFOS 9 char form plus a 1.txt appended to it
# i.e. CCCNNNXXX1.txt
# The variables www (WFO), rrr(RFC) will be changed by either the localization or
# at runtime.
# The variables ddd(WFO) and eee(STATE) will be changed at runtime based upon
# user input if necessary. The default will be obtained from environment
# variables.
#
# NOTE: Please make sure that the appropriate filename tags are typed
# for the filenames.
BUTTON_DOWNLOAD
Airport Weather Warning
A
wwwAWWddd1.txt
BUTTON_DOWNLOAD
Tornado Warning
D
wwwTORddd1.txt
... [lines omitted]
PULLDOWN
Aviation Area Forecasts
v
/data/ldad/public/nwswwas
Begin
BUTTON_DOWNLOAD
Aviation Area Forecasts Part 1
1
wwwFA1ddd1.txt
...
SEPARATOR
End
PULLDOWN
Winds Aloft Forecasts
W
/data/ldad/public/nwswwas
Begin
...
PULLDOWN
Hydro Observations
y
NULL
Begin
PULLDOWN
Products beginning with AA
A
/data/ldad/public/point/hydro
Begin
BUTTON_DOWNLOAD
# NOTE: the = character below is so that no mnemonic (fast jump character)
# is used for this menu item
AANG1 - Pchtr Creek, GA
=
AANG1.txt
BUTTON_DOWNLOAD
ABNG1 - Albany, GA
=
ABNG1.txt
...
End
PULLDOWN
Mesonet Observations
M
/data/ldad/public/point/mesonet
Begin
BUTTON
No Products Available Currently
N
# NOTE: NOT_IMPLEMENTED in the action field displays a "feature not implemented"
message
NOT_IMPLEMENTED
BUTTON
Enter b or <- to return to previous menu
N
NOT_IMPLEMENTED
SEPARATOR
End
PULLDOWN
For Data Providers
i
NULL
Begin
PULLDOWN
CDOT QC
C
/data/ldad/public/qcmessages
Begin
BUTTON_DOWNLOAD
006QCH.txt - Hourly Summary File
0
006QCHddd1.txt
...
PULLDOWN
User Utilities
s
NULL
Begin
#
# Each site must update the target state and WFO ids
# NOTE : Each site should put the default on as a choice for users
# to be able to back ti it.
# NOTE : As a SOP we will use the first choice as the default.
# The samples below are for the DEN/BOU Area, customize this file.
PULLDOWN
Select Target State
S
SetSTATE
Begin
BUTTON
Colorado (CO)
C
SetState
BUTTON
Kansas (KS)
K
SetState
...
End
PULLDOWN
Select Target WFO
W
SetWFO
Begin
BUTTON
Denver (DEN)
D
SetWfo
BUTTON
Boulder (BOU)
B
SetWfo
...
End
BUTTON
Send E-mail
E
Send_EMAIL
SEPARATOR
End
SEPARATOR
End
# END MENU DESCRIPTION
|
The hierarchical menus are bracketed with Begin/End pairs which are nested for sub-menu definitions. Lines beginning with a # character are ignored and can be used to comment or document items in the file. Empty lines are also ignored. Please note that you should have a maximum of 10 menu item entries in a page. If there are more than 10 entries for a particular category, then a separate page is required for each subset of 10. See the Hydrological User's Menu A and B for an example of this breakdown.
Each definable menu item consists of four parts: A predefined type, a name, a mnemonic character, and an action. The four types are used as follows:
PULLDOWN - used to define a pulldown menu which contains other submenu items BUTTON_DOWNLOAD - used to define a viewable and downloadable menu item BUTTON - used to define an action item that may result in a particular function being invoked SEPARATOR - to display a separator line between the last menu item and the selection choice line
The name component of a menu item definition is simply the verbatim name displayed on the screen for this menu item. The mnemonic character, which should be unique for a given menu screen, is a way for the user to quickly jump to this menu item using a single character. The action can specify one of four different things. For PULLDOWN types it can be used to specify the full pathname of the directory where the submenu items representing viewable file information are stored. For BUTTON_DOWNLOAD items it specifies the file name of the viewable or downloadable file (in the PULLDOWN defined directory). For BUTTON items it may specify an action item which results in a particular function being invoked. The SEPARATOR type (if used) must immediately precede an "End" keyword which terminates the definition of a submenu (or the main menu). The details for the entries for the various sections are listed below.
The beginning of this section includes four entries in tmenu.mb: top level section phrase, a mnemonic, a directory name, and the word Begin. Each menu item also requires four entries. The entries for the Area Weather Update item in the Emergency Management User's Menu A are described in detail below.
Emergency Management User's Menu A- the top level section phrase
E- the mnemonic/key that the user can type to select this section
/data/ldad/public/nwswwas- the directory where the datafiles are located for this section
Begin- start of a new section
BUTTON_DOWNLOAD- This signifies that the menu entry is a downloadable item. Clients can press the d key and the BBS system will automatically download the product onto the client's system using the preset protocol (XMODEM, YMODEM, or KERMIT)
Area Weather Update- the exact phrase that will be displayed on the screen
U- the mnemonic/key that the user can type to select this product
DENAWUddd1.txt- the data file that will be dumped to the client
- [additional BUTTON_DOWNLOAD groups here]
End- End of the section
The beginning of this section includes four entries in tmenu.mb: top level section phrase, a mnemonic, a directory name, and the word Begin. Each section can have multiple entries or menu items. Each menu item also requires four entries. For example, the entries for the Pchtr Creek, GA gauge item in the Hydro Observation's Products beginning with AA are described in detail below.
Products beginning with AA- The top level section phrase. This statement could be changed to "Gauges between A-G" if the Automatic LARC gauge dial-out is set up to dial out to up to 10 gauges with names beginning with A-G. Otherwise divide out the sections to where you have at most 10 gauges for each section.
A- the mnemonic/key that the user can type to select this section
/data/ldad/public/hydro- the directory where the datafiles are located for this section
Begin- start of a new section
BUTTON_DOWNLOAD- This signifies that the menu entry is a dowloadable item. Clients can press the d key and the BBS system will automatically download the product onto the client's system using the preset protocol (XMODEM, YMODEM, or KERMIT)
AANG1 - Pchtr Creek, GA- the exact phrase that will be displayed on the screen
=- no mnemonic (fast jump character)
AANG1.txt- the data file that will be dumped to the client
End- end of the section
The beginning of this section includes four entries: top level section phrase, a mnemonic, a directory name, and the word Begin. Each section can have multiple entries or menu items. Each item also requires four entries. For example, the entries for the Hourly Summary QC data for the first data provider registered to the QC System are described in detail below.
For Data Providers- The top level section phrase
I- the mnemonic/key that the user can type to select this section
NULL- The directory where the datafiles are located for this section. This is set to
NULLbecause the directory information is available from the respective QC sub-section as shown below.Begin- start of a new section
PULLDOWN- this signifies that selecting this item moves the client the next section
CDOT QC- The top level section phrase. Each WFO should change this name with the appropriate information for the 6th provider from the LDAD QC system file /awips/fxa/ldad/MSAS/fslparms/sfchqcin.dat.
C- the mnemonic/key that the user can type to select this section
/data/ldad/public/qcmessages- the directory where the datafiles are located for this section
Begin- start of a new section
BUTTON_DOWNLOAD- This signifies that the menu entry is a dowloadable item. Clients can press the d key and the BBS system will automatically download the product onto the client's system using the preset protocol (XMODEM, YMODEM, or KERMIT).
006QCH.txt - Hourly Summary File- the exact phrase that will be displayed on the screen
0- the mnemonic/key that the user can type to select the product
006QCHddd1.txt- the data file that will be dumped to the client
End- end of the section
The tmain process writes log messages to file /data/logs/ldad/<yyyymmdd>/tmain<pid><host><hhmmss>. The system administrator should periodically look at this file for any serious error messages. The only other logging associated with the BBS is by the CO_serv process, which logs a "P" message (signifying a request for upload/download protocol) followed by a "B" message (signifying sending of the BBS port info) when tmain starts up.
A BBS menu item (under User Utilities) allows users to send e-mail to the LDAD system administrator. The message will go to all recipients listed in file /data/ldad/public/sysop.address on the LDAD server. To edit the list, select "External e-mail address" from the Edit LDAD Configuration Files area of the LDAD System Administration page (Section 8.1.1).
| Pre-Processor Name | Data Type |
|---|---|
| preProcessLDAD.pl | Wrapper, calls appropriate preprocessor based on data type. Acceptable data types are ALERT_R5, ALERT_WX, ALERT_WL, CDOT, LARC, IFLOWS (STREAM, RAINDATA), and MESONET. |
| preprocessIFLOWS.pl * | IFLOWS data, Rain and Stream data |
| preprocessRRS.pl | WorkStation Automated Radio-Theodolite/Radiosonde Replacement System (WsART/RRS) data |
| preprocessSUA.pl | ASOS and MicroART data containing surface and upper air (SUA) obs |
| preprocessLARC.pl * | national LARC gauges, of format HANDAR_coll_sess_1, 2, 12, & 21 |
| preprocessmesonet.pl * | generic mesonet data |
| preProcessWAN_NWWS.pl | generic - routes products to WAN and NWWS uplink |
| preprocessShefAlert.pl | generic - forwards first and last fields to output file |
| preprocessCAMPBELL.pl | Campbell data files |
| preprocessDTMF.pl | DTMF-transmitted ROSA data files |
| preprocessCDOT.pl * | Colorado Dept. of Transportation |
| preprocessALERT.pl * | Denver Urban Drainage ALERT _r5, _wl, and _wx data |
Site managers can either write a whole new pre-processor independent of the wrapper or alternatively, use the wrapper and some additional utilities in LdadUtils.pl (see Appendix C) to create additional pre-processors. It is suggested that site managers use one of the current pre-processors as a template for their own. Note that all pre-processors need to be placed in the $LDAD_HOME/bin directory on both ds1 and ds2 (for failover support).
For ASCII data files with some form of delimiter, the pre-processor need only reformat the date field and change the delimiter to be a comma. Any line beginning with a # symbol will be treated as a comment and ignored.
A number of configuration files are used for the decoder and storage classes. These ASCII files, found in $LDAD_INTERNAL_DATA, are described briefly below.
See Appendix A for an outline of how to use these files to add a new LDAD CSV data source.
The LDADinfo.txt file includes one row for each data type to be decoded, as shown below. Fields are delimited by | (the pipe symbol).
Note that the MSAS QC entries are a special case. They are here to accommodate passing them to the WHFS database via the SHEF encoder and decoder.
This is the default LDADinfo.txt file delivered with Build 5.1.1.
# LDADinfo.txt # # # MSAS QC filenames must occur BEFORE the data type they reference # # QC data Station.txt # .desc file file #-------------- ----- msas_qc.alert_wx | ALERT | msas_qc.alert_wl | ALERT | msas_qc.alert_r5 | ALERT | msas_qc.LARC | LARC | msas_qc.RAWS | RAWS | msas_qc.SCHLNET | SCHLNET | msas_qc.internet | INTERNET | # #data Station netCDF|plot LDAD storage preprocess #type .txt dataKey type directory script #------- ----- ---- --- -------- --------- ----------------- alert_wx |ALERT |87 |86 |CVS_TYPE |mesonet |preProcessLDAD.pl | alert_wl |ALERT |89 |90 |CVS_TYPE |hydro |preProcessLDAD.pl | alert_r5 |ALERT |89 |90 |CVS_TYPE |hydro |preProcessLDAD.pl | cdot |CDOT |87 |86 |CVS_TYPE |mesonet |preProcessLDAD.pl | iflow_ky |IFLOWS |89 |90 |CVS_TYPE |hydro |preProcessLDAD.pl | iflow_s |IFLOWS |89 |90 |CVS_TYPE |hydro |preProcessLDAD.pl | LARC |LARC |89 |90 |CVS_TYPE |hydro |preProcessLDAD.pl | ROSA |ROSA |88 | |CVS_TYPE |manual | | # CAMPBELL |CAMPBELL| 89 |90 |CVS_TYPE | hydro |preprocessCAMPBELL.pl| SCHLNET |SCHLNET | 87 |90 |CVS_TYPE | mesonet | moveSCHLNETproduct.pl| RAWS |RAWS | 87 |90 |CVS_TYPE | mesonet | moveRaws.pl | # # RRS | | | | | |preprocessRRS.pl | testIPRCP|IFLOWS |89 |90 |CVS_TYPE |hydro | | TSTMTRTST|ROSA |88 | |CVS_TYPE | |preProcessTextDB_FSL.pl -i TSTMTRTST -s /awips/fxa/ldad/bin/distProduct.pl| wwwRR3www|ROSA |88 | |CVS_TYPE | |preProcessTextDB_FSL.pl -i wwwRR3www -s /awips/fxa/ldad/bin/distProduct.pl| wwwRR4www|ROSA |88 | |CVS_TYPE | |preProcessTextDB_FSL.pl -i wwwRR4www -s /awips/fxa/ldad/bin/distProduct.pl| SUA |SUA |256 |257 |CVS_TYPE |mesonet |preprocessSUA.pl | WAN_NWWS |OUP |256 |257 |CVS_TYPE |mesonet |preProcessWAN_NWWS.pl | # # #mm5 |mm5 |87 |86 |CVS_TYPE | mesonet |getAFile.csh | #testMESONET|testMESONET|87|86 |CVS_TYPE |mesonet |testMESONET.pl | # # use key 87 for netcdf mesonet data files # use key 88 for netcdf manual data files # use key 89 for netcdf hydro data files # |
(Note that the LDAD type string really is CVS_TYPE, not CSV, as you might expect. There's a typo in the processing code.)
LDADProvider.txt includes information about each data provider. It's used to provide a lookup for contact information in the data acquisition and dissemination monitors. The file consists of one row for each data provider. The sample file delivered with 5.1.1 is shown here. Of course, it will need to be significantly modified at each field site, using the LDAD Administration GUI. (Since the first two metadata lines are quite long, we've split them after the phone number to reduce horizontal scrolling. In the actual file, each set of metadata is on a single line.)
# This file contains the data provider metadata for the LDAD data sources. # Each "Check" line in lda.conf and ldd.conf refers to one of these entries, # and the information is displayed when the user clicks on a Data link in the # acquisition or dissemination monitor. # # Record format: # Dummy|ProviderKey|ProviderName|Contact|Phone1|Phone2|Fax|Street|City|State|ZIP|e-mail # Dummy: This field is no longer used; just make it a space. # ProviderKey: The provider of the data. This key is used in lda/ldd.conf. # ProviderName: The full name of the provider organization # Contact: Point of contact in the organization # Phone1: Primary phone number # Phone2: Secondary phone number # Fax Fax number # Street: Street address # City: City # State: State # ZIP: ZIP Code # e-mail: e-mail Address # |UDFCD|Denver Urban Drainage and Flood Control District|Kevin Stewart|3034556277|| 3034557880|2480 W 26th Ave. Suite 156B|Denver|CO|80211|kstewart@udfcd.org |CDoT|Colorado Department of Transportation|Ron Sprengeler|3037579800|| 3037579727|4201 E. Arkansas Avenue, Rm 212|Denver|CO|80222|ron.sprengeler@dot.state.co.us |WSFO-LMK|Weather Forecast Office at LMK|John Bolinger|5029698842|||||||john.bollinger@noaa.gov |FX|Internal AWIPS|Chandran Subramaniam|3034976015|||||||cvsubra@fsl.noaa.gov |NWSRAWS|RAWS from Denver WFO|Eric Thaler|3033610674|||||||eric.thaler@noaa.gov |
The [DataID]Station.txt file is organized as a series of rows, one for each station. The first several lines of RAWSStation.txt are shown here, as an example. This file is delivered with 5.1.1 and can serve as a template.
# Station Table format: # # ProviderID|AFOS(HB5)ID|StationName|Elevation|Latitude|Longitude|dataTZ| # LocationDesc|StationType|NumInst|NumLevels|Maint/CalSch|SiteDesc| # # ProviderID - Data provider station ID # AFOS(HB5)ID - The AFOS or Handbook 5 ID (NWS assigned) # StationName - Text name of station # Elevation - Elevation of station (m) # Latitude - Latitude of station (decimal degrees, north positive) # Longitude - Longitude of station (decimal degrees, east positive) # dataTZ - Reporting time zone (e.g. UTC, PST8PDT) # LocationDesc- Location/address of station (Text) # StationType - Type of station (tower, surface, floating platform, etc.) # NumInst - The number of reporting instruments for the station # NumLevels - Number of Reporting Levels for the data (Level information # should be provided in the instrument table.) # Maint/CalSch- Frequency of Maintenance/Calibration # SiteDesc - A text description of the site surroundings # # *Note* Use '#' in column 1 of any of the rows below to comment out the entry. # 32445380 | 32445 | ALASKA PORTABLE #2 | 74.676 | 64.92417 | -147.59887 | UTC | ALASKA PORTABLE #2 , AK ||1|||| 12614702 | 12614 | ALCAN HWY MI-1244 | 548.6 | 62.81667 | -141.46667 | UTC | ALCAN HWY MI-1244 , AK ||1|||| 1260C3EC | 1260C | BEAR MTN. | 586.7 | 66.05944 | -142.76944 | UTC | BEAR MTN. , AK ||1|||| FA6026DA | FA602 | BEN CREEK AIRSTRIP | 563.9 | 65.28333 | -143.06667 | UTC | BEN CREEK AIRSTRIP , AK ||1|||| |
Station files are read during the -station localization process, to produce the .spi control files for displaying LDAD data. The entries in column 2 will appear on the map. Whenever a station file is modified, you'll need to run, as user fxa, ~ldad/bin/relocalizeForLDAD.sh on the ds, or ~/data/localization/scripts/mainScript.csh -station on each workstation, and restart D2D, to see the effects.
As illustrated in the example below, the [DataType].desc file provides information on the content of the [DataType] report files. This includes name, type, and units of each data item in the CSV data file. Note that it describes the format of the file after any preprocessor is run on it.
# alert_wl.desc # # Data Source | Missing | Data Provider # # -The Data Source must match one of the entries in the first field # of LDADinfo.txt in order for the data to be decoded and stored. # -Multiple "missing" strings may be separated by commas or whitespace. # -The Data Provider field will be used to identify the data in the # netCDF files. # alert_wl | -999 | UDFCD # ---- Header Group ---- dataSourceDescription | STRING | STRING | STRING reportTime | DATE_TIME | DATE_TIME_STRING | ABSTIME timeZone | STRING | STRING | STRING timeStep | STRING | STRING | STRING # ---- Data Group ------ #netcdf Var Name | data type | input units | stored units providerId | STRING | STRING | STRING observationTime | DATE_TIME | DATE_TIME_STRING | ABSTIME riverStage | FLOAT | ft | m riverFlow | FLOAT | ft3/s | m3/s stationId | STRING | STRING | STRING |
Notes:
A sample alert_wl dataset is shown here. Note the correspondence between the data descriptions and the CSV reports.
ALERT Water Level Report 99/08/05 14:17:02 GMT, Units: FEET WLID, Last Report , Stage , Flow , SHEF 103 , 99/08/05 10:37:34, 21.49 , 78 , CRTC2 113 , 99/08/05 13:31:14, 44.40 , 0 , RALC2 123 , 99/08/05 14:15:47, 1.05 , 14 , CRPC2 133 , 99/08/05 12:25:21, 1.16 , 41 , SMMC2 203 , 99/08/04 21:37:53, -0.04 , 0 , LEYC2 213 , 99/08/05 13:51:11, 1.04 , 22 , LCNC2 323 , 99/08/05 05:19:15, 2.54 , 296 , SCXC2 333 , 99/08/05 10:27:45, 3.76 , 584 , VBCC2 |
Whenever a .desc file is modified, it's necessary to restart the decode and store processes to effect the changes. Use the LDAD Admin page on the monitor, or, on ds1 as user ldad, type stopLdadIngest and startLdadIngest.ds1.
This file governs how the raw text messages (before being passed through any preprocessor specified in LDADinfo.txt) are stored in the text database. The file is found on the data server in ~fxa/data, and can be edited through the LDAD Administration GUI. The default file delivered with Build 5.1.1 is shown here. Local changes will be needed, obviously, including adjusting the WMO ID to something appropriate for your location.
#LDAD Product ID | WMO ID | AWIPS ID # | TTAAii CCCC | nnnxxx # -------------- | ------ ---- | ------ alert_wx | SXUS55 KBOU | LCOWX alert_wl | SXUS55 KBOU | LCOWL alert_r5 | SXUS55 KBOU | LCOR5 alertfcl_wx | SXUS55 KBOU | LCOFWX alertfcl_wl | SXUS55 KBOU | LCOFWL alertfcl_r5 | SXUS55 KBOU | LCOFR5 rosa | SXUS55 KBOU | LCORSA CDOT | SXUS55 KBOU | LCOCDT RAWS | SXUS55 KBOU | LCORAW utmesnet | SXUS55 KBOU | LCOUTM SCHLNET | SXUS55 KBOU | LCOSCH internet | SXUS55 KBOU | LCOINT # MSAS products msas_qc.alert_wx | SXUS55 KBOU | LCOMWX msas_qc.alert_wl | SXUS55 KBOU | LCOMWL msas_qc.alert_r5 | SXUS55 KBOU | LCOMR5 msas_qc.LARC | SXUS55 KBOU | LCOMLC msas_qc.RAWS | SXUS55 KBOU | LCOMRW msas_qc.SCHLNET | SXUS55 KBOU | LCOMSC msas_qc.utmesnet | SXUS55 KBOU | LCOMUT |
Each entry in the first column must match one in the first column of LDADinfo.txt.
Any time this file is changed, you will need to restart the decoding/storage suite (from the GUI).
Two text files, a "reject" and an "accept" list, reside on as2 in /awips/fxa/ldad/MSAS/fslparms/reject.txt and accept.txt. These provide the capability to subjectively override the results of the automated QC checks provided by the AWIPS Quality Control and Monitoring System (QCMS) (described in the D2D User's Manual, section 10). The reject list is a list of stations and associated input observations that will be labeled as bad, regardless of the outcome of the QC checks; the accept list is the corresponding list of stations that will always be labeled as good. Applications reading the lists (e.g. the MSAS analysis described in the D2D User's Manual, section 2) will then reject or accept the stations specified. In both cases, observations associated with the stations in the lists can be individually flagged.
For example, the reject list in Figure 9.8 specifies that SLP and altimeter observations for station KP38 should be labeled as bad, whereas KP38 observations of other variables should be quality controlled normally.
QCMS QC Reject List: "1" => Flag observation as bad
StaID |Provider |SLP |Alt |Pchng |T |Td |Wind |P |VV |RR
begin--------------------------------------------------------------------------
KP38 |METAR |1 |1 |0 |0 |0 |0 |0 |0 |0
end----------------------------------------------------------------------------
Example Entries ***************************************************************
KGCC |METAR |1 |1 |0 |0 |1 |0 |0 |0 |0
WPG |METAR |0 |1 |0 |0 |0 |0 |0 |0 |1
4010 |UDFCD |0 |0 |0 |0 |0 |1 |0 |0 |0
|
The QCMS station monitoring procedures and text summary files (see Section 8.4) are not affected by the intervention lists. This allows WFO personnel to continue to monitor the performance of the stations contained in the reject and accept lists. For example, a forecaster or HMT may notice a station with wind observations that fail the QC checks a large percentage of the time, and choose to have that station added to the reject list. However, once the observation failure rate at the station falls back to near zero (possibly due to an anemometer repair), the station can be deleted from the list.
QCMS reject and accept lists may be edited by hand (with any text editor), or through the LDAD Monitoring and Control System (see next section). For each station, both station and provider names are required. Provider names are found in /awips/ldad/MSAS/fslparms/sfchqcin.dat. The first 5 entries in sfchqcin.dat (SAO, BUOY, NPN [NOAA Profiler Network], AUTO, ASOS) are "national" datasets, and always appear. Any others are added dynamically by the 5-minute QC program, qcstg1_2, based on dataProvider names found in LDAD/mesonet/netCDF files (LDAD-collected data). A blank provider name is allowed, but may result in multiple stations with the specified station name being rejected or accepted. This occurs when different data providers use the same station name.
The QCMS also provides the capability to accept or reject observations from all stations with a specified provider name. To exercize this option, fill in the provider and observation columns, but leave the StaID column blank.
Non-text datasets are sent to the ls by script pollForData.pl, which runs on the ds. (Text product dissemination is described in Section 4.2.3.) Actions are controlled by pollForData.conf, which is outlined here.
################################################################### # pollForData.conf: configuration file for the LDAD polling function # # Each line describes one dataset via 7 fields # Tag Source of data (unused) # Key Description of the data # Type "model, obs, radar, nwstext, localtext" # Source "laps, msas, ifp, maps40, ruc, eta, mesoeta, avn, mrf"; # "metar, qcmesonet, hydro, manual"; # "r1r, r2r, etc.."; (radar IDs substituted during localization) # "warning, watch, forecast, outlook, report, statement, qctext"; # "warning, watch, forecast, outlook, report, statement, qctext" # DataType "Reflectivity, 1HourRain, 3HourRain, TotalRain, Tornado" # (This field applies to radar only.) # Scale "local, national" (dissemination system display scale) # SrcDir Source directory (on ds). ::: For a file to be "seen" by pollForData.pl, it must be of the form ::: Anything.TimeStamp, where TimeStamp is yyyymmdd[[_]hhmm]. For example, ::: RUC.20000508_2100 or myFavouriteMartian.wow.20000508 or 200005082100 ::: are valid, but RUC.20000508_2100.gz or 20000508_2100.tmp are not. # Time(s) The source directory is checked at these times (minutes past # the hour) to see if the data have updated. If -1, checks at # each pollForData cycle (30 seconds). # Script "/awips/ldad/bin/sendLDADnotification.pl" # Arguments Parameters passed to the script - always 'R "KKK FFF" HHH PPP' # R means send to ls and notify hmIngest # pollForData.pl makes substitutions for the rest with the key, # file name, host, and internal port number, respectively. # DestDir Destination directory (on ls). # ################################################################## ::: Many of the lines are quite long; they have been split here for ease ::: of viewing. Each non-comment line actually starts with FSL. #LAPS Analysis FSL|model:laps:local|/data/fxa/Grid/FSL/netCDF/LAPS_Grid/LAPS|30,35,40| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/LAPS #MSAS Analysis FSL|model:msas:national|/data/fxa/Grid/FSL/netCDF/MSAS|30,35| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/MSAS # IFP Grids FSL|model:ifp:Official:national|/data/fxa/LDAD/ifp/Official|10,30| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/ifp/Official # IFP Grids FSL|model:ifp:RUC:national|/data/fxa/LDAD/ifp/RUC|30,50| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/ifp/RUC #MAPS 40 km Model #FSL|model:maps40:national|/data/fxa/Grid/FSL/netCDF/MAPS_National/40km_MAPS|30,35| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/MAPS40 #RUC Model #FSL|model:ruc:national|/data/fxa/Grid/SBN/netCDF/CONUS211/RUC|25,45| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/RUC #Eta Model #FSL|model:eta:national|/data/fxa/Grid/SBN/netCDF/CONUS211/Eta|20,45| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/Eta #Mesoeta model #FSL|model:mesoeta:national|/data/fxa/Grid/SBN/netCDF/CONUS215/MesoEta|20,45| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/MesoEta #AVN Model #FSL|model:avn:national|/data/fxa/Grid/SBN/netCDF/CONUS211/AVN|20,45| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/AVN #MRF Model #FSL|model:mrf:national|/data/fxa/Grid/SBN/netCDF/CONUS202/MRF|20,45| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/Grid/MRF #National Metar Observations FSL|obs:metar:national|/data/fxa/point/metar/netcdf|20,50| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/point/metar #Local QC-Mesonet Observation FSL|obs:qcmesonet:local|/data/fxa/LDAD/mesonet/qc|5,15,25,35,45,55| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/point/qcmesonet #Local Hydro Observation FSL|obs:hydro:local|/data/fxa/LDAD/hydro/netCDF|2,12,22,32,42,52| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/point/hydro #Local Manual Observations FSL|obs:manual:local|/data/fxa/LDAD/manual/netCDF|1,11,21,31,41,51| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/point/manual #Nexrad r1r Reflectivity FSL|radar:r1r:Reflectivity:local|/data/fxa/radar/r1r/Z/elev0_5/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r1r/Z #Nexrad r1r One Hour Precip FSL|radar:r1r:1HourRain:local|/data/fxa/radar/r1r/OHP/layer0/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r1r/OHP #Nexrad r1r Three Hour Precip FSL|radar:r1r:3HourRain:local|/data/fxa/radar/r1r/THP/layer0/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r1r/THP #Nexrad r1r Storm Total FSL|radar:r1r:TotalRain:local|/data/fxa/radar/r1r/STP/layer0/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r1r/STP #Nexrad r1r Tornado Vorticity Signature FSL|radar:r1r:Tornado:local|/data/fxa/radar/r1r/TVS|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r1r/TVS #Nexrad r2r Reflectivity FSL|radar:r2r:Reflectivity:local|/data/fxa/radar/r2r/Z/elev0_5/res1/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r2r/Z #Nexrad r2r One Hour Precip FSL|radar:r2r:1HourRain:local|/data/fxa/radar/r2r/OHP/layer0/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r2r/OHP #Nexrad r2r Three Hour Precip FSL|radar:r2r:3HourRain:local|/data/fxa/radar/r2r/THP/layer0/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r2r/THP #Nexrad r2r Storm Total FSL|radar:r2r:TotalRain:local|/data/fxa/radar/r2r/STP/layer0/res2/level16|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r2r/STP #Nexrad r2r Tornado Vorticity Signature FSL|radar:r2r:Tornado:local|/data/fxa/radar/r2r/TVS|-1| /awips/ldad/bin/sendLDADnotification.pl|R "KKK FFF" HHH PPP| /data/ldad/public/nexrad/r2r/TVS ::: The pattern above is repeated for r3r, r4r, ... , r9r to accommodate ::: up to 9 local radars. # # This part is used on ls1 by hmIngest to determine where to store text # products. Do not edit this section; all message types are listed. FSL|nwstext:warning:local|||||/data/ldad/public/nwswwas FSL|nwstext:watch:local|||||/data/ldad/public/nwswwas FSL|nwstext:forecast:local|||||/data/ldad/public/nwswwas FSL|nwstext:outlook:local|||||/data/ldad/public/nwswwas FSL|nwstext:report:local|||||/data/ldad/public/nwswwas FSL|nwstext:statement:local|||||/data/ldad/public/nwswwas FSL|nwstext:qctext:local|||||/data/ldad/public/qcmessages FSL|localtext:report:local|||||/data/ldad/public/nwswwas |
The LDAD Monitor and Control System uses configuration files for all its activities. This section lists the configuration files. The Site Administrator can modify these files from the Administration page.
The following table outlines the contents of ldadAdmin.conf on as1 and as2. The page resulting from using these definitions was shown previously as Figure 8.4.
#<LDADADMIN> # ldadAdmin.conf # ::: The default username that the process uses when it performs its ::: activities. User | ldad ::: Path to the icons directory, relative to the document root. Not used. ICONDIR | /icons ::: Log file Name LOGFILE | LDAD_Admin.log ::: Root location of all Monitor files. The rest of the directories are ::: appended to this root. ROOTDIR | /awips/fxa/htdocs/ldadMon ::: binaries BINDIR | /bin ::: /log LOGDIR | /log ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all html files are written PRODDIR | /products ::: the Debug filename DEBUGFILE | ldadAdmin.dbg ::: the Error filename ERRORFILE | ldadAdmin.err ::: the title for the Process monitor PAGETITLE|LDAD System Administration ::: place-holder icon invisible_gif | invisible.gif ::: status icons: on time, late, more late, very late, missing green_status | yes_50.gif blue_status | tardy_50.gif yellow_status | caution_50.gif red_status | no_50.gif missing_status | unknown_50.gif ::: state, info, remark, error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: The default number of columns columns | 4 ::: The maximum log size Log_size | 20971520 ::: Table Header messages if needed. motd | Help link not enabled. ::: The outage interval for the system (seconds). Outage_threshold | 600 ::: Here, we list the tables to be created on the Admin page. #!! DO NOT EDIT THIS LINE #Table|Key|Title|Columns Table|Restart|Start/Restart Processes|3 Table|Stop|Stop Processes|3 Table|Edit|Edit LDAD Configuration Files|3 Table|Monitor|Configure Monitor Tables|3 ::: Now, each of the tables is defined, in turn. ::: For Restart and Stop, each entry shows the table name, node on which ::: the process runs, full pathname of the process, name to show on the ::: Admin page, and username under which it runs. #Restart|Node|ProcessName|Description|Username Restart|ds|/awips/ldad/bin/startLDAD.csh|All LDAD Processes|ldad #Restart|ds|/awips/ldad/bin/startWatchDogProcesses.sh|LDAD Watch Process|ldad Restart|as2f|/awips/ldad/MSAS/WFOA_MSAS_Restart|MSAS System|fxa Restart|ds|/awips/fxa/bin/startLdadIngest.ds1|LDAD Decode/Store|ldad #Stop|Node|ProcessName|Description|Username Stop|ds|/awips/ldad/bin/stopLDAD.sh|All LDAD Processes|ldad #Stop|ds|/awips/ldad/bin/stopWatchDogProcesses.sh|LDAD Watch Process|ldad Stop|ds|/awips/fxa/bin/stopLdadIngest|LDAD Decode/Store|ldad ::: For Edit and Monitor, each entry lists the file to edit, paths to ::: that file, nodes on which it resides, title, and owner. The edit ::: will update both copies. ::: In general, one should not edit the contents of these lines. In ::: particular, it's important to retain the 2-node setup. Lines may ::: be added or removed, as appropriate. ::: Some of the lines are quite long, so we've split them to reduce ::: horizontal scrolling. #Edit|Config Files|path|systems|Title|Username Edit|LDADinfo.txt|/awips/ldad/data:/ldad/data|ds:ls1|LDAD Conf. Table|ldad Edit|LARC.desc|/awips/ldad/data:/ldad/data|ds:ls1|LARC desc|ldad Edit|LARCStation.txt|/awips/ldad/data:/ldad/data|ds:ls1|LARC stns|ldad Edit|ALERTStation.txt|/awips/ldad/data:/ldad/data|ds:ls1|ALERT Station|ldad Edit|alert_r5.desc|/awips/ldad/data:/ldad/data|ds:ls1|ALERT Rain Desc.|ldad Edit|alert_wl.desc|/awips/ldad/data:/ldad/data|ds:ls1|ALERT Stream Desc.|ldad Edit|alert_wx.desc|/awips/ldad/data:/ldad/data|ds:ls1|ALERT Weather Desc.|ldad Edit|iflows_s.desc|/awips/ldad/data:/ldad/data|ds:ls1|IFLOWS Stream Desc.|ldad Edit|iflows_ky.desc|/awips/ldad/data:/ldad/data|ds:ls1|IFLOWS Rain Desc.|ldad Edit|IFLOWSStation.txt|/awips/ldad/data:/ldad/data|ds:ls1|IFLOWS Station|ldad Edit|accept.txt|/awips/fxa/ldad/MSAS/fslparms:/awips/fxa/ldad/MSAS/fslparms| as1:as2|QC Accept Table|fxa Edit|reject.txt|/awips/fxa/ldad/MSAS/fslparms:/awips/fxa/ldad/MSAS/fslparms| as1:as2|QC Reject Table|fxa Edit|LDADProvider.txt|/awips/ldad/data:/ldad/data|ds:ls1|LDAD Provider List|ldad Edit|LdadPatterns.txt|/awips/ldad/data|ds|ingest data patterns|ldad Edit|sysop.address|/data/ldad/public|ls1|External e-mail address|ldad Edit|LdadWmoAwipsId.tbl|/awips/fxa/data|ds|Text storage control|fxa #View|Config Files|path|systems|Title|Username ######################################################################## #The config info below is to reconfigure the Monitor itself #Please be careful. ######################################################################## #Monitor|Config Files|path|systems|Title|Username Monitor|ldadAdmin.conf|/awips/fxa/htdocs/ldadMon/conf:/awips/fxa/htdocs/ldadMon/conf| as1:as2|Administration|fxa Monitor|ldadRestart.conf|/awips/fxa/htdocs/ldadMon/conf:/awips/fxa/htdocs/ldadMon/conf| as1:as2|Restart|fxa Monitor|proc_internal.conf|/awips/fxa/htdocs/ldadMon/conf:/awips/fxa/htdocs/ldadMon/conf| as1:as2|Internal Processes|fxa Monitor|proc_external.conf|/ldad/data|ls1|External Processes|ldad Monitor|lda.conf|/ldad/data|ls1|Data Acquisition|ldad Monitor|ldd.conf|/ldad/data|ls1|Data Dissemination|ldad Monitor|summary.conf|/awips/fxa/htdocs/ldadMon/conf:/awips/fxa/htdocs/ldadMon/conf| as1:as2|Summary|fxa |
The following table outlines the contents of ldadRestart.conf on as1 and as2. The page resulting from using this file was shown previously as Figure 8.5.
#<LDADADMIN> ::: The default username that the process uses when it performs its ::: activities. User | ldad ::: Path to the icons directory. Not used. ICONDIR | /icons ::: Log file Name LOGFILE | LDAD_Restart.log ::: Root location of all Monitor files. The rest of the directories are ::: appended to this root. ROOTDIR | /awips/fxa/htdocs/ldadMon ::: binaries BINDIR | /bin ::: /log LOGDIR | /log ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all html files are written PRODDIR | /products ::: the Debug filename DEBUGFILE | ldadRestart.dbg ::: the Error filename ERRORFILE | ldadRestart.err ::: the title for the Process monitor PAGETITLE|LDAD Restart Administration ::: place-holder icon invisible_gif | invisible.gif ::: status icons: on time, late, more late, very late, missing green_status | yes_50.gif blue_status | tardy_50.gif yellow_status | caution_50.gif red_status | no_50.gif missing_status | unknown_50.gif ::: state, info, remark, error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: The default number of columns columns | 4 ::: The maximum log size Log_size | 20971520 ::: Table Header messages if needed. motd | Help link not enabled. ::: The outage interval for the system (seconds). Outage_threshold | 600 ::: Here, we list the tables to be created on the Restart page. #!! DO NOT EDIT THIS LINE #Table|Key|Title|Columns Table|Restart|Start/Restart Processes|3 ::: Now, the table is defined. ::: For Restart, each entry shows the table name, node on which ::: the process runs, full pathname of the process, name to show on the ::: Admin page, and username under which it runs. #Restart|Node|ProcessName|Description|Username Restart|ds|/awips/ldad/bin/startLDAD.csh|All LDAD Processes|ldad #Restart|ds|/awips/ldad/bin/startWatchDogProcesses.sh|LDAD Watch Process|ldad Restart|as2f|/awips/ldad/MSAS/WFOA_MSAS_Restart|MSAS System|fxa Restart|ds|/awips/fxa/bin/startLdadIngest.ds1|LDAD Decode/Store|ldad |
#<LDADADMIN> ::: The default username that the process uses when it performs its ::: activities. User | ldad ::: Debug flag. 0=false; 1=true Debug | 1 ::: Verbose flag. 0=false; 1=true Verbose | 1 ::: Path to the icons directory. ICONDIR | /icons ::: Log file Name LOGFILE | LDAD_Process.log ::: Root location of all Monitor files. The rest of the directories are ::: appended to this root. ROOTDIR | /data/ldad/www/htdocs/ldadmon ::: binaries BINDIR | /bin ::: /log LOGDIR | /data/logs/ldad ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all html files are written PRODDIR | /products ::: the Debug filename DEBUGFILE | MakePROCpage.dbg ::: the Error filename ERRORFILE | MakePROCpage.err ::: the output html filename HTMLFILE | proc_external.html ::: the title for the External System monitor PAGETITLE | LDAD External System Monitor ::: the refresh rate (seconds) written to the HTML file PageRefresh | 60 ::: place-holder icon invisible_gif | invisible.gif ::: status icons: up, not-reporting, partial-up, down green_status | yes_50.gif blue_status | down_50.gif yellow_status | caution_50.gif red_status | no_50.gif ::: state, info, remark, and error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: fping executable, used to see if the systems are alive Fping | /usr/local/bin/fping ::: the interval (seconds) for checking the systems Run_interval | 300 ::: the outage interval (seconds) for the systems Outage_threshold | 300 ::: the maximum log size Log_size | 1048576 ::: Table Header messages if needed. motd | Help link not enabled. ::: ICMP check file ICMP_check | icmp.log ::: TCP check file TCP_check | tcp.log ::: fping timne-out value (seconds) Ping_Time_out | 5 ::: Here, we list the nodes to be checked. On the External System ::: Monitor page, "server" items appear under server, while "comm" ::: and "gateway" are listed under network. #!! DO NOT EDIT THIS LINE #Node|Keynodes|Full Name|Address|CategoryKey|CategoryName Node|LS|Primary Ldad Server|ls1|server Node|TS1|TS 7e1out|7e1out|comm Node|TS2|TS 8n1out|8n1out|comm Node|GW|Gateway|ls1|gateway ::: These are the processes to check. These items will show up ::: under Processes on the External System Monitor page. The ::: node key below relates back to the section above to get the ::: actual node name. #Process|KeyNodes|ProcessName|FullProcessName Process|LS|httpd|Web Server Process|LS|newLDADdataNotification|LDAD Data Notifier Process|LS|CO_serv|Communication Server Process|LS|MakePROCpage|External Process Monitor Process|LS|watchDogExternal.sh|External WatchDog Process Process|LS|java|HmIngest Process|LS|ROSA_Acq DTMF|ROSA Touch-Tone Data Acq Process|LS|ROSA_Acq PAD|ROSA PAD Data Acq Process|LS|MakeLDApage /ldad/data/ldd.c|Dissemination Monitor Process|LS|MakeLDApage /ldad/data/lda.c|Data Acquisition Monitor |
#<LDADADMIN> # proc_internal.conf # ::: The default username that the process uses when it performs its ::: activities. User | ldad ::: Debug flag. 0=false; 1=true Debug | 1 ::: Verbose flag. 0=false; 1=true Verbose | 1 ::: Path to the icons directory. ICONDIR | /icons ::: Log file Name LOGFILE | LDAD_Process.log ::: Root location of all Monitor files. The rest of the directories are ::: appended to this root. ROOTDIR | /awips/fxa/htdocs/ldadMon ::: binaries BINDIR | /bin ::: log directory LOGDIR | /data/logs/fxa ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all html files are written PRODDIR | /products ::: the Debug filename DEBUGFILE | MakePROCpage.dbg ::: the Error filename ERRORFILE | MakePROCpage.err ::: the output html filename HTMLFILE | proc_internal.html ::: the Administration URL (in this case, the CGI script that creates it) ADMINURL | /ldadAdmin-cgi/ldadAdmin.pl ::: the Admin config file ADMINCONFIG | /awips/fxa/htdocs/ldadMon/conf/ldadAdmin.conf ::: the Restart URL (in this case, the CGI script that creates it) RESTARTURL | /awipsAdmin-cgi/ldadRestart.pl ::: the Restart config file RESTARTCONFIG | /awips/fxa/htdocs/ldadMon/conf/ldadRestart.conf ::: the title for the Internal System monitor PAGETITLE | LDAD System Monitor ::: the refresh rate (seconds) written to the HTML file PageRefresh | 60 ::: place-holder icon invisible_gif | invisible.gif ::: status icons: on time, late, more late, very late, missing green_status | yes_50.gif blue_status | tardy_50.gif yellow_status | caution_50.gif red_status | no_50.gif missing_status | unknown_50.gif ::: state, info, remark, and error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: fping executable, used to see if the systems are alive Fping | /usr/local/bin/fping ::: the interval (seconds) for checking the systems Run_interval | 300 ::: the outage interval (seconds) for the systems Outage_threshold | 300 ::: the maximum log size Log_size | 1048576 ::: Table Header messages if needed. motd | Help link not enabled. ::: A redefinition of the outage interval (??) Outage_threshold | 600 ::: ICMP check file ICMP_check | icmp.log ::: TCP check file TCP_check | tcp.log ::: fping timne-out value (seconds) Ping_Time_out | 5 ::: title for the Server table serverTitle|Server ::: title for the Communication table commTitle|Communication Devices ::: Here, we list the nodes to be checked. On the System Monitor ::: Monitor page, "server" items appear under server, while "comm" ::: and "gateway" are listed under network. #!! DO NOT EDIT THIS LINE #Node|Keynodes|Full Name|Address|CategoryKey|CategoryName Node|AS1F|Primary Application Server|as1f|server Node|AS2F|Secondary Application Server|as2f|server Node|DS|Current Data Server|ds|server Node|DS1|Primary Data Server|ds1|server Node|DS2|Secondary Data Server|ds2|server Node|GW|Gateway|ds|gateway ::: These are the processes to check. These items will show up ::: under Processes on the System Monitor page. The node key below ::: relates back to the section above to get the actual node name. #Process|Keynodes|ProcessName|FullProcessName Process|AS1F|MakePROCpage|Process Monitor Process|AS1F|MakeSUMMpage|LDAD Monitor Summary Process|DS|listener|Inter-Gateway Communication Process|DS|pollForData.pl|Internal-External Data Transfer Process|DS|routerLdadDecoder|LDAD Decoder Process|DS|routerStoreNetcdf|netCDF Storage Process|DS|routerShefEncoder|SHEF Encoder Process|DS|routerStoreText|Text Storage Process|DS|DataController LDAD_ROUTER|LDAD Data Controller Process|DS|CommsRouter LDAD_ROUTER|LDAD Comms Router Process|DS|watchDogInternal.sh|Internal Watchdog Process |
#<LDADADMIN> # THIS IS A GENERATED FILE; DO NOT EDIT UNLESS YOU ARE USING THE GUI. # MANUAL EDITING CHANGES MAY CAUSE PROGRAMS THAT RELY ON THIS FILE TO # CEASE WORKING PROPERLY. # # lda.conf # Last Modified Fri Aug 6 21:37:14 GMT 1999 # # The date above will be modified when this file is edited using the GUI. # # First, define a number of variables. The only thing here that field # sites might change is the Run_interval. # ::: 0 - False, 1 - True Debug | 1 Verbose | 1 ::: Root location of all Monitor files. The rest of the directories ::: are appended to this root. ROOTDIR | /data/ldad/www/htdocs/ldadmon ::: Path to the icons directory. ICONDIR | /icons/ ::: binaries BINDIR | /bin ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all HTML files are written PRODDIR | /products ::: logs LOGDIR | /data/logs/ldad ::: log file name LOGFILE | lda_monitor.log ::: the debug filename DEBUGFILE | lda_monitor.dbg ::: the error filename ERRORFILE | lda_monitor.err ::: the output HTML filename HTMLFILE | lda.html ::: the title of the HTML page PAGETITLE | Local Data Acquisition ::: the provider URL (the CGI script that creates it) PROVIDERURL | /cgi-bin/provider.pl?file=/ldad/data/LDADProvider.txt ::: refresh rate of the page (seconds) PageRefresh | 60 ::: the interval (seconds) for checking the system Run_interval | 300 ::: the background icon invisible_gif | invisible.gif ::: icon for missing data missing_status | unknown_50.gif ::: state, info, remark, error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: This next section lists each acquisition dataset to be monitored. ::: Local changes will be necessary, of course. ::: (The lines are very long, so we've split them. Each line actually ::: begins with Check and ends with gov.) # # (These comments will disappear when the file is edited, but we include # them here for the record.) # # Datasets to be monitored, their time-outs, and messages. # Each entry begins with the word "Check". # "title" is the name to appear in the Data column of the acquisition monitor. # "data key" should match an entry in LDADProvider.txt. ::: Data file names here are acquired files on ls1, so all are directed ::: to /data/ldad/Processed. # Names of the gifs should not be modified. # Times less than "green" will show green, those between "green" and "blue" # will show blue, between "blue" and "yellow" show yellow, and greater than # "yellow" shows red. The "red" time-out doesn't matter, just > "yellow." # [siteId] is replaced by the site name during installation. # #!! DO NOT EDIT OR DELETE THIS LINE! #Check|Title|data key (LDADProvider.txt)|data file name|(don't change gifs) green|blue|yellow|red||"Acquisition"|Mail receiver(s) if any Check|SchoolNet|SCHLNET|/data/ldad/Processed/schlnet.log|yes_50.gif;600;On time| tardy_50.gif;1200;10-20 min late|caution_50.gif;3600;20-60 min late| no_50.gif;86400;>1 hr late - Check with provider.|| Acquisition| ldad@[siteId].noaa.gov Check|RAWS|NWSRAWS|/data/ldad/Processed/RAWS.dat|yes_50.gif;1800;On time| tardy_50.gif;3600;30-60 min late|caution_50.gif;7200;1-2 hrs late.| no_50.gif;86400;>2 hrs late - Check with provider.|| Acquisition| ldad@[siteId].noaa.gov Check|CDoT|CDoT|/data/ldad/Processed/cdot_ssi.dat|yes_50.gif;900;On time| tardy_50.gif;1800;15-30 min late|caution_50.gif;3600;30-60 min late.| no_50.gif;86400;>1 hr late - Check with provider.|| Acquisition| ldad@[siteId].noaa.gov Check|ALERT-Wx|UDFCD|/data/ldad/Processed/alert_wx.dat|yes_50.gif;900;On time| tardy_50.gif;1800;15-30 min late|caution_50.gif;3600;30-60 min late.| no_50.gif;86400;>1 hr late - Check with provider.|| Acquisition| ldad@[siteId].noaa.gov Check|ALERT-Rain |UDFCD|/data/ldad/Processed/alert_r5.dat|yes_50.gif;900;On time| tardy_50.gif;1800;15-30 min late|caution_50.gif;3600;30-60 min late| no_50.gif;86400;>1 hr late - Check with provider.|| Acquisition| ldad@[siteId].noaa.gov Check|ALERT-Stream |UDFCD|/data/ldad/Processed/alert_wl.dat|yes_50.gif;900;On time| tardy_50.gif;1800;15-30 min late|caution_50.gif;3600;30-60 min late| no_50.gif;86400;>1 hr late - Check with provider.|| Acquisition| ldad@[siteId].noaa.gov |
#<LDADADMIN> # THIS IS A GENERATED FILE; DO NOT EDIT UNLESS YOU ARE USING THE GUI. # MANUAL EDITING CHANGES MAY CAUSE PROGRAMS THAT RELY ON THIS FILE TO # CEASE WORKING PROPERLY # # ldd.conf # Last Modified Fri Aug 6 21:37:14 GMT 1999 # # The date above will be modified when this file is edited using the GUI. # # First, define a number of variables. The only thing here that field # sites might change is the Run_interval. # ::: 0 - False, 1 - True Debug | 0 Verbose | 1 ::: Path to the icons directory. ICONDIR | /icons/ ::: log file name LOGFILE | ldd_monitor.log ::: Root location of all Monitor files. The rest of the directories ::: are appended to this root. ROOTDIR | /data/ldad/www/htdocs/ldadmon ::: binaries BINDIR | /bin ::: logs LOGDIR | /data/logs/ldad ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all HTML files are written PRODDIR | /products ::: the debug filename DEBUGFILE | ldd_monitor.dbg ::: the error filename ERRORFILE | ldd_monitor.err ::: the output HTML filename HTMLFILE | ldd.html ::: the title of the HTML page PAGETITLE | Local Data Dissemination Monitor ::: the provider URL (the CGI script that creates it) PROVIDERURL | /cgi-bin/provider.pl?file=/ldad/data/LDADProvider.txt ::: refresh rate of the page (seconds) PageRefresh | 60 ::: the interval (seconds) for checking the system Run_interval | 300 ::: the background icon invisible_gif | invisible.gif ::: missing status icon missing_status | unknown_50.gif ::: state, info, remark, error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: This next section lists each dissemination dataset to be monitored. ::: The defaults of course will need to be changed on-site. ::: (The lines are very long, so we've split them. Each line actually ::: begins with Check and ends with gov.) # # (These comments will disappear when the file is edited, but we include # them here for the record.) # # Datasets to be monitored, their time-outs, and messages. # Each entry begins with the word "Check". # "title" is the name to appear in the Data column of the dissemination monitor. # "data key" should match an entry in LDADProvider.txt. # Names of the gifs should not be modified. # Times less than "green" will show green, those between "green" and "blue" # will show blue, between "blue" and "yellow" show yellow, and greater than # "yellow" shows red. The "red" time-out doesn't matter, just > "yellow." # [siteId] is replaced by the site name during installation. # #!! DO NOT EDIT OR DELETE THIS LINE! #Check|title|provider (LDADProvider.txt)|data file name|(don't change gifs) green|blue|yellow|red||"Dissemination"|Mail receiver(s) if any Check|DEN Special Weather Statement|DEN|/data/ldad/public/nwswwas/DENSPSDEN1.txt| yes_50.gif;86400;On time|||no_50.gif;86400;Very late - Check triggers|| Dissemination|ldad@[siteId].noaa.gov Check|CYS Special Weather Statement|CYS|/data/ldad/public/nwswwas/CYSSPSCYS1.txt| yes_50.gif;86400;On time|||no_50.gif;86400;Very late - Check triggers|| Dissemination|ldad@[siteId].noaa.gov Check|DEN NOWCAST|DEN|/data/ldad/public/nwswwas/DENNOWDEN1.txt| yes_50.gif;86400;On time|||no_50.gif;86400;Very late - Check triggers|| Dissemination|ldad@[siteId].noaa.gov #Check|DEN 006 QC daily message|DEN|/data/ldad/public/qcmessages/006QCDDEN1.txt| yes_50.gif;86400;On time|||no_50.gif;86400;Very late - Check triggers and QC|| Dissemination|ldad@[siteId].noaa.gov #Check|DEN 007 QC daily message|DEN|/data/ldad/public/qcmessages/007QCDDEN1.txt| yes_50.gif;86400;On time|||no_50.gif;86400;Very late - Check triggers and QC|| Dissemination|ldad@[siteId].noaa.gov #Check|DEN 007 QC weekly message|DEN|/data/ldad/public/qcmessages/007QCWDEN1.txt| yes_50.gif;86400;On time|||no_50.gif;86400;Very late - Check triggers and QC|| Dissemination|ldad@[siteId].noaa.gov Check|DEN METAR|DEN|/data/ldad/public/nwswwas/DENMTRDEN1.txt| yes_50.gif;3600;On time|||no_50.gif;7200;Very late - Check triggers|| Dissemination|ldad@[siteId].noaa.gov |
#<LDADADMIN> # THIS IS A GENERATED FILE; DO NOT EDIT UNLESS YOU ARE USING THE GUI. # MANUAL EDITING CHANGES MAY CAUSE PROGRAMS THAT RELY ON THIS FILE TO # CEASE WORKING PROPERLY # # summary.conf # Last Modified Thu Aug 12 15:39:41 GMT 1999 # The date above will be modified when this file is edited using the GUI. # # First, define a number of variables. In general, these should not be # changed at field sites. # ::: The default username that the process uses when it performs its ::: activities. User | fxa ::: Debug flag. 0=false; 1=true Debug | 1 ::: Verbose flag. 0=false; 1=true Verbose | 1 ::: Root location of all Monitor files. The rest of the directories are ::: appended to this root. ROOTDIR | /awips/fxa/htdocs/ldadMon ::: binaries BINDIR | /bin ::: log directory LOGDIR | /data/logs/fxa ::: Log file Name LOGFILE | LDAD_Summary.log ::: the maximum log size Log_size | 1048576 ::: Path to the icons directory. Could be a NameTrans Variable ::: (NS-Fasttrack Add'l Document Dir). ICONDIR | /icons ::: temp directory TMPDIR | /tmp ::: configuration files CONFIGDIR | /conf ::: where all html files are written PRODDIR | /products ::: the Debug filename DEBUGFILE | MakeSUMMpage.dbg ::: the Error filename ERRORFILE | MakeSUMMpage.err ::: the output html filename HTMLFILE | summary.html ::: the title for the Summary page PAGETITLE | LDAD Monitor ::: number of columns in the display columns | 4 ::: string to add to produce full domain name Truncate_domain_display | .noaa.gov ::: refresh rate of the page (seconds) PageRefresh | 60 ::: the interval (seconds) for checking the systems Run_interval | 300 ::: place-holder icon invisible_gif | invisible.gif ::: state, info, remark, and error icons state_icon | trans_s_small.gif info_icon | trans_i_small.gif remark_icon | trans_r_small.gif error_icon | trans_check_red.gif ::: status icons: on time, late, more late, very late, missing green_status | yes_50.gif blue_status | tardy_50.gif yellow_status | caution_50.gif red_status | no_50.gif missing_status | unknown_50.gif ::: These define the components of the main display. Generally, there ::: should be no need to edit these lines. (Note: the Path entry is ::: relative to the document root of the Web server on the respective ::: machines.) #!! DO NOT REMOVE THIS LINE. It is needed for automatic editing through the GUI! #Page|Page number|Node|Protocol|Path|File|Title Page|Page1|ls1|http|ldadmon/products|lda.html|Data Acquisition Page|Page2|ls1|http|ldadmon/products|ldd.html|Data Dissemination Page|Page3|ls1|http|ldadmon/products|proc_external.html|External Process and System Page|Page4|as1f|http|ldadmon/products|proc_internal.html|Internal Process and System |
|
|
|