jifmemo: jif_memo_040409
John Ward
wd22jw at ncep.noaa.gov
Fri Apr 9 16:02:21 EDT 2004
-> John Ward <wd22jw at ncep.noaa.gov> writes:
April 9, 2004
MEMORANDUM FOR: Distribution E
FROM: John H. Ward
Chief, Production Management Branch
SUBJECT: Changes to the NWS Production Suite
On Tuesday, 20 April 2004, the configuration of the Eta Model
will be modified to allow for the 0 - 84 hour forecast to run in the
current 0 - 60 hour time slot. This will be a permanent change.
The expected new delivery times for Eta products are as follows:
24 hr products: 5-10 minutes earlier
36 hr products: 15-20 minutes earlier
48 hr products: 15-20 minutes earlier
60 hr products: 20-25 minutes earlier
84 hr products: 35-40 minutes earlier
More precise times will be provided when they become available.
Details on the change can be found at:
http://www.nws.noaa.gov/om/notifications/tin04-07eta_compression.txt
Currently, on both the NCEP and TOC FTP servers, over 50% of
the gridded NCEP model data sets are not being accessed. A review
of file access on both FTP servers was conducted during the months
of January and February 2004. From this review a list of the
unaccessed data sets has been generated. It is NCEP's intention to
remove these data sets, but we would like to first solicit feedback
from our users. The proposed list of data sets to be removed has
been posted on the NCEP web site at the following location:
http://www.nco.ncep.noaa.gov/pmb/products/ftp_removal_list.html
An E-mail address has been provided on this page to allow for
users to provide feedback. Data sets were only included on this
list if they were unused on both the TOC and NCEP servers. User
comments on this list will be accepted through 15 June 2004. On 29
June 2004 all files deemed unnecessary will be removed from the NCEP
ftp server. This process will provide the additional server
resources to host higher temporal and spatial resolution data not
currently available.
To unsubscribe from the JIF memo use the link provided at:
http://www.nco.ncep.noaa.gov/pmb/jifmemo/
* * * * * * * * * * * * * * * * * * * * * * * * * * * *
Synopsis of Major System Changes
Week of 12 April 2004
CCS Production:
13 April 2004
-- RUC model Physics upgrade
Dataflow:
7 April 2004 (Unadvertised change from this week)
-- System: CCS, TGSV10/11
- Eta model output on grid 242 (AK 11.25 km) is now being
delivered to the TOC ftp server.
12 April 2004
-- System: NCOSRV
- The creation of individual NDFD files will be ceased.
The monolithic files will remain
15 April 2004
-- System: NCOSRV
- The SPIDER satellite acquisition server is being modified
to accept DPD imagery.
* * * * * * * * * * * * * * * * * * * * * * * * * * * *
The following items will be implemented at 12Z on Tuesday, 13
April:
245. RUC2_HYBFRONT - IBM Jobs WW0RUC2A_PREP_00-23. (Manikin,
NP22). This program performs the ingest, quality control, and
analysis function for the RUC2 analysis and forecast system. The
3DVAR initialization is being modified to use a PBL-based
assimilation of surface observations.
246. RUC2_HYBCSTPR - IBM Jobs WW0RUC2A_PREP_00-23. (Manikin,
NP22). This program reads in and prepares fields for use in the
RUC2 forecast model. New vegetation type and soil fixed fields are
being used to improve model performance along coastlines.
247. RUC2_HYBCST - IBM Jobs WW0RUC2A_FCST_00-23. (Manikin,
NP22). This program is the forecast module for the RUC model. It
is being modified to; 1) add a more conservative centered
differencing for vertical advection of moisture replacing an
upstream differencing technique to improve precipitation forecasts;
2) change the micro physics and convective scheme are being modified
to reduce the generation of excess cirrus clouds and 3) to improve
forecasts of convective precipitation.
248. RUC2_HYBPOST - IBM Jobs WW0RUC2A_POST_00-23,
WW0RUC2B_POST_00, 12. (Manikin, NP22). This program is the post
processor for the RUC model. A check is being been added to make
certain that hydrometer mixing ratios are not negative values.
249. SCRIPT - IBM Jobs 615, 625, 635, 645, 675, 685. (Vuong,
NP11). This utility script mkawpgrb.sh produces AWIPS GRIB
bulletins. The script is being modified to use new GFS data cards
to produce the AWIPS GRIB bulletins.
250. DATACARD - IBM Jobs 615, 625, 635, 645, 675, 685.
(Vuong, NP11). The data cards used to create GFS AWIPS grids are
being reconstructed to separate them by the following grid ID's 201,
202, 203, 211, 213 and 225.
251. SMS - IBM Job jgfs_post_t254. (Gilbert, NP11). This job
executes the GFS post processor to generate new GRIB files that
contain a "master" set of parameters on a 0.5x0.5 degree global grid
for forecast hours 00 through 84. The SMS scripts and definition
files are being modified to rename jgfs_master_post0.5.
252. SCRIPT - IBM Job jgfs_post_t254. (Gilbert, NP11). This
program executes the GFS post processor to generate new GRIB files
that contain a "master" set of parameters on a 0.5x0.5 degree global
grid for forecast hours 00 through 84. The program is being renamed
jgfs_master_post0.5.
253. SMS - IBM Job jgfs_pgrb2_t254. (Gilbert, NP11). This
program produces GFS half degree fields in GRIB edition 2 format.
The program has an SMS dependency on a job that is being renamed, so
the dependency is being changed appropriately.
254. SMS - IBM Job jgfs_master_post1.0. (Gilbert, NP11).
This new job executes the GFS post processor to generate new GRIB
files that contain a "master" set of parameters on a 1x1 degree
global grid for forecast hours 87 through 180. Appropriate SMS
scripts and definition files are being added to the SMS for this new
job.
255. SCRIPT - IBM Job jgfs_master_post1.0. (Gilbert, NP11).
New job script JGFS_MASTER_POST1.0.sms.prod executes the GFS post
processor which generates new GRIB files that contain a "master" set
of parameters on a 1x1 degree global grid for forecast hours 87
through 180.
256. SMS - IBM Job jgfs_master_post2.5. (Gilbert, NP11).
This new job executes the GFS post processor to generate new GRIB
files that contain a "master" set of parameters on a 2.5x2.5 degree
global grid for forecast hours 192 through 384. Appropriate SMS
scripts and definition files are being added to the SMS for this new
job.
257. SCRIPT - IBM Job jgfs_master_post2.5. (Gilbert, NP11).
New job script JGFS_MASTER_POST2.5.sms.prod executes the GFS post
processor which generates new GRIB files that contain a "master" set
of parameters on a 2.5x2.5 degree global grid for forecast hours 192
through 384.
258. DATACARD - Many IBM Jobs. (Keyser, NP22). The fixed
file bufr_dumplist contains the current table listing of valid data
group mnemonic names for the various BUFR data types and subtypes in
the observational /dcom database on the IBM CCS. This text file is
being modified as follows:
1) remove the existing mnemonics "qksca1" and "qksca2"
2) add BUFR type 000, subtype 002 (mobile synoptic) to group
mnemonic "sflnd", message type "000" and dump group mnemonic
"adpsfc";
3) add BUFR type 002, subtype 012 (RASS) and subtype 013 (JMA
profiler winds) to group mnemonic "vsndn" and message type "002";
4) add BUFR type 006, subtype 002 (NEXRAD Level 2.5 radial wind
superobs) to group mnemonic "radar" and message type "006";
5) add BUFR type 031 NAVOCEANO ALPS high-resolution global altimeter
subtypes 101 (ERS2), 102 (GOES follow-on), 103 (TOPEX), 104 (JASON)
and 105 (GOES follow-on Navy IGDR winds) to group mnemonic "ocean"
and message type "031";
6) add BUFR type 255 Mesonet subtypes 013 (NWS COOP), 014 (NWS
HADS), 016 (Iowa Environmental), 017 (Oklahoma) and 018 (Colorado
DOT) to group mnemonic "local", message type "255" and dump group
mnemonic "msonet";
7) add BUFR type 004, subtype 006 [European AMDAR (E-ADAS) aircraft
data from BUFR] to dump group mnemonic "aircft";
8) add profiler wind BUFR type 002, subtypes 011 (CAP) and 013 (JMA)
to dump group mnemonic "proflr";
9) create new dump group mnemonic "rassda" consisting of BUFR type
002, subtype 012;
10) create new single type mnemonics "synopm" for mobile synoptic
(BUFR type 000, subtype 002), "rass" for RASS (type 002, subtype
012), "prflrj" for JMA profiler winds (type 002, subtype 013),
"radw30" for NEXRAD Level 3.0 radial wind superobs (type 006,
subtype 001), "radw25" for NEXRAD Level 2.5 radial wind superobs
(type 006, subtype 002), "nersal" for NAVOCEANO ALPS ERS2
high-resolution global altimeter (type 031, subtype 101), "ngfoal"
for NAVOCEANO ALPS GOES follow-on high-resolution global altimeter
(type 031, subtype 102), "ntpxal" for NAVOCEANO ALPS TOPEX
high-resolution global altimeter (type 031, subtype 103), "njsnal"
for NAVOCEANO ALPS JASON high-resolution global altimeter (type 031,
subtype 104), "ngfnal" for NAVOCEANO GOES follow-on Navy IGDR winds
altimeter (type 031, subtype 105), "msocob" for NWS COOP mesonet
(type 255, subtype 013), "msohad" for NWS HADS mesonet (type 255,
subtype 014), "msoaws" for AWS mesonet (type 255, subtype 015),
"msoien" for Iowa Environmental mesonet (type 255, subtype 016),
"msokla" for Oklahoma mesonet (type 255, subtype 017) and "msocol"
for Colorado DOT mesonet (type 255, subtype 018);
11) remove the following POES 1B radiance mnemonics which refer to
obsolete IEEE format dumps: "h1bn11" (NOAA-11 HIRS2), "m1bn11"
(NOAA-11 MSU), "h1bn12" (NOAA-12 HIRS2), "m1bn12" (NOAA-12 MSU),
"h1bn14" (NOAA-14 HIRS2), "m1bn14" (NOAA-14 MSU), "h1bn15" (NOAA-15
HIRS3), "a1bn15" (NOAA-15 AMSU-A), "b1bn15" (NOAA-15 AMSU-B),
"sbvn09" (NOAA-09 SBUV), "sbvn11" (NOAA-11 SBUV), "sbvn14" (NOAA-14
SBUV), "h1bn16" (NOAA-16 HIRS3), "a1bn16" (NOAA-16 AMSU-A), "b1bn16"
(NOAA-16 AMSU-B), "sbvn16" (NOAA 16 SBUV);
12) modify dump alert on "msothr" (Mesonet "other", BUFR type 255
subtype 030) so that a low-dump alert can never be triggered since
the number of Mesonets in this type can change as new types are
periodically pulled out of here and written into their own, new
unique subtypes;
13) improved the text description for many of the dump types - this
description appears in the updated dump count status file sent to
the SDM.
259. SCRIPT - Many IBM Jobs. (Keyser, NP22). Script dumpjb
is an all purpose dump utility for time windowing, geographical
filtering, eliminating duplicates, and applying corrections to BUFR
observational database tank data. This script is being modified to
:1) include European AMDAR (E-ADAS) aircraft (BUFR type 004, subtype
006) in the duplicate checking performed by BUFR_DUPAIR; 2) to
process subtypes in the order in which they are listed for a group
mnemonic in file bufr_dumplist, rather than always in ascending
order (necessary because, for BUFR type 002 in the "proflr" dump,
subtype 009 must be read last, and thus after the new subtypes 011
and 013, in the PREPOBS_PREPDATA and BUFR_LISTDUMPS interface
routines; 3) echo the mnemonic group name into unit 30 read by
BUFR_CHKBFR - this allows counts by satellite id to now be listed;
4) to no longer execute the Timex command prior to the Poe command
(a necessary change on IBM Frost/Snow, but never noticed because poe
is not used in operations); 5) correct an error which had resulted
in the test for LALO = 0 failing when LALO includes spaces (LALO = 0
means do not execute BUFR_GEOFIL to geographically filter reports);
6) no longer execute BUFR_QUIPC unless one or more dumps in the
group include BUFR types 001, subtypes 001 through 005 since these
are the only types acted upon by this program which applies quips
quality markers and corrections to marine data; 7) to no longer
execute BUFR_EDTBFR unless one or more dumps in the group include
BUFR types 000, 001, 002, 004 or 005 (any subtypes within) since
these are the only types acted upon by this program which applies
manual and reject list quality markers to reports; 8) to now exit
with RC=99 if either BUFR_EDTBFR or BUFR_QUIPC does not complete
properly instead of ignoring any problems in these two programs; 9)
to now perform duplicate checking of Mesonet data in BUFR type 255
using BUFR_DUPMAR followed immediately by BUFR_DUPCOR for each
subtype instead of using only BUFR_DUPCOR which had incorrectly
defined some unique reports as duplicates (this will still remove
one of two reports with different ID's and possibly different data
that are located at the "same" place out to the four significant
digits of lat/lon precision in the BUFR file); 10) to now copy dump
files that contain zero reports to the working directory and ensure
that they contain dictionary messages and "dummy" center and dump
time messages - this will always result in a valid BUFR file being
opened by BUFR_CHKBFR (before, the BUFRLIB was generating warning
messages when attempting to open null files when files with zero
reports were not copied).
260. BUFR_DUPMAR - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, processing
non-profile BUFR database report parts with correction choosing,
duplicate checking, report part merging and trimming to exact time
window (day down to minute). It currently operates on all surface
marine report types, METAR surface reports and all Mesonet surface
report types. The program is being modified to handle Mesonet data
in either its old or new BUFR report structure: it now checks for
reports with high-resolution lat/lon if the default low-resolution
lat/lon are missing; and it will not fail if the receipt time
sequence is not replicated in the BUFR table, but if it is
replicated, it will store the receipt times for the report parts in
1:1 correspondence with the new filename replication sequence which
contains the original FSL netCDF filenames containing each report
part (stored in the run-history archive). The program is also being
modified to bump up the number of reports it can process from 100K
to 300K, and to replace the call to in-line subroutine
COMPRESS_CHECK with a call to BUFRLIB routine MESGBC in order to
check for message compression.
261. BUFR_DUPSAT - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, processing
satellite BUFR database reports (all types except for POES 1B
radiances from TOVS, ATOVS and AIRS) with duplicate checking and
trimming to exact time window (day down to second). An updated
report id is also generated for derived winds from GOES, METEOSAT,
INSAT and GMS and for NESDIS retrievals and processed radiances from
GOES. The program is being modified to prevent array overflow when
more than the maximum number of reports which can be processed are
read in, to improve diagnostic print, and to replace the call to
in-line subroutine COMPRESS_CHECK with a call to BUFRLIB routine
MESGBC in order to check for message compression. Also added a
logic to handle future MODIS (Aqua/Terra) winds.
262. BUFR_EDTBFR - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, applying real-time
interactive quality control flags, generated from either a reject
list maintained by NCEP/NCO or by the NCEP/NCO SDM, to all types of
surface land (BUFR message type 000), surface marine (message type
001), upper-air profile (message type 002), aircraft (message type
004) or satellite-derived wind (message type 005) report data. The
program is being modified to add flexibility in reading and checking
input file names, to replace calls to BUFRLIB routine "BORT" with
calls to W3LIB routine "ERREXIT", to streamline the code, to improve
Docblocks and comments, to add more descriptive standard output
print, to add a status file in unit 60 that is written to only when
this program completes successfully (this information is then
transferred to the parent dumpjb script), to now check to see if
each date entry in the flag file is between the exact time window
specification (out to fraction of hour at each end) rather than
between the nearest integer of the time window at each end, to
increase the date tolerance for satellite-derived winds [ABS(Obs.
Date - Flag File Entry Date)] to 2.01 hours in order to mark all
winds in the dump (still set to 0.01 hours for all other types), and
finally to correct an error in calculating date tolerance when it is
greater than or equal to 1.0 hours.
263. BUFR_CHKBFR - Many IBM Jobs. (Keyser, NP22). This
program checks a list of BUFR files (all in the same group, usually
all of the same message type and always all using the same
dictionary messages), to determine whether they contain any data or
not and then return a specific status code based on the degree of
available data. It also prints a status line which provides the
dump count for each subtype in each file. The program is being
modified to add flexibility in reading and checking input file
names, to replace calls to "STOP XX" with calls to W3LIB routine
"ERREXIT(XX)", to add prints of counts by satellite id for
applicable types (printed under total dump count for each satellite
subtype) and to replace the call to in-line subroutine
COMPRESS_CHECK with a call to BUFRLIB routine MESGBC in order to
check for message compression. In addition, report counts for radar
data in BUFR type 006 are now the total number of replication
"levels" (true observations) amongst all subsets in the BUFR
messages rather than just the number of subsets, as before, since
each subset represents only a radar station and tilt angle for a
specific hour and each subset can have up to several hundred radial
wind observations.
264. BUFR_DUPS1B - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, processing POES 1B
radiance BUFR database reports from TOVS, ATOVS and AIRS with
(optionally) geographical filtering, trimming to exact time window
(day down to second) and duplicate checking. The program is being
modified to correct a potential array overflow problem in duplicate
checking variables and to correct a subroutine argument misalignment
in-line routine CHDIST (both of these bugs could cause memory to be
clobbered). It is also being modified to skip reports with missing
lat or lon - this prevents array overflow since the duplicate
checking logic depends upon a valid lat and lon. The program is
also being modified to replace the call to in-line subroutine
COMPRESS_CHECK with a call to BUFRLIB routine MESGBC in order to
check for message compression, and to fix a bug in the duplicate
check logic that had allowed some duplicate reports to get through.
265. BUFR_DUMPMD - Many IBM Jobs. (Keyser, NP22). This
program dumps data from BUFR database file(s) as specified by the
user-supplied time window. It runs very fast since only the BUFR
message date in Section 1 is examined (no reports are unpacked).
The program is being modified to replace the call to in-line
subroutine COMPRESS_CHECK with a call to BUFRLIB routine MESGBC in
order to check for message compression.
266. BUFR_QUIPC - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, applying real-time
interactive quality control flags and corrections, generated from by
NCEP/OPC using "CREWSS" software, to surface marine ship, buoy,
C-MAN platform or tide gauge data (BUFR message type 001, subtypes
001-005). It also checks for reports in the Great Lakes since this
can affect the corrections. The program is being modified to add
flexibility in reading and checking input file names, to replace
calls to BUFRLIB routine "BORT" with calls to W3LIB routine
"ERREXIT", to streamline the code, to improve Docblocks and
comments, to add more descriptive standard output print, to fix a
few minor bugs, to add a status file in unit 60 that is written to
only when this program completes successfully (this information is
then transferred to the parent dumpjb script), to correct an error
in calculating date tolerance when it is greater than 0.0 hours, to
attempt to read a future moisture quality marker from column 45 and
a future wave height quality marker from column 54 of quips flag
file entries (neither is yet being written into quips flag file, so
these both remain blank in the flag file) and process these into the
dump file under mnemonics "QMDD" and "QMWH", respectively.
267. BUFR_GEOFIL - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, geographically
filtering BUFR database dump files with uncompressed messages using
either a lat/lon box filter or a center point and radius filter
(latter not yet available). The program is being modified to
replace the call to in-line subroutine COMPRESS_CHECK with a call to
BUFRLIB routine MESGBC in order to check for message compression,
and to correct a subroutine argument misalignment in-line routine
CHDIST (this bug could potentially cause memory to be clobbered).
268. BUFR_LISTDUMPS - IBM Jobs ETA_DUMP_POST, EDS_DUMP_POST,
AVN_DUMP_POST, FNL_DUMP_POST, CDS_DUMP_POST, RUC2A_DUMP_POST,
RUC2B_DUMP_POST, SRUC_DUMP_POST. (Keyser, NP22). This program
lists the contents of time windowed observational BUFR data dump
files in an easy to read format. The program is being modified to:
1) boundary layer (BUFR type 002, subtype 011) and Japanese (type
002, subtype 013) wind profilers are being added to the processing
of wind profilers not originating from "PILOT" bulletins;
2) RASS reports (BUFR type 002, subtype 012) are now processed from
the "rassda" data dump file and listed here using the new in-line
subroutine LISTRASS;
3) mobile surface land synoptic reports (BUFR type 000, subtype 002)
in the "adpsfc" dump file can now be processed and their elevation
quality marker can be listed;
4) European AMDAR (E-ADAS) aircraft reports (BUFR type 004, subtype
006) in the "aircft" dump file can now be processed;
5) more variables are now listed for Mesonet data in the "msonet"
dump file;
6) accounts for the transition of Mesonet reports from low- to
high-resolution lat/lon (in early 2004) and modified the lat/lon
listing for surface data to now print four significant digits
instead of two;
7) add the option to select different wind and height assignments
for GOES satellite-derived winds (although these are still hardwired
to the "final" NESDIS values);
8) ACARS flight number is now decoded from the "aircar" dump file
and listed.
9) add logic to process future MODIS (Aqua/Terra) satellite winds in
subroutine IW3UNPBF.
269. BUFR_DUPRAD - IBM Jobs ETA_DUMP, EDS_DUMP. (Keyser,
NP22). This program runs in the observational data dump step,
processing WSR-88D NEXRAD radar radial wind BUFR database reports
with duplicate checking and trimming to exact time window (day down
to minute). The program is being modified to handle future Level
"2.5" superob data, where the upstream ingest code splits a single
radar site and tilt angle into "pieces" (subsets) which each contain
no more than 500 "levels". (The ingest code does this to avoid
ending up with a subset that is actually larger than a BUFR
message.) This change to BUFR_DUPRAD will prevent it from tossing
the extra subsets (which all have the same lat, lon, time, height
and tilt angle) in the duplicate check by including the requirement
than a new mnemonic representing the BUFR message number ("MGPT",
added in the upstream ingest step and unique for each split subset)
must match in order for the duplicate check to be satisfied. In
addition, the program is being modified to replace the call to
in-line subroutine COMPRESS_CHECK with a call to BUFRLIB routine
MESGBC in order to check for message compression.
270. SCRIPT - IBM Job ETA_DUMP. (Keyser, NP22). Job script
JETA_DUMP.sms.prod initiates the job executing the model script
exeta_dump.sh.sms which performs the Eta network observational data
dumping from the BUFR /dcom database on the IBM CCS. The model
script is being modified to: 1) add the dump of new data type
"rassda" (RASS temperature data) with a 1.5 hour time window radius
(this is not yet written into PREPBUFR or assimilated); 2) expand
the "proflr" (wind profiler) dump time window radius from 1.5 to 2.5
hours in preparation for the future upgrade of the PREPOBS_PROFCQC
program (for now, the PREPBUFR time window radius for these data
remains 1.5 hours so there is no change to the amount of data
available to the assimilation); 3) include Cooperative Agency
Profiler (CAP) reports (BUFR type 002, subtype 011) in the "proflr"
dump (these are not yet written into PREPBUFR or assimilated); 4)
remove the dumping of IEEE satellite 1B types "h1bn14", "h1bn15",
"h1bn16", "m1bn14", "a1bn15", "a1bn16", "b1bn15" and "b1bn16" (these
are no longer assimilated, the BUFR dumps are used); 5) no longer
include 1x1 FOV GOES cloud data (BUFR type 003, subtype 002) in the
"goesnd" (GOES sounding) dump (and subsquently in the ETA PREPBUFR
files) since these data are written into hourly PREPBUFR files in
separate jobs for nudging by the forecast component of EDAS (prior
to the creation of these hourly jobs in July 2003, these data in the
ETA PREPBUFR files were used to nudge the last three hours of the
EDAS); 6) include five new subtypes in the "msonet" (BUFR type 255)
dump [NWS Cooperative Observer network (subtype 013), NWS HADS
network (subtype 014), Iowa Environmental (subtype 016), Oklahoma
MesoNet (subtype 017) and Colorado DOT (subtype 018)] all with a
time window radius of 1.5 hours (none of the Mesonet data are yet
written into PREPBUFR or assimilated); 7) include Euopean AMDAR
(E-ADAS) reports (BUFR type 004, subtype 006) with a time window
radius of 3.25 hours in the "aircft" dump (these are not yet written
into PREPBUFR or assimilated). As a result of 4 above, the job
script is being modified to remove variables pertaining to the
processing of IEEE 1B dumps ("TANKsat", "ushscript_satdump" and
"PROCESS_satDUMP").
271. BUFR_DUPMRG - Many IBM Jobs. (Keyser, NP22). This
program runs in the observational data dump step, processing
upper-air profile BUFR database reports that arrive in parts with
correction choosing, duplicate checking, report part merging and
trimming to exact time window (day down to minute). It currently
operates on all radiosonde, dropwinsonde and pibal reports as well
as on wind profiler reports which arrive in "PILOT" format. The
program is being modified to replace the call to in-line subroutine
COMPRESS_CHECK with a call to BUFRLIB routine MESGBC in order to
check for message compression, to better define the nature of the
error in an abnormal return from BUFRLIB routine RDMEMM, to refine
the check for possible up-stream lat and lon corrections
(adjustments) to the same dropwinsonde report ID (necessary so the
drops can still be duplicate checked using lat/lon coordinates like
all other types) such that lat/lon adjustments will only be made if
the correction indicator is both greater than zero and non-missing
(before, the check was only for non-missing correction and this led
to the elimination of unique Taiwanese drops which all contained the
same hardwired ID "DRP99A" but had a correction indicator of zero).
272. SCRIPT - IBM Jobs RUC2A_DUMP, RUC2B_DUMP. (Keyser,
NP22). The model script exruc2_dump.sh.sms performs the hourly
RUC-2A (early) and 12 hourly RUC-2B (catch up) network observational
data dumping from the BUFR /dcom database on the IBM CCS. The model
script is being modified to:
1) add the dump of new data type "rassda" (RASS temperature data)
with a 0.5 hour time window radius (this is not yet written into
PREPBUFR or assimilated);
2) expand the "proflr" (wind profiler) dump time window radius from
0.5 to 1.5 hours in preparation for the future upgrade of the
PREPOBS_PROFCQC program (for now, the PREPBUFR time window radius
for these data remains 0.5 hours so there is no change to the amount
of data available to the assimilation);
3) include Cooperative Agency Profiler (CAP) reports (BUFR type 002,
subtype 011) in the "proflr" dump (these are not yet written into
PREPBUFR or assimilated);
4) include five new subtypes in the "msonet" (BUFR type 255) dump
[NWS Cooperative Observer network (subtype 013), NWS HADS network
(subtype 014), Iowa Environmental (subtype 016), Oklahoma MesoNet
(subtype 017) and Colorado DOT (subtype 018)] all with a time window
radius of 0.5 hours (none of the Mesonet data are yet written into
PREPBUFR or assimilated).
273. SCRIPT - IBM Job EDS_DUMP. (Keyser, NP22). Job script
JEDS_DUMP.sms.prod initiates the job executing the model script
exedas_dump.sh.sms which performs the EDAS network observational
data dumping from the BUFR /dcom database on the IBM CCS. The model
script is being modified to:
1) add the dump of new data type "rassda" (RASS temperature data)
with a 1.5 hour time window radius (this is not yet written into
PREPBUFR or assimilated);
2) expand the "proflr" (wind profiler) dump time window radius from
1.5 to 2.5 hours in preparation for the future upgrade of the
PREPOBS_PROFCQC program (for now, the PREPBUFR time window radius
for these data remains 1.5 hours so there is no change to the amount
of data available to the assimilation);
3) include Cooperative Agency Profiler (CAP) reports (BUFR type 002,
subtype 011) in the "proflr" dump (these are not yet written into
PREPBUFR or assimilated);
4) remove the dumping of IEEE satellite 1B types "h1bn14", "h1bn15",
"h1bn16", "m1bn14", "a1bn15", "a1bn16", "b1bn15" and "b1bn16" (these
are no longer assimilated, the BUFR dumps are used);
5) no longer include 1x1 FOV GOES cloud data (BUFR type 003, subtype
002) in the "goesnd" (GOES sounding) dump (and subsequently in the
ETA PREPBUFR files) since these data are written into hourly
PREPBUFR files in separate jobs for nudging by the forecast
component of EDAS (prior to the creation of these jobs in July 2003,
these data in the EDAS PREPBUFR files were used to nudge the first
nine hours of the EDAS);
6) include five new subtypes in the "msonet" (BUFR type 255) dump
[NWS Cooperative Observer network (subtype 013), NWS HADS network
(subtype 014), Iowa Environmental (subtype 016), Oklahoma MesoNet
(subtype 017) and Colorado DOT (subtype 018)] all with a time window
radius of 1.5 hours (none of the Mesonet data are yet written into
PREPBUFR or assimilated)];
7) include European AMDAR (E-ADAS) reports (BUFR type 004, subtype
006) with a time window radius of 3.25 hours in the "aircft" dump
(these are not yet written into PREPBUFR or assimilated).
As a result of 4 above, the job script is being modified to remove
variables pertaining to the processing of IEEE 1B dumps ("TANKsat",
"ushscript_satdump" and "PROCESS_satDUMP").
274. SCRIPT - IBM Jobs AVN_DUMP, FNL_DUMP, CDS_DUMP. (Keyser,
NP22). Job scripts JAVN_DUMP.sms.prod, JFNL_DUMP.sms.prod and
JCDS_DUMP.sms.prod initiate the jobs executing the model script
exglobal_dump.sh.sms which performs the global observational data
dumping from the BUFR /dcom database on the IBM CCS in the AVN
(GFS), FNL (GDAS) and CDAS networks, respectively. The model script
is being modified to:
1) remove the logic which split the dumping of QuikSCAT data into
two parts (one for the first 1.5 hours of the 3.0 hour time window
and a second for the last 1.5 hours) and then combined the split
dumps into a final "qkscat" dump and the split reprocessed dumps
into a final "qkswnd" dump (the QuikSCAT dump had been split up to
increase computational efficiency back in the days of the Cray) -
now a single "qkscat" dump is performed over the full 3.0 time
window radius followed by the generation of the reprocessed dump
"qkswnd";
2) add the dump of new data type "rassda" (RASS temperature data)
with a 3.0 hour time window radius (this is not yet written into
PREPBUFR or assimilated);
3) add the dump of new data type "airs" (AQUA AIRS IR and AMSU-A 1B
radiances) with a 3.0 hour time window radius (this is not yet
assimilated);
4) add the dump of new data type "radwnd" (NEXRAD radial wind data)
with a 3.0 hour time window radius in the CDAS network only (this
will never be assimilated by the CDAS but is generated here to
provide a late, complete data dump file for archival purposes);
5) expand the "proflr" (wind profiler) dump time window radius from
3.0 to 4.0 hours in preparation for the future upgrade of the
PREPOBS_PROFCQC program (for now, the PREPBUFR time window radius
for these data remains 3.0 hours so there is no change to the amount
of data available to the assimilation);
6) include Cooperative Agency Profiler (CAP) reports (BUFR type 002,
subtype 011) and JMA wind profiler reports (BUFR type 002, subtype
013) in the "proflr" dump (neither of these are yet written into
PREPBUFR or assimilated);
7) remove the dumping of IEEE satellite 1B types "h1bn14", "h1bn15",
"h1bn16", "m1bn14", "a1bn15", "a1bn16", "b1bn15" and "b1bn16" as
well as sbuv type "sbvn16" (these are no longer assimilated, the
BUFR dumps are used);
8) include mobile synoptic surface reports (BUFR type 000, subtype
002) with a time window radius of 3.0 hours in the "adpsfc" dump
(these are not yet written into PREPBUFR or assimilated);
9) include five new subtypes in the "msonet" (BUFR type 255) dump
which is performed in the CDAS network only [NWS Cooperative
Observer network (subtype 013), NWS HADS network (subtype 014), Iowa
Environmental (subtype 016), Oklahoma MesoNet (subtype 017) and
Colorado DOT (subtype 018)] all with a time window radius of 3.0
hours (none of the MesoNet data will ever be assimilated by the CDAS
but the "msonet" dump is generated here to provide a late, complete
data dump file for archival purposes);
10) include European AMDAR (E-ADAS) reports (BUFR type 004, subtype
006) with a time window radius of 3.25 hours in the "aircft" dump
(these are not yet written into PREPBUFR or assimilated).
As a result of 7 above, the three job scripts are all being modified
to remove variables pertaining to the processing of IEEE 1B dumps
("TANKsat", "ushscript_satdump" and "PROCESS_satDUMP").
275. SCRIPT - IBM Job SRUC_DUMP. (Keyser, NP22). The model
script exsfcruc_dump.sh.sms performs the hourly surface RUC network
observational data dumping from the BUFR /dcom database on the IBM
CCS. The model script is being modified to include five new
subtypes in the "msonet" (BUFR type 255) dump [NWS Cooperative
Observer network (subtype 013), NWS HADS network (subtype 014), Iowa
Environmental (subtype 016), Oklahoma MesoNet (subtype 017) and
Colorado DOT (subtype 018)] all with a time window radius of 0.5
hours (none of the MesoNet data are yet assimilated).
276. SCRIPT - IBM Job DUMP_MONITOR. (Keyser, NP22). Job
script JDUMP_MONITOR.sms.prod initiates the job executing the model
script exdump_monitor.sh.sms which runs at 30 minutes past each
hour, dumping data from a unique one-hour time window, normally +/-
0.5 hours relative to the current cycle time for non-satellite data
and -1.0 to 0.0 hours or earlier relative to the current cycle time
for satellite data, in order to monitor the ingest of observational
data in the BUFR /dcom database on the IBM CCS. The model script is
being modified to:
1) add the dump of new data type "rassda" (RASS temperature data)
with a 0.5 hour time window radius;
2) add the dump of new data type "airs" (AQUA AIRS IR and AMSU-A 1B
radiances) with a -2.5 to -1.5 hour time window;
3) add the dump of new data type "radwnd" (NEXRAD radial wind data)
with a -1.5 to -0.5 hour time window;
4) include Cooperative Agency Profiler (CAP) reports (BUFR type 002,
subtype 011) and JMA wind profiler reports (BUFR type 002, subtype
013) in the "proflr" dump, both with a 0.5 hour time window radius;
5) removed dumping of obsolete IEEE satellite 1B types "h1bn14",
"h1bn16", "m1bn14", "a1bn15", "a1bn16", "b1bn15" and "b1bn16" as
well as sbuv type sbvn16;
6) include mobile synoptic surface reports (BUFR type 000, subtype
002) with a time window radius of 0.5 hours in the "adpsfc" dump;
7) include five new subtypes in the "msonet" (BUFR type 255) dump
[NWS Cooperative Observer network (subtype 013), NWS HADS network
(subtype 014), Iowa Environmental (subtype 016), Oklahoma MesoNet
(subtype 017) and Colorado DOT (subtype 018)] all with a time window
radius of 0.5 hours;
8) include European AMDAR (E-ADAS) reports (BUFR type 004, subtype
006) with a time window radius of 0.5 hours in the "aircft" dump;
9) change the time window for dumping marine tide gauge reports in
the "sfcshp" dump (BUFR type 001, subtype 005) from a -0.5 to +0.5
hours to -1.0 to 0.0 hours since these data seem to be arriving
later than before.
As a result of 5 above, the job script is being modified to remove
variables pertaining to the processing of IEEE 1B dumps (TANKsat",
"ushscript_satdump" and "PROCESS_satDUMP").
277. SCRIPT - IBM Jobs 314, SRUC_DUMP, RUC2A_DUMP, RUC2B_DUMP,
ETA_DUMP, EDS_DUMP, AVN_DUMP, FNL_DUMP, CDS_DUMP, DUMP_MONITOR,
ETA_PCPN_ANAL . (Keyser, NP22). The ush script bufr_dump_obs.sh is
the driver for the observational data dump script in the various
network runs. The script is being modified to remove logic which
handled the merging of split dumps and reprocessed dumps of QuikSCAT
data (one each for the first 1.5 hours of the 3.0 hour time window
and a second of each for the last 1.5 hours) into a final "qkscat"
dump and into a final "qkswnd" reprocessed dump. The QuikSCAT dump
had been split up to increase computational efficiency back in the
days of the Cray, but now a single "qkscat" dump is performed over
the full 3.0 time window radius followed by the generation of the
reprocessed dump "qkswnd".
278. DATACARDS - IBM Jobs 220, 281, 530, 820, 850, RUC2A_PREP,
RUC2B_PREP, CDAS_PREP1, AFPLT, GDAS_RECOVER_PREP. (Keyser, NP22).
The parm card files prepobs_prepdata.avn.parm,
prepobs_prepdata.cdas.parm, prepobs_prepdata.eta.parm,
prepobs_prepdata.fnl.parm and prepobs_prepdata.ruc.parm are read by
the program PREPOBS_PREPDATA which prepares observational data for
subsequent quality control programs and for subsequent analysis in
the AVN, CDAS, ETA/EDAS, FNL and RUC networks, respectively. The
parm card file prepobs_prepdata.afosplt.parm is read by the program
PREPOBS_PREPDATA which prepares early cutoff upper-air observational
data for subsequent AWIPS plots. In all cases, these data are then
encoded into the PREPBUFR file using a BUFR mnemonic table. The
parm cards are being modified to:
1) exclude CAP profiler wind reports from processing since they are
now in the "proflr" dump that is read in by PREPOBS_PREPDATA (all
files except "afosplt" and "cdas") - these data types must be tested
before they can written into the operational PREPBUFR files and then
assimilated;
2) exclude JMA profiler wind reports from processing since they are
now in the "proflr" dump that is read in by PREPOBS_PREPDATA ("avn"
and "fnl" files only) - these data types must be tested before they
can written into the operational PREPBUFR files and then
assimilated;
3) exclude European AMDAR (E-ADAS) aircraft reports from processing
since they are now in the "aircft" dump that is read in by
PREPOBS_PREPDATA (all files except "ruc") - these data types must be
tested before they can written into the operational PREPBUFR files
and then assimilated;
4) exclude mobile synoptic surface reports from processing since
they are now in the "adpsfc" dump that is read in by
PREPOBS_PREPDATA ("avn", "cdas" and "fnl" files only) - these data
types must be tested before they can written into the operational
PREPBUFR files and then assimilated;
5) switch off the processing of GOES 1x1 FOV cloud data since they
are no longer included in the "goesnd" dump that is read in by
PREPOBS_PREPDATA ("eta" file only) (these data have not been
assimilated from the ETA and EDAS PREPBUFR files since July 2003).
* * * * * * * * * * * * * * * * * * * * * * * * * * * *
The following data flow item was implemented this past week:
279. DATAFLOW - System TGFTP. (Gilbert, NP11). The
parsing.tbl controls actions taken by DBNET in response to an alert.
Entries have been added to the table on tgsv10/11 so that the Eta
GRIB fields on Alaska grid 242 will be available on the TOC Ftp
server. Done at 12Z on 6 April.
* * * * * * * * * * * * * * * * * * * * * * * * * * * *
The following data flow item will be implemented at 12Z on
Monday, 12 April:
280. DATAFLOW - System NCOSRV. (Freeman, NP11). The
parsing.tbl defines actions taken in response to an alert. The
process_ndfd.pl script reads the date of an NDFD file and
concatenate it to the correct hourly file. With these changes we
will stop making individual NDFD grid and only create a combined
hourly file.
* * * * * * * * * * * * * * * * * * * * * * * * * * * *
The following data flow item will be implemented at 12Z on
Thursday, 15 April:
281. DATAFLOW - System NCOSRV. (Freeman, NP11). The
spider system downloads and processes satellite imagery. It is
being modified to download DPD imagery for HPC.
-> Posted to jifmemo.
More information about the NCEP.List.Jifmemo
mailing list