[NCEP.List.Jifmemo] jif_memo_070907

Fri Sep 7 13:04:54 EDT 2007

                                    September 7, 2007 

MEMORANDUM FOR:    Distribution E 

FROM:		      John H. Ward 
		      Chief, Production Management Branch 

SUBJECT:	      Changes to the NWS Production Suite

1.  An unanticipated operational dependency on the Coastal Ocean Forecast System (COFS) was detected prior to its scheduled termination.  Product dissemination from the COFS was terminated on Wednesday, 5 September.  However, the COFS will continue to be run internally until all operational dependencies can be removed. 

2.  All GRIB1 data currently residing on the NCEP & TOC FTP servers will be removed on 28 January 2008.  The schedule for the GRIB1 to GRIB2 conversion for NOAAPORT data has not yet been established.  A schedule will be published as soon as it becomes available.  It is anticipated that no GRIB1 data will be removed from NOAAPORT prior to January 2008. 

NCEP provides software to convert GRIB2 formatted data back into GRIB1. Please use the link below to obtain the software utility to convert formats: 

http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/ 

Information regarding product equivalents for GRIB1 and GRIB2 on the NWS FTP TOC servers (tgdata.nws.noaa.gov and tgftp.nws.noaa.gov) and NCEP ftp server (ftp.ncep.noaa.gov) can be found via the link below: 

http://www.nco.ncep.noaa.gov/pmb/docs/GRIB1_to_GRIB2.shtml 

Please direct all questions concerning the software codes to the following NCO PMB Dataflow Helpdesk: NCEP.PMB.Dataflow at noaa.gov 

3.  Upcoming major changes to NCEP's production suite and details on any product or timing changes associated with upcoming implementations can be viewed at the following website: 

http://www.nco.ncep.noaa.gov/pmb/changes/ 

Test data for each of these proposed changes should be available six weeks prior to implementation.  Anyone wishing to receive formal notice of the final pre-implementation testing and the opportunity to provide feedback on the proposed changes can subscribe to the Model Evaluation mailing list via: 

https://lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.modelevalinfo 

4.  Maintenance of subscriptions to the JIF memo mailing list can be performed via a new list server at: 

https://lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.jifmemo

    *  *  *  *  *  *  *  *  *  *  *  *  *  *  *  *

    The following items will be implemented at 12Z on Tuesday, 11 September:     

    891.  HIRESW-COVER - IBM Job HIRESW.  (Gordon, NP11).  The High Resolution Window (HIRESW) forecast system is run on the CCS in production in the same slot as the hurricane runs, when resources allow.  There are two models run (NMM WRF and NCAR ARW) over two domains, four cycles per day.  If 1 or more hurricane runs are requested by TPC or CPHC, the amount of HIRESW runs made are reduced.  The following changes are being implemented:
• Increase the horizontal resolution of the WRF-NMM from 5.2 km to 4.0 km.
• Increase the horizontal resolution of the WRF-ARW from 5.8 km to 5.1 km.
• Upgrade WRF-NMM code: to keep pace with changes to the official public version of WRF code distributed by NCAR (from version 1.3 to 2.2 plus NCEP local changes reflected in the NAM) and to use IJK storage & loop ordering for ~10% improvement in efficiency.
• Upgrade WRF-ARW code to keep pace with changes to the official public version of WRF code distributed by NCAR (from version 1.3 to 2.2).
• Reconfigure the three CONUS (East, Central, and West) HIRESW domains into two larger domains (East-Central and West-Central)
• Change schedule of runs to: East-Central domain run twice/day at 00Z/12Z, West-Central domain run at 06Z and Alaska domain run at 18Z.
• Additional capability in the post-processor to generate new output fields: e.g. simulated GOES satellite fields and improved simulated radar reflectivity.
• Each CONUS run (00Z, 06Z & 12Z) will output two of the original HIRESW output grids.  This is a AOP milestone for EMC. 

    892.  HIRESW_NMM_REAL - IBM Job HIRESW_PRELIM.  (Pyle, NP2).  This program ingests output from the WPS program, defines the WRF-NMM vertical coordinate and vertically interpolates data to the model surfaces, and produces initial and lateral boundary condition files for the WRF-NMM model.  The program is being modified to update the code reflecting changes made over the last two years to the WRF-NMM running as the NAM, and brings the WRF infrastructure code up to the latest version 2.2 release.      

    893.  HIRESW_ARW_REAL - IBM Job HIRESW_PRELIM.  (Pyle, NP2).  This program ingests output from the WPS program, defines the WRF-ARW (EM) vertical coordinate and interpolates to the model surfaces, and produces initial and lateral boundary condition files for the WRF-ARW model.  The program is being modified to upgrade the HIRESW WRF code from version 1.3 to version 2.2 (about 2.5 year’s worth of upgrades).  In addition, the program is fixing bugs in various public releases of the WRF code.  This code is needed to ingest and vertically interpolate the WPS output files.     

    894.  HIRESW_WPS - IBM Job HIRESW_PRELIM.  (Pyle, NP2).  This new program processes GRIB output from the NAM/GFS, and interpolates horizontally onto the WRF model domain.  This new WRF Preprocessing System (WPS) replaces most of the functionality of the "wrsfi" code.      

    895.  HIRESW_NMM_FCST - IBM Job HIRESW_FCST.  (Pyle, NP2).  This program makes a WRF-NMM HIRESW forecast.  The following changes are being implemented:
1) Software type changes: represents an upgrade from V1.3 to V2.2 of the WRF software infrastructure 
2) Changes to the actual model:
a) More permissive horizontal diffusion: 
There is a restriction in the horizontal diffusion code so it is only applied for very flat model surfaces.  In the older WRF-NMM code, horizontal diffusion between points would only be performed if the slope (change in height of a model surface) between two points was < 5.9 m / 12 km.  This limit has been raised by a factor of nine, so slopes < ~54 m / 12 km allow diffusion.  This change alone doesn't increase the strength of horizontal diffusion, but greatly increases the number of points for which at least some horizontal diffusion is applied.  Previously much of the western U.S. felt absolutely no horizontal diffusion at lower levels of the atmosphere.
b) Stronger horizontal diffusion:  Horizontal diffusion is 8X stronger than the nearly non-existent levels previously used in the HIRESW.  New larger value matches the diffusion strength coefficient (COAC) used in the operational NAM.
c) "External" mode divergence damping:
Divergence damping in the NMM previously was strictly done level by level, damping the so-called "internal" modes.  New code was added to damp divergence in an integrated sense, reducing the "external" mode. 
The combination of the above three model changes should reduce spurious noise and overly localized behavior in the WRF-NMM run of the HIRESW.
d) IJK indexing: Previous version had IKJ indexing on 3D variables, which reduces performance relative to an IJK model on the CCS.    

    896.  HIRESW_ARW_FCST - IBM Job HIRESW_FCST.  (Pyle, NP2).  This program makes the HIRESW model forecast using the WRF-EM model.  This program is being replaced with a new source code.  Numerous bug fixes and upgrades are obtained by using the latest (v2.2) code.  
Also, the previous 5.8 km resolution in the ARW run of the HIRESW was too coarse to justify running without parameterized convection.  

    897.  HIRESW_POST - IBM Job HIRESW_POST.  (Pyle, NP2).  The WRF post-processing code reads binary forecast output (wrfout* files) from the WRF-NMM or WRF-ARW and generates GRIB output on the native horizontal grid of each model.  The program is being upgraded; this includes the capability to produce fields such as simulated reflectivity that were not available from the older HIRESW wrfpost code.  In addition, provision for ingesting the IJK ordered output from the new WRF-NMM model code is being included.   

    898.  HIRESW_PRDGEN - IBM Job HIRESW_PRDGEN.  (Pyle, NP2).  This program horizontally interpolates native GRIB output from the HIRESW runs onto standard output grids.  This program is being modified to no longer run within the HIRESW post job; it will run separately as it is done in the NAM.  The program is needed to process the large dimensions of the input and output GRIB files.  New weights files are used for interpolation from the revised native grids to the same standard output grids.  There is no longer a "cent" integration domain, but the 8 km central output grid will be filled by output from the west and east integration domains.  New output grids that cover the computational domain at 4 km grid spacing are being created for the two CONUS domains (west and east).  

    899.  HIRESW GRAPHICS - IBM Job JHIRESW_GEMPAK_GIF.  (Sager, NP12).  Script JHIRESW_GEMPAK_GIF creates a series of graphics from HIRESW WRF model output for the Model Analysis and Forecast webpage.  The scripts are being modified to handle changes to the HIRESW WRF model upgrade planned to be implemented in mid-September 2007.  The changes include expanding West region by 20 degrees further east and East region 20 degrees further west.  The old Central region is being dropped.  A new creation time schedule for both ARW and NMM regions are being adopted as follows: 
00Z and 12Z -- Eastern Region, 
06Z -- Western Region, 
18Z -- Alaska Region.  
Also, many changes are being made in the GEMPAK scripting to cope with the increased resolutions output by the updated HIRESW WRF model.    

    900.  DATAFLOW-PMB - Systems tgp06, tgp07.  (O'reilly, NP11).  The DBNet parsing.tbl defines actions to be taken in response to an alert.  The parsing.tbl is being modified to begin sending the HIRESW upgrade to the TOC and WOC systems in GRIB2 format.  DBNet GRIB2 conversion of these files is being eliminated.  Entries for sending the parallel set of these data are also being eliminated.  This is for the HIRESW upgrade, NCEP Q4 milestone.  

    901.  SCRIPT - IBM Job JRTGSST.  (Katz, NP2).  Script exrtgsst.sh.sms generates the real-time global half-degree (RTG_SST) SST analysis.  With this change, a file containing satellite bias corrections will be generated by program SST2DVAR_BIASCOR and stored in /com.  This script is being modified to have the same capability as script exrtgssthr.sh.sms, which generates the real-time global 1/12-degree (RTG_SST_HR) analysis.  It is being changed to have a convenient means of comparing the satellite bias corrections from the RTG_SST analysis with those from the RTG_SST_HR analysis.

    902.  SST2DVAR_BIASCOR - IBM Job JRTGSST.  (Katz, NP2).  This program corrects biases in satellite SST data used by the real-time, half-degree resolution global (RTG_SST) SST analysis.  The program is being modified to write out the bias correction fields for subsequent display.  This will enable program SST2DVAR_BIASCOR to have the same capability as program RTGSSTHR_BIASCOR in the real-time, 1/12-degree resolution global (RTG_SST_HR) SST analysis.  It is being changed to have a convenient means of comparing the satellite bias corrections from the RTG_SST analysis with those from the RTG_SST_HR analysis. 

    903.  SCRIPT AND DATA FILES - IBM Jobs GFSMOS_METAR_FCST/EXTFCST, GFSMOS_STATION_PRDGEN/EXTPRDGEN.  (Cosgrove, OST22).  These scripts and files run various portions of the GFS MOS process -- generating forecasts for the METAR stations and packaging the MOS forecasts in text and binary products for dissemination to the public.  The gfsmos job is being modified to add stations to those available in the visibility, obstruction to vision, and probability of precipitation characteristic guidance.  Also, being updating are the climatologically maximum and minimum temperature normals available in the extended-range GFS MOS text product to use the PRISM climatologies where possible.  The addition of stations was promised to users when resources allowed us to do it, and users have been asking us to update the normals for quite some time.

    904.  MDLLIB AND MDL_MOSPRED - Many MDL jobs.  (Cosgrove, OST22).  The MDLLIB library contains subroutines used by the various MDL programs.  The code mdl_mospred interpolates model predictors in MDL's packed format to station locations and produces computed predictors for input to the MOS equations.  These routines are being changed to update the max/min normals in the MEX text product to use the PRISM climatologies where available.    

    905.  MDL_GFSMEXTX - IBM Jobs GFSMOS_STATION_EXTPRDGEN_00, 12.  (Cosgrove, OST22).  This program creates the GFS-based extended-range MOS text products.  The program is being modified to use the PRISM normals for max/min temperature climatology where available.    

    906.  MDL_SEQ2RA - IBM Jobs GFSMOS_METAR_EXTFCST_00, 12.  (Cosgrove, OST22).  This new program transforms MOS data from one internal binary format to another.  

    *  *  *  *  *  *  *  *  *  *  *  *  *  *  *  *

    The following data flow item will be implemented at 12Z on Monday, 10 September:    

    907.  DATAFLOW-SMS - IBM Job machines smsfmt2, smsgb1, and smsgb2.  (Cooke, NP11).  The SMS server is responsible for scheduling and submitting jobs to the CCS.  The SMS server code is being modified to have the variable SMSINTERVAL set to 20 seconds instead of the current 60.  It has been noted that occasionally jobs are submitted nearly a minute after their scheduled time.  This appears to be due to a 60 second "wait" that the SMS server performs before checking to see if any jobs need to be submitted. NCO management has directed this change.

    *  *  *  *  *  *  *  *  *  *  *  *  *  *  *  *

    The following data flow items will be implemented at 12Z on Tuesday, 11 September:

    908.  DATAFLOW-PMB - System ncosrv.  (Freeman, NP11).  The McIDAS software is used by Spider to download images from NESDIS servers.  It is being modified to install the latest McIDAS software in a new mcidastest account to be used by test Spider.  It is being changed to test the upgrade requested by NESDIS required to processes new MTS imagry.

    909.  DATAFLOW-PMB - System ncosrv.  (Freeman, NP11).  The scppush.pl script pushes files to remote hosts using SCP protocol.  The script is being modified to turn off the SCP feed to front5.  The front5 has been replaced with new host, front.  The parallel feed has been in place for several weeks. 

    *  *  *  *  *  *  *  *  *  *  *  *  *  *  *  *

    The following data flow items will be implemented at 12Z on Wednesday, 12 September:

    910.  DATAFLOW - SCRIPT - IBM Job SR_HH.  (O'reilly, NP11).  Script JSRUC.sms.prod sets up environment and initiates job SR_HH which produces surface RUC GRIB output.  The script is being modified to turn on GRIB2 DBNet alerts.  The DBNet GRIB2 processing of these data is being turned off in related JIF, and GRIB2 will now be sent from production.  The alerts need to be turned on so DBNet will be able to send the production GRIB2.

    911.  DATAFLOW - PMB - Systems CCS, WOC Diskserver, tgp06, tgp07.  (O'reilly, NP11).  The parsing.tbl defines actions to be taken in response to an alert. The scp.tbl defines files to be ftp'd to remote sites. The exec_table defines executable actions to be taken in response to an alert.   The DBNet tables on the CCS and WOC diskserver are being modified to begin sending GRIB2 RUCS data created in production to the WOC diskserver and the tgp06/07 machines.  Also, the DBNet conversion of these data on the CCS will be eliminated.  GRIB2 to GRIB1 processing will be added to the WOC diskserver.  This change is to eliminate the DBNet GRIB conversion, and begin sending GRIB2 created in production.  Also, it will eliminate sending both GRIB1 and GRIB2 datasets to the WOC diskserver.

    912.  DATAFLOW-PMB - Systems MIST/DEW.  (Cooke, NP11).  The ctab file describes what belongs in DBNet's cron.  An entry is being added to run a script that will send DBNet code and configuration files to the subversion system.  This program is being change to have an automated way to make use of the subversion repository.

    *  *  *  *  *  *  *  *  *  *  *  *  *  *  *  *

    The following data flow item will be implemented at 12Z on Thursday, 13 September:

    913.  DATAFLOW-PMB - System CCS.  (Freeman, NP11).  The parsing table defines actions to be taken in response to an alert.  This change is to create the push queues to the new WOC system and turn on a parallel feed to the new WOC server for the 12Z RUC only.  This change will enable direct comparison of transfer rates and latencies between the old and new server.