[Ncep.list.pmb-dataflow] Re: GOES-13 acting for GOES-12
Russ Treadon
Russ.Treadon at noaa.gov
Tue Dec 16 10:53:45 EST 2008
All,
Thanks, Dennis for looking into what's happening with the satellite
winds.
I added some print statements in the Q1FY09 global_gsi and checked the
satellite ID for the GOES satwnd data which we assimilate. Just as
Dennis indicates, we are now assimilating satellite winds from SAID=257
(GOES-13). The global assimilates prepbufr report types 245 (ir) and
246 (wv-imager) GOES derived satellite winds.
Routine read_prepbufr of the GSI code along with the GSI convinfo file
could be modified to NOT assimilate the GOES-13 satellite winds. I now
modifying my copy of the Q1FY09 global_gsi code and convinfo file to
test these changes.
As far as assessing the quality of the GOES-13 satellite winds, I'm not
sure what to do. Ideally, there would be a period of GOES-12 and
GOES-13 overlap during which we could assimilate both independently and
quantify the analysis / forecast impact of the two different data
feeds. We can't do this now, at least not in real-time. We could
compare analysis increments which arise from the assimilation of GOES-12
and -13 data for different cases. This would allow us to get a rough
feel for GOES-12 -vs- GOES-13 analysis increments. Hopefully the wind
obs from the two platforms generate comparable increments. Due to
differences in the location of the satellite (GOES-13 @ 105W, GOES-12 @
75W) this type of comparison is far from being ideal.
The above checks are only for the GDAS. The NAM/NDAS and RUC also
assimilate GOES satellite derived winds. GOES-13 impact on these
analysis / forecast systems may also need to be assessed.
Russ
Dennis Keyser wrote:
> All,
>
> It appears that NESDIS has fixed the problem whereby the value for
> "satellite derived wind calculation method" (BUFR descriptor 0-02-023,
> NCEP BUFR mnemonic "SWCM") was set to "missing" for the GOES-13 IR,
> WV-imager and visible winds. The "good" news is that these are now
> being ingested, dumped in the "satwnd" files, and encoded into the
> PREPBUFR files. The "maybe?" bad news is that all of out production
> networks are now assimilating the GOES-13 IR and wv-imager satwinds.
> Hopefully these new platform data are of sufficient quality that they
> will not degrade any analyses.
>
> Dennis
>
> On 12/15/2008 2:30 PM, Thomas Renkevens wrote:
>> I will pass this along to winds folks...
>>
>> As to the satellite number vs "E" or "W," from my recollection after
>> the switch from GOES-8 to GOES-12 in April 2001, at the request of
>> EMC (don't recall who...) we switched products from the numbers to
>> the "E" and "W" to avoid confusion in the numbers, but that, as we
>> see here, can cause confusion the other way...
>>
>> I need to run to a meeting to discuss GOES-12 now, but will follow up
>> more late today or tomorrow. But if EMC feels strongly about
>> switching to a number scheme, this request may needed to be vetted
>> through the SPSRB for action.
>>
>> Tom
>>
>> Dennis Keyser said the following on 12/15/2008 2:14 PM:
>>> All,
>>>
>>> I received the following e-mail from Justin:
>>>
>>>> Dennis,
>>>>
>>>> I just got a call from Hongming Qi of NESDIS about the files that
>>>> are still missing according to Big Brother, basically all of the
>>>> satwnd data is flagging. I don't know too much about the satellite
>>>> ingest processing but looking at the output of the
>>>> satwnd_1535.o2684151 job in /com/output/prod/today on Dew I see
>>>> many lines saying:
>>>>
>>>> The value of satellite derived wind method is missing or not a
>>>> valid value
>>>> The value of satellite derived wind method is missing or not a
>>>> valid value
>>>> The value of satellite derived wind method is missing or not a
>>>> valid value
>>>> The value of satellite derived wind method is missing or not a
>>>> valid value
>>>>
>>>> Is this because the data is from GOES-13?
>>>>
>>>> Thanks,
>>>> Justin
>>>
>>> What this indicates is that NESDIS has indeed swapped GOES-13 with
>>> GOES-12 in the various "goesE" satwind files we pull from their
>>> satepsdist1e.nesdis.noaa.gov server. I had noted in my earlier
>>> e-mails below that I had not seen any GOES-13 data in the satwind
>>> dumps which I had assumed was because NESDIS had not performed this
>>> swap (and that instead no "goesE" satwind files were being
>>> produced). Instead what is actually is happening is that NESDIS is
>>> putting GOES-13 in the "goesE" satwinds files but our satellite wind
>>> ingest is not processing them because NESDIS is erroneously setting
>>> the BUFR code table value value for "satellite derived wind
>>> calculation method" (BUFR descriptor 0-02-023, NCEP BUFR mnemonic
>>> "SWCM") to "missing". This value is supposed to be set as follows:
>>>
>>> 1 - IR
>>> 2 - VISIBLE
>>> 3 - WATER VAPOR (IMAGER AND SOUNDER) - CLOUD TOP
>>> 4 - COMBINATION OF SPECTRAL CHANNELS OR PICTURE TRIPLET
>>> 5 - WATER VAPOR (IMAGER AND SOUNDER) - CLEAR AIR (DEEP LAYER)
>>> 6 - RESERVED FOR OZONE
>>> 7 - WATER VAPOR (IMAGER AND SOUNDER) - CLOUD TOP OR CLEAR AIR UNKNOWN
>>>
>>> For visible, wv-imager and ir from GOES-13 this value is missing and
>>> is triggering the hundreds of lines of diagnostic print Justin is
>>> seeing above. Interestingly, for wv-sounder from GOES-13 it is
>>> being set to the proper values of 3 and 5 and is being ingested. We
>>> do not dump wv-sounder winds from GOES, however.
>>>
>>> So here two wrongs have made a right. The missing value for SWCM in
>>> GOES ir, wv-imager and visible wind reports is preventing these data
>>> from being ingested. Otherwise they would be dumped and assimilated
>>> (ir and wv-imager only) by the NAM/ and GBL/GSI as well as the RUC.
>>> We will continue to see these diagnostic lines in the ingest output
>>> for GOES-13 as long as SWCM is missing.
>>>
>>> This further points to the need for NESDIS to either not produce
>>> "goesE" files when a swap is made (or "goesW" for that matter) or,
>>> better yet, begin generating wind and sounding files with the
>>> specific satellite number in them rather than "goesW" and "goesE".
>>>
>>> Dennis
>>>
>>>
>>> On 12/15/2008 1:35 PM, Dennis Keyser wrote:
>>>> Tom,
>>>>
>>>> Please see my comments below.
>>>>
>>>> Thanks,
>>>> Dennis
>>>>
>>>> On 12/15/2008 12:44 PM, Thomas Renkevens wrote:
>>>>> Dennis,
>>>>>
>>>>> First, a GOES-12 update - we will be running in a GOES-13
>>>>> configuration until at least tomorrow. More will be learned and
>>>>> shared late afternoon as to the status and recovery of GOES-12.
>>>> Thanks for the update.
>>>>>
>>>>>
>>>>> We apologize for the confusion... I do not recall any specific
>>>>> agreement, but in one of my emails to you and others in September
>>>>> preparing for a possible GOES-11 failure, I stated :/*"*As to BUFR
>>>>> ID, I have repeatedly passed along the information about the SATID
>>>>> to the products folks, so that any GOES-13 products that are made
>>>>> have this appropriate ID, unlike some of the problems from the
>>>>> GOES-12 vs GOES-10 issues last December." /and perhaps we failed
>>>>> to fully communicate our intentions.
>>>> I may have misunderstood the agreement, sorry about that. I think
>>>> our concern is based on past experience where the wrong satellite
>>>> number was introduced into the satellite sounding files. In our
>>>> opinion, it would be better to not even produce the GOES-E sounding
>>>> files (when there is a satellite swap) than to produce them with
>>>> the possibility of an error in the satellite id. This time it
>>>> worked properly, but we have been burned in the past, as you know,
>>>> and we are always a bit paranoid that it could happen again in the
>>>> future. In addition, while the GFS and NAM parse out satellite id
>>>> in their (GSI) analyses, I have no idea what the RUC does. Luckily,
>>>> it appears that the PREPBUFR processing is flagging the GOES-13 PW
>>>> and cloud obs but I'm still not 100% sure that RUC isn't attempting
>>>> to assimilate these.
>>>>
>>>> In addition, neither the GSI (NAM, GFS) nor RUC analyses examine
>>>> satellite id for the satwinds, so if the GOES-13 winds begin
>>>> appearing in the GOES-E satwind files in place of GOES-12, we will
>>>> assimilate these in all networks. Earlier when I looked, it
>>>> appeared that NESDIS was not producing GOES-E satellite wind files
>>>> since the switch, so we seem to be ok right now.
>>>>>
>>>>> As a result from the GOES-12 to GOES-10 satellite switch last
>>>>> December 2007 (also due to thruster problems on GOES-12), where
>>>>> NESDIS erroneously created products with wrong satellite IDs in
>>>>> the BUFR files, our production procedures are as follows:
>>>>>
>>>>> * *For products with the satellite number within the file name
>>>>> (i.e G11 or G12), these products would be produced with file
>>>>> names containing "G13" from GOES-13*
>>>>> o *These are only the imager radiance files from the DDS
>>>>> and the CSBT from SATEPSDIST1.
>>>>> *
>>>>>
>>>> This would actually be the preferred file nomenclature for all GOES
>>>> files we pull from you - soundings, winds, etc. By explicitly
>>>> identifying the GOES satellite number in the file names, we can
>>>> then pull only the files we want (e.g., right now we'd pull G11 and
>>>> G12 and not G13, so if you stopped making G12 and started making
>>>> G13 it would be transparent to our system). Is there any
>>>> possibility of transitioning all GOES files names from goesE and
>>>> goesW to Gxx where xx is the specific satellite? All of the POES
>>>> satellite files we pull from DDS are identified with the satellite
>>>> number in the file names and this works out very well in our
>>>> processing.
>>>>>
>>>>> o * *
>>>>> * *For products (winds, sounder products) with the satellite
>>>>> location with the file name (i.e. "E" for East or "W" for
>>>>> West) these products would be produced with the /same/ file
>>>>> names in the same file directory location, but the WMO SATID
>>>>> number within the file /would be changed/ to reflect the
>>>>> satellite ID number:*
>>>>> o *GOES-11 255*
>>>>> o *GOES-12 256*
>>>>> o *GOES-13 257*
>>>>> o *GOES-14 258*
>>>>> o *GOES-15 259*
>>>>>
>>>> Again see my above comment about changing these files names to
>>>> explicitly include the satellite number.
>>>>
>>>>
>>>> Thanks Tom.
>>>>
>>>>> * *With these conventions, users (NWS and others) can identify
>>>>> the satellite number either through the file name or the WMO
>>>>> Satellite ID in the BUFR file and make their own appropriate
>>>>> choices to assimilate or not to assimilate the satellite
>>>>> products. We were under the impression that those who grab
>>>>> these files, are able to decode the satellite ID and make
>>>>> decisions as to ingest or not based on the satellite ID.*
>>>>>
>>>>> (FYI... With McIDAS, there is a different name convention as follows:)
>>>>> # / Sensor Source
>>>>> 76 GOES-11 Imager
>>>>> 77 GOES-11 Sounder
>>>>> 78 GOES-12 Imager
>>>>> 79 GOES-12 Sounder
>>>>>
>>>>> 180 GOES-13 Imager
>>>>> 181 GOES-13 Sounder
>>>>> 182 GOES-14 Imager
>>>>> 183 GOES-14 Sounder
>>>>> 184 GOES-15 Imager
>>>>> 185 GOES-15 Sounder
>>>>>
>>>>>
>>>>> If you find that we have NOT changed the SATID in the files,
>>>>> please let us know, but we have been assured by programmers that
>>>>> this has taken place.
>>>>> If you have further questions, please feel free to call or email.
>>>>>
>>>>> Thanks
>>>>> Tom
>>>>>
>>>>>
>>>>> Dennis Keyser said the following on 12/15/2008 11:19 AM:
>>>>>> Justin and everyone,
>>>>>>
>>>>>> I checked the dump status files. We are currently only dumping
>>>>>> satellite wind data from GOES-11 (sat. id 255), so NESDIS did not
>>>>>> replace GOES-12 with GOES-13 in their GOES-E files
>>>>>> (satwnd.bufrCD.goesE.D?????.T??:??:??Z,
>>>>>> satwnd.bufrVZ02.goesE.D?????.T??:??:??Z,
>>>>>> satwnd.bufrWV.goesE.D?????.T??:??:??Z). Thus we are only
>>>>>> assimilating winds from GOES-11 right now. At this point, we
>>>>>> would not want to introduce GOES-13 satwinds without some testing
>>>>>> so this is the correct action taken by NESDIS. NESDIS has agreed
>>>>>> not to swap satellites in their goesE and goesW satwind files.
>>>>>> Especially if this is just a short (hopefully!) GOES-12 outage.
>>>>>> As for the radiances (and layer PW and cloud data for the RUC) -
>>>>>> I had thought this same agreement (not to swap satellites in the
>>>>>> goesE and goesW files) was also in place. However, in the dumps
>>>>>> I now see GOES-13 (sat. id 257) replacing GOES-12 (sat. id 256)
>>>>>> in the GOES-E 1x1 sounding/PW file
>>>>>> (satsnd.goesE.D??????????.T????Z.1B1) and in the GOES-E cloud
>>>>>> file (goesE.RMD.D?????.T????.Z). This validates what Russ is
>>>>>> seeing in the assimilation. As Russ noted, the GFS/GSI monitors
>>>>>> GOES-13 radiances, but from what I see the current NAM/GSI also
>>>>>> monitors GOES-13 radiances (see
>>>>>> /nwprod/fix/nam_regional_satinfo.txt) - Russ had indicated that
>>>>>> the NAM/GSI did not see GOES-13 radiances so maybe the satinfo
>>>>>> file I'm looking at isn't the ops file(?). *In any event, in
>>>>>> these cases of satellite swapping, NESDIS should not have swapped
>>>>>> GOES-12 with GOES-13 in their GOES-E sounding files (just as they
>>>>>> correctly did not do so in their GOES-E satwind files).*
>>>>>>
>>>>>> I checked the RUC and switches in the PREPBUFR processing are set
>>>>>> to flag GOES-13 cloud and layer PW data with q.m. 15 (don't use),
>>>>>> so as long as the RUC is honoring these quality marks it should
>>>>>> not be using the GOES-13 cloud or layer PW data.
>>>>>>
>>>>>> Dennis
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 12/15/2008 9:32 AM, Justin Cooke wrote:
>>>>>>> Dennis,
>>>>>>>
>>>>>>> I'm not sure if you're aware or not but there was a failure of
>>>>>>> GOES-12 yesterday around 2235Z, starting around 0430Z today
>>>>>>> NESDIS made GOES-13 act for GOES-12. At the 9:00 am operations
>>>>>>> meeting Tom Renkevens mentioned that NESDIS should be encoding
>>>>>>> all of the GOES-13 products that we are ingesting as GOES-13
>>>>>>> products and not GOES-12. Last year GOES-12 had a similar
>>>>>>> failure and GOES-10 acted for it but products were made as "look
>>>>>>> alike", this made some of our ingesting inaccurate since there
>>>>>>> are differences between GOES-12 and GOES-10. We were wondering
>>>>>>> if the ingest processing is capable of successfully using data
>>>>>>> from GOES-13? All of our dump processes are working without
>>>>>>> error, but beyond that I'm not sure how to investigate this.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Justin
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 12/15/2008 8:05 AM, Russ Treadon wrote:
>>>>>>> Bill,
>>>>>>> The use of GOES radiances in the operations and parallel
>>>>>>> GFS/GDAS and NAM/NDAS looks OK. No action is needed. GOES-12
>>>>>>> data is no longer present. GOES-13 data is being read and
>>>>>>> processed as GOES-13 data.
>>>>>>>
>>>>>>> The GFS/GDAS use GOES derived satellite winds. I see a decrease
>>>>>>> in the total number of type 245 and 246 satwnd obs being
>>>>>>> assimilated in both the GFS and GDAS. This is consistent with
>>>>>>> the loss of GOES-12. Should GOES-13 based satellite derived
>>>>>>> wind replace the GOES-12 product? I don't know how NESDIS
>>>>>>> generates the product. It may take some time for them the
>>>>>>> switch the data feed for this processing. Or it may be that we
>>>>>>> need to change something in our processing upstream of the
>>>>>>> global analysis in order to pick up GOES-13 derived wind
>>>>>>> products. I don't know.
>>>>>>>
>>>>>>> I am less familiar with the use of GOES products in the
>>>>>>> NAM/NDAS. I checked Dennis' NAM/NDAS prepbufr report type
>>>>>>> table. The NAM/NDAS appears to use the same GOES products that
>>>>>>> the global uses - namely, satellite derived winds. Does the
>>>>>>> parallel (to-be implemented) NAM/NDAS use other GOES derived
>>>>>>> products? I don't know.
>>>>>>>
>>>>>>> I don't know what action, if any, we need to take for the RUC.
>>>>>>> Russ
>>>>>>>
>>>>>>> Bill Lapenta wrote:
>>>>>>>> Gentlemen--
>>>>>>>>
>>>>>>>> Are there any actions (i.e., RFC's) that we should consider
>>>>>>>> taking?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Bill
>>>>>>>>
>>>>>>>> Russ Treadon wrote:
>>>>>>>>
>>>>>>>>> All,
>>>>>>>>>
>>>>>>>>> As described in the ESPCoperations email below, we are no longer
>>>>>>>>> receiving GOES-12 sounder radiances. The GOES-13 data feed
>>>>>>>>> was swapped
>>>>>>>>> into the GOES-EAST slot.
>>>>>>>>>
>>>>>>>>> I checked the GFS/GDAS and see that the global_gsi is no longer
>>>>>>>>> processing GOES-12 data. The 20081215 00Z cycle had a noticeable
>>>>>>>>> decrease in the number of GOES-12 sounder radiances. The 06Z
>>>>>>>>> cycle
>>>>>>>>> contains no GOES-12 sounder radiances. GOES-13 sounder
>>>>>>>>> radiances are
>>>>>>>>> present in the 20081215 06Z cycle. We are not assimilating
>>>>>>>>> the GOES-13
>>>>>>>>> radiances. The usage flag for this data is set to -1 (do not
>>>>>>>>> assimilate). Thus, GOES-13 data is only being monitored (as
>>>>>>>>> intended).
>>>>>>>>>
>>>>>>>>> The last NAM cycle with GOES-12 data is 18Z, 20081214.
>>>>>>>>> Today's 00Z and
>>>>>>>>> 06Z NAM "fits" files indicate no GOES-12 data. The tm12 and
>>>>>>>>> tm09 steps
>>>>>>>>> of today's 06Z NDAS cycle contain GOES-12. The tm06 and tm03
>>>>>>>>> NDAS steps
>>>>>>>>> were without GOES-12 radiances. Today's 12Z NDAS cycle
>>>>>>>>> contains no
>>>>>>>>> GOES-12 data in any of the steps from tm12 to tm03.
>>>>>>>>>
>>>>>>>>> The operational NAM/NDAS does not include GOES-13 data in the
>>>>>>>>> satinfo
>>>>>>>>> file nor in the ex*gsireg analysis script. The to-be
>>>>>>>>> implemented
>>>>>>>>> NAM/NDAS system includes GOES-13 data - but only in monitor
>>>>>>>>> mode as is
>>>>>>>>> done in the global. The NCO para NAM/NDAS gsistat files show
>>>>>>>>> GOES-13 radiances in the
>>>>>>>>> 20081215 06Z NAM cycle and the tm06 and tm03 steps of the
>>>>>>>>> 20081215 12Z
>>>>>>>>> NDAS cycle. The GOES-13 data is being monitored, not
>>>>>>>>> assimilated (as
>>>>>>>>> intended).
>>>>>>>>>
>>>>>>>>> I do not know if/how the RUC uses GOES-12 data. I did not
>>>>>>>>> check the
>>>>>>>>> status of GOES-12 derived products (satwnd, cloud products,
>>>>>>>>> precipitable
>>>>>>>>> water, etc). I only checked the usage of GOES-12 radiances.
>>>>>>>>>
>>>>>>>>> We experienced a GOES-12 data outage last December. During
>>>>>>>>> the initial
>>>>>>>>> stages of this outage, GOES-10 sounder radiances were
>>>>>>>>> mislabeled as
>>>>>>>>> GOES-11. As a result we briefly assimilated GOES-10 data as
>>>>>>>>> though it
>>>>>>>>> were GOES-11 data.
>>>>>>>>>
>>>>>>>>> Russ
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -------- Original Message --------
>>>>>>>>> Subject: GOES-12/13 OUTAGE UPDATE #3
>>>>>>>>> Date: Mon, 15 Dec 2008 04:01:43 -0500
>>>>>>>>> From: ESPCoperations <ESPCoperations at noaa.gov>
>>>>>>>>> To: _NESDIS OSDPD ESPC OUTAGE Notification
>>>>>>>>> <ESPC.Notification at noaa.gov>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *UPDATE: 12/15/2008, 0845 UTC, 03:45 AM, EST
>>>>>>>>>
>>>>>>>>> *ESPC is receiving GOES-13 imagery on the GINIs and IQM
>>>>>>>>> (SATEPSDIST
>>>>>>>>> servers), but several users are not
>>>>>>>>> receiving any data or partial data. Also, GOES-13 data is
>>>>>>>>> being ingested
>>>>>>>>> and distributed on GOES AFEP-2 (primary),
>>>>>>>>> but it is not getting to OPUS Diamond and OPUS DDS to be
>>>>>>>>> processed. A
>>>>>>>>> SOCC engineer, the Sys Admin for
>>>>>>>>> AFEPS and the Sys Admin for OPUS Diamond and OPUS DDS are
>>>>>>>>> working to
>>>>>>>>> resolve the problem(s).
>>>>>>>>>
>>>>>>>>> *UPDATE:* GOES-13 will be switched to GOES-East beginning at
>>>>>>>>> 0500 UTC
>>>>>>>>> (12:00 p.m. EST).
>>>>>>>>> *
>>>>>>>>> Details of the Outage: *At around 2200 UTC, GOES-12 suffered a
>>>>>>>>> thruster
>>>>>>>>> anomaly causing the suspension of GOES-12 imaging and sounding
>>>>>>>>> schedules. At 2303 UTC, GOES-11 (GOES-West) was placed into
>>>>>>>>> Full Disk
>>>>>>>>> mode to cover for the GOES-12 outage. At that time, the GOES
>>>>>>>>> ingest and
>>>>>>>>> NOAAPORT Interface (GINI) was switched to alternate mode to allow
>>>>>>>>> delivery of GOES-11 data to East users.
>>>>>>>>>
>>>>>>>>> At around 0500 UTC, once ingest of GOES-13 data is verified by
>>>>>>>>> ESPC, the
>>>>>>>>> GINI and all GOES-East products will be fed with GOES-13
>>>>>>>>> imager and
>>>>>>>>> sounder data.
>>>>>>>>>
>>>>>>>>> ALL Services (LRIT, EMWIN, DCS, SARSAT) will remain on
>>>>>>>>> GOES-12, and
>>>>>>>>> GOES-13 GVAR will be retransmitted through GOES-12. Users
>>>>>>>>> currently
>>>>>>>>> receiving services including GVAR from GOES-12 should not need to
>>>>>>>>> repoint to GOES-13.
>>>>>>>>>
>>>>>>>>> It is anticipated that this configuration will remain in place
>>>>>>>>> for at
>>>>>>>>> least 24 hours.
>>>>>>>>> * *
>>>>>>>>> *Data Affected by the Outage:
>>>>>>>>> *
>>>>>>>>> All products from GOES-12
>>>>>>>>>
>>>>>>>>> *Date and Time of the Outage:
>>>>>>>>> *
>>>>>>>>> 2218 UTC , 5:18 pm EST
>>>>>>>>>
>>>>>>>>> *Length of the Outage:
>>>>>>>>> *Unknown* *
>>>>>>>>>
>>>>>>>>> *Contact Point for Questions:
>>>>>>>>> ESPC Operations OCL (Opera*
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
More information about the Ncep.list.pmb-dataflow
mailing list