This page provides a detailed description of the quality control (QC) processing and data structures for the radiosonde dataset. For a general overview of MADIS quality control, click here.
MADIS Radiosonde Variables with QC ---------------------------------------------------------------------------------------------------- Code Name Max Level | Level 1: | Level 2: Possible | | QC Level | Validity | Super-Adiabatic Hydrostatic Wind-Shear | | Lapse Rate ---------------------------------------------------------------------------------------------------- Integrated Sounding Sig Mandatory ------------------- --- --------- DD wind direction 1 2 X FF wind speed 1 2 X U u wind component 1 2 X V v wind component 1 2 X T air temperature 2 2 X X X + correction TV virtual temperature 2 2 X X X + correction TD dewpoint temperature 1 1 X RH relative humidity 1 1 X Q specific humidity 1 1 X DPD dewpoint depression 1 1 X AH absolute humidity 1 1 X WVMR water vapor mix ratio 1 1 X P pressure 1 1 X HT geopotential height 1 2 X X + correction Mandatory Levels ---------------- DDMAN wind direction 2 X X FFMAN wind speed 2 X X UMAN u wind component 2 X X VMAN v wind component 2 X X TMAN air temperature 2 X X X + correction TVMAN virtual temperature 2 X X X + correction TDMAN dewpoint temperature 1 X RHMAN relative humidity 1 X QMAN specific humidity 1 X DPDMAN dewpoint depression 1 X AHMAN absolute humidity 1 X WVMRMAN water vapor mix ratio 1 X PMAN pressure 1 X HTMAN geopotential height 2 X X + correction Significant Levels ------------------ TSIG air temperature 2 X X TVSIG virtual temperature 2 X X TDSIG dewpoint temperature 1 X RHSIG relative humidity 1 X QSIG specific humidity 1 X DPDSIG dewpoint depression 1 X AHSIG absolute humidity 1 X WVMRSIG water vapor mix ratio 1 X PSIG pressure 1 X HTSIG geopotential height 1 X Wind Levels ----------- DDWIND wind direction 1 X FFWIND wind speed 1 X UWIND u wind component 1 X VWIND v wind component 1 X PWIND pressure 1 X HTWIND geopotential height 1 X Maximum Wind Levels ------------------- DDMXWND wind direction 1 X FFMXWND wind speed 1 X UMXWND u wind component 1 X VMXWND v wind component 1 X PMXWND pressure 1 X HTMXWND geopotential height 1 X Tropopause Levels ----------------- DDTROP wind direction 1 X FFTROP wind speed 1 X UTROP u wind component 1 X VTROP v wind component 1 X TTROP air temperature 2 X X TVTROP virtual temperature 2 X X TDTROP dewpoint temperature 1 X RHTROP relative humidity 1 X QTROP specific humidity 1 X DPDTROP dewpoint depression 1 X AHTROP absolute humidity 1 X WVMRTROP water vapor mix ratio 1 X PTROP pressure 1 X HTTROP geopotential height 1 X
The level 1 validity checks restrict each observation to falling within a TSP-specified set of tolerance limits. Observations not falling within the limits are flagged as failing the QC check. The following table lists the tolerance limits:
--------------------------------------------------- Validity Checks Geopotential Height(m) Temp.(C) Max Wind Level(mb) Low High Low High Speed(Kts) --------------------------------------------------- 1000 -588 601 -65 60 70 850 634 1853 -50 45 90 700 2101 3473 -50 30 120 500 4505 6121 -57 5 200 400 5870 7791 -66 -10 250 300 7726 9952 -72 -20 300 250 8835 11274 -76 -25 300 200 10260 12699 -78 -30 300 150 12094 14533 -85 -30 200 100 14000 17500 -95 -30 200 70 16496 19596 -95 -25 200 50 18402 21602 -95 -15 200 30 21003 25503 -95 -5 200 20 23501 28001 -95 5 200 10 27003 33003 -95 15 200 <10 N/A 45000 -95 15 200
The level 2 hydrostatic, super-adiabatic lapse rate, and wind shear checks ensure hydrostatic consistency between vertical layers, and also reasonable vertical consistency for the temperature and wind data.
The hydrostatic check is based on techniques developed by Schwartz and Govett (1992). In the MADIS implementation, the check is applied to the surface and mandatory level heights and temperatures, which may also be corrected by the check. Moving a layer at a time up from the surface, the height at the top level of the layer is estimated from reported pressure and temperature, then compared against the reported height. The height residual is then tested against a threshold that allows a margin of error based on the depth of the layer, which can span one missing mandatory level. If the residual is greater than the threshold, the next layer up is examined and that information is used in an attempt to correct either the height or the temperature of the level between the two layers. If a correction cannot be made, the estimate is recalculated using any available significant level temperatures in the suspect layer in the hope that this will better represent the layer's temperature and reduce the residual to below the threshold. The check continues until the top of the sounding is reached, or until a gap of more than one mandatory level occurs, or until a large residual is found and the criteria for correction are not met. In that event, both the height and temperature at the suspect level are failed by the QC, and the check is not applied to data higher up in the sounding. The following table lists the absolute value of the allowable residuals for each layer:
--------------------------------------------------- Hydrostatic Check Allowable Residuals (Absolute) Layer (mb) Value (m) --------------------------------------------------- 1099 - 1000 20 1000 - 925 20 925 - 850 21 850 - 700 20 700 - 500 25 500 - 400 20 400 - 300 20 300 - 250 20 250 - 200 25 200 - 150 30 150 - 100 36 100 - 70 40 70 - 50 45 50 - 30 50 30 - 20 55 20 - 10 60 10 - 7 65 7 - 5 70 5 - 3 75 3 - 2 80 2 - 1 85The level 2 super-adiabatic lapse rate check, based on Atkins (1985), fails for the temperature layer between levels n and n+1 if the following detection criteria is met:
C > [ ( (Pn / Pn+1 ) k * Tn+1) - Tn ]
where,
Note: level n is below level n+1, and thus Pn / Pn+1 is always > 1 )
Pn : pressure at level n (mb) Pn+1 : pressure at level n+1 (mb) Tn : temperature at level n (K) Tn+1 : temperature at level n+1 (K) k : kappa = 2/7 C : tolerance
= 2 K if Pn < 800 mb
= 2 K if Pn > 800 mb
The level 2 wind shear check (DiMego et al., 1985) is applied to mandatory level winds. For each mandatory level, the nearest wind that meets the following criteria is used for the check:
Next mandatory level up (m2)
Midpoint between Pm1 and Pm2
Mandatory level being checked (m1) < Search area
Midpoint between Pm0 and Pm1
Next mandatory level down (m0)
Also note that this wind will only be selected for use in the check if the mandatory level being checked is between 500 and 150 mb.
( FFmean < 30 kt and FFdiff < FFthr ) or( FFmean < 39 kt and FFdiff < FFthr and DDdiff < 70 deg ) or
( FFmean < 51 kt and FFdiff < FFthr and DDdiff < 55 deg ) or
( | FFm1 - FFs | < FFthr and DDdiff < 40 deg )
where,
If s is a mandatory level, there is an additional restriction that the magnitude of the vector difference between the two levels be less than 80 (kt).
m1 : mandatory level being checked s : level used to check m1 FFmean : mean wind speed (kt) between levels m1 and s FFdiff : absolute speed difference between the two levels FFthr : acceptable speed shear threshold (kt) between levels m1 and s
= 50 kt if s is a wind level
= 80 kt if s is a maximum wind or mandatory levelDDdiff : absolute directional difference between the two levels
*It should be noted that while the QC checks discussed here are generally applied to the form of the variable stored in the database, the QC results will also be applied to any forms of the variable that are requested by the user and are derived from the primary variable. For example, specific humidity will get the QC results from the checks applied to dewpoint temperature.
Two text files, a "reject" and an "accept" list provide the capability to subjectively override the results of the automated QC checks. The reject list is a list of stations and associated input observations that will be labeled as bad, regardless of the outcome of the QC checks; the accept list is the corresponding list of stations that will be labeled as good, regardless of the outcome of the QC checks. In both cases, observations associated with the stations in the lists can be individually flagged. For example, wind observations at a particular station may be added to the reject list, but not the temperature observations.
Here are the current subjective intervention lists in use at ESRL/GSD:
The MADIS QC information available for each variable includes the following QC structures: a single-character "data descriptor", intended to define an overall opinion of the quality of each observation by combining the information from the various QC checks, and for users desiring detailed information, a "QC applied" bitmap indicating which QC checks were applied to each observation, and a "QC results" bitmap indicating the results of the various QC checks.
The following table provides a complete list of the data descriptors and the bits used in the bitmaps:
--------------------------------- MADIS QC Information - Radiosonde --------------------------------- QC Data Descriptor Values ------------------------- No QC available: Z - Preliminary, no QC Automated QC checks: C - Coarse pass, passed level 1 S - Screened, passed levels 1 and 2 V - Verified, passed levels 1, 2, and 3 X - Rejected/erroneous, failed level 1 Q - Questioned, passed level 1, failed 2 or 3 where level 1 = validity level 2 = hydrostatic, super adiabatic lapse rate, and wind shear checks level 3 = N/A Subjective intervention: G - Subjective good B - Subjective bad Interpolated/Corrected observations: I - Interpolated W - Raw data was "wrong", has been corrected using estimates from automated QC checks T - Virtual temperature could not be calculated, air temperature passing all QC checks has been returned Bitmask for QC Applied and QC Results ------------------------------------- Bit QC Check Decimal Value --- -------- ------------- 1 Master Check 1 2 Validity Check 2 3 Reserved 4 4 Reserved 8 5 Reserved 16 6 Hydrostatic Check 32 7 Reserved 64 8 Super Adiabatic Lapse Rate Check 128 9 Wind Shear Check 256 10 Reserved 512The QC bitmask is used in the QC applied and QC result "words" returned along with the QC data descriptor. By examining the individual bits, the user can determine which checks were actually applied, and the pass/fail status of each check that was applied.
In the QC applied word, a bit value of 1 means the corresponding check was applied, a bit value of 0 indicates the check wasn't applied.
In the QC results word, a bit value of 1 means the corresponding check was applied and failed, a bit value of 0 indicates the check passed (given that the check was applied).
The "Master Check" is used to summarize all of the checks in a single bit. If any check at all was applied, this bit will be set in the QC applied word. If the observation failed any QC check, it will be set in the QC results word.
When read as decimal numbers, the different bits that are set in the bitmask are summed together. For example, a QC applied value of 131 should be interpreted as 1 + 2 + 128, meaning the validity and super-adiabatic lapse rate checks were applied.
Atkins, M.J., 1985: Quality Control, Selection and Data Processing of Observations in the Meteorological Office Operational Forecast System. Workshop on the Use and Quality Control of Meteorological Observations, Reading, U.K., European Centre for Medium Range Weather Forecasts, 255-290.
DiMego, G.J, P.A. Phoebus, and J.E. McDonnel, 1985: Data Processing and Quality Control for Optimum Interpolation Analysis at the National Meteorological Center. NMC Office Note 306, NOAA, U.S. Dept. of Commerce, 38 pp.
Schwartz, B. and M. Govett, 1992: A Hydrostatically Consistent North American Radiosonde Data Base at the Forecast Systems Laboratory. NOAA Technical Report ERL FSL-4, 81 pp.
Technique Specification Package 88-21-R2 For AWIPS-90 RFP Appendix G Requirements Numbers: Quality Control Incoming Data, 1994. AWIPS Document Number TSP-032-1992R2, NOAA, National Weather Service, Office of Systems Development.