Annotated Bibliography: Uncertainty Analysis and its Application to Hydraulic Measurements and Experimentation

Uncertainty Analysis - General Information

Available on the web at http://www.barringer1.com/drbob-bio_files/ASME83.pdf

This paper gives history, background, and discussion of committee activities leading to the development of the new (in 1985) ASME measurement uncertainty methodology, which is the basis for two ASME/ANSI standards (ASME PTC 19.1 Measurement Uncertainty among them).

The 1995 version of this standard has been proposed by Muste for adoption by the hydraulic engineering community.  This is the latest version.

·       September 1991 Editorial on Experimental Uncertainty.  {Wahl PDF}

·       Journal of Fluids Engineering Policy on Reporting Uncertainties in Experimental Measurements and Results.  {Wahl PDF}

·       Journal of Fluids Engineering Editorial Policy Statement on the Control of Numerical Accuracy.  {Wahl PDF}

Guidelines for presenting uncertainty analysis results for papers submitted to the Journal of Fluids Engineering.  JFE has had some requirement for uncertainty analysis in submitted papers since 1975.

Guidelines for presenting uncertainty analysis results for papers submitted to the Journal of Heat Transfer.

This paper is a brief overview of how the Boeing Aerodynamics Laboratory is applying the following standards:  1) Advisory Group for Aerospace Research and Development AGARD-AR-304 "Quality assessment for wind tunnel testing" and 2) American Institute of Aeronautics and Astronautics AIAA S-071-1995 "Assessment of wind tunnel data uncertainty."

This work presents uncertainty information for measurements (temperature, pressure, rotation speed, valve positions, etc.).  It has a good overview of uncertainty analysis and a good section on how to assess the uncertainty in linear regression.  SSME stands for Space Shuttle Main Engine.  Delivery Order (D.O.) 106 dealt with experimental uncertainty in venturi meter flow rates (this is summarized in Brown et al. (1998)).

The authors present a method for approximating the effect of correlated bias errors in experimental uncertainty analysis.  This is a very interesting paper.

The authors apply uncertainty propagation techniques to the linear regression analysis equations.  They demonstrate their methodology on the calibration of a venturi meter.  This is a good paper.

Very mathematical discussion of the use of least squares regression.

Technical paper discussing methods for estimating bias of reference parameters.  Has a section that talks about drift. 

The authors present the assumptions and approximations of ISO's Guide to the expression of uncertainty in measurement (1993).  They also compare this method with previously published uncertainty analysis approaches. 

The authors of this text have been two of the leading developers of modern uncertainty analysis techniques and have been involved in the preparation of many of today's most commonly referenced guides on the topic.  The book addresses basic concepts and statistics as well as applications to experimental planning, design, debugging, execution, data analysis, and reporting.  Appendices provide theoretical background and a comparison of alternative and previously used uncertainty analysis methods.  This is an excellent resource.

This paper outlines a formal approach for modeling uncertainty in multidisciplinary design problems.  A new method for propagating the uncertainty to find robust design solutions is developed and described.

Doiron, T., and Stoup, J., (1997).  Uncertainty and Dimensional Calibrations.  Journal of Research of the National Institute of Standards and Technology, Vol. 102, No. 6. 30 pp.  {Wahl PDF}

Describes implementation by NIST of measurement uncertainty standards for dimensional calibrations of several types of gages, including sieves.

These papers stress the need to clearly state the uncertainties of reported values.  It has a nice qualitative discussion of uncertainty and some tables with the "appropriate" format for reporting uncertainty estimates of various measurements and values.

Ellison, S.L.R., Rosslein, M., and Williams, A., eds.  (2000).  Quantifying uncertainty in analytical measurement.  EURACHEM/CITAC Guide CG 4.  Available on the web at http://www.measurementuncertainty.org/mu/quam2.pdf {Wahl and Wahlin PDF}

Despite the general-sounding title, this guide is specific to quantitative chemical analysis.  It is of interest primarily to see how another field has developed a guide for application of the ISO-GUM.  Contains lots of examples.

Extensive overview of all aspects of uncertainty analysis.  It includes lots of examples as well as appendices with general statistical information.  It follows ISO's Guide to the expression of uncertainty in measurement. 

This publication presents a methodology for expressing uncertainty for accredited calibration laboratories.  It is very similar to ISO's Guide to the expression of uncertainty in measurement.  There are many examples at the end of this publication. 

Contains some basic uncertainty analysis stuff.  Mostly concentrates on measurement techniques for various parameters (e.g., temperature, pressure, and flow).

This paper demonstrates that correlated precision errors are not always negligible and that they can have a significant influence on the results of an uncertainty analysis.

Qualitative assessment of uncertainty for greenhouse gas inventories.  Relies heavily on ISO standards.

The ISO Guide to the Expression of Uncertainty in Measurement, or GUM, 1995, is available in U.S. ($25) and international ($92) editions. The editions contain the same material, differing only in decimal marker, spelling, and size. The ISO International Vocabulary of Basic and General Terms in Metrology (VIM), 1993, a companion document to the GUM, is available only in an international edition ($71). The U.S. edition of the GUM is entitled: American National Standard for Expressing Uncertainty--U.S. Guide to the Expression of Uncertainty in Measurement, ANSI/NCSL Z540-2-1997

I do not have this publication.

This is one of the first papers on uncertainty analysis.  Apparently, this is the first paper to present the root-mean-sum equations for uncertainty.

This was the introductory paper in a special issue of JFE that presented several papers from a 1983 symposium on uncertainty analysis.  The paper touches such topics as the concepts of uncertainty in experiments, the uses of uncertainty analysis, and whether or not uncertainty analyses are worthwhile.  The second appendix to the paper presents several persuasive case histories that illustrate the potential value of uncertainty analysis.

This is an excellent summary of the outcomes of the 1983 symposium on uncertainty analysis.  Kline emphasizes two important conclusions:

1. Uncertainty analysis is an essential ingredient in planning, controlling, and reporting experiments.  The important thing is that a reasonable uncertainty analysis be done.  ALL DIFFERENCES OF OPINION ABOUT APPROPRIATE METHODS ARE SUBSIDIARY TO THIS CONCLUSION.  (my emphasis)
2. It is particularly important to use an uncertainty analysis in the planning and check out stages of an experiment!

A detailed review of statistical concepts that relate to uncertainty analysis.  The postscript on statistical graphics is a good source of ideas for how to examine experimental results graphically during the debugging phase of an experiment.

From the 1983 symposium on uncertainty analysis.  This short, thought-provoking paper addresses philosophical questions related to barriers to our understanding and use of uncertainty analysis concepts and procedures.

This work is a good overview of basic statistical concepts and control charts.  This work was developed for health-related fields of research and is a good presentation of familiar topics from a different perspective.

Some basic uncertainty analysis.  Concentrates mostly on reliability issues.

Moffat presents three levels of uncertainty analysis which he calls zeroth, first, and Nth order.  In zeroth order analysis the experiment is assumed rock-steady in time, so that the only errors are due to interpolation uncertainty (i.e., errors in reading experimental displays).  This is used mainly for preliminary planning.  A first order analysis includes process unsteadiness effects and interpolation effects, but instrument properties (i.e., their uncertainty characteristics) are assumed fixed in time.  This is useful during the developmental stage of an experiment.  The Nth order analysis includes uncertainties due to interpolation, unsteadiness, and instrument calibration variations during the experiment.  It is used for reporting the final results.  This paper was developed in connection with heat transfer and fluid mechanics research experiments.  The concepts introduced here have been incorporated into modern uncertainty analysis methodology.  The discussions of this paper by Dowdell and Abernethy are also very interesting (published in same volume).

Moffat outlines a procedure for using uncertainty analysis in the planning of experiments.  This paper is from the 1983 symposium on uncertainty analysis.

This paper presents a general description of the sources of errors in engineering measurements and the relationship between error and uncertainty.  The basic mathematics of both single-sample and multiple-sample analysis are also presented.  Geared for heat transfer and fluid mechanics.

A British publication that discusses the issues which can affect the quality of flow data and considers data analysis techniques which can be used to indicate flow meter performance.  This publication has some good examples.

More information on how the measurement standards industry is applying the ISO standards for expressing uncertainty.

Extends ISO's Guide to the Expression of Uncertainty in Measurement by accounting for uncorrected systematic errors.

Available on web at http://physics.nist.gov/Document/tn1297.pdf and http://physics.nist.gov/Pubs/guidelines/contents.html

An example of one industry's guidelines for applying the ISO standards.  Very comprehensive, with lots of definitions, appendices, and references.

Very good introduction to uncertainty analysis.  It is written in plain English and is very easy to understand.  The 1982 edition of this text was recommended by S.J. Kline in his closure to the 1983 ASME Symposium on Uncertainty Analysis.

Uses confidence and prediction intervals to determine the uncertainty in the calibration procedure.

Tung gives an overview of uncertainty analysis, including some techniques that I have never heard of such as Mellin Transformation and Rosenblueth's Probabilistic Point Estimation Method.  Most of the chapter concentrates on reliability analysis.

This manual has chapters on the uncertainty of discharge-probability functions, stage-discharge functions, and stage-damage functions.

This book is a collection of papers on reliability and uncertainty analysis.  It covers everything from new advances in uncertainty analysis to dam safety to coastal floods.  There is one article by Tung and Yen about new advances in uncertainty analysis.  I have never heard of some of these new methods.

Web Resources On Uncertainty Analysis

http://emtoolbox.nist.gov/Main/Main.asp - Engineering Metrology Toolbox: Selected Publications.  Provides links to several NIST publications related to uncertainty.

Applying Uncertainty Analysis to Hydraulic Engineering

This conference paper advocates the use of AIAA standard S-071-1995 as the standard that ASCE adopts for uncertainty analysis.  It also gives a brief overview of S-071-1995.  There is an updated version of this standard (S-071A-1999).

This paper outlines the objectives of a previous formulation of this task committee.

This report proposes using AIAA standard S-071-1995 for uncertainty analysis.  They also use a lot of stuff from Coleman and Steele.  The example they present is on the uncertainty of the measurement of density and kinematic viscosity.  There is an updated version of this standard (S-071A-1999).

Hydraulic Parameters - Miscellaneous

This paper uses first-order reliability methods to incorporate uncertainties involved in open-channel hydraulics and compares the results to the traditional method.

This paper presents a summary of uncertainty work that has been done in the past.  Parameters examined include:  Manning's n, channel slope, particle size, friction slope, sediment specific weight, and flow velocity.

The hydraulic uncertainty is evaluated through a first-order analysis of uncertainties of Manning's equation.  This study shows that the roughness coefficient and the friction slope dominate the variation in hydraulic uncertainty in that they account for 95% of the hydraulic uncertainty.

The author introduces a mathematical technique called the Mellin transform.  He gives two examples to demonstrate the application of the Mellin transform to uncertainty analysis of hydrologic and hydraulic problems.

This paper analyzes the uncertainties of a pit migration model using three methods including the first-order variance estimation method, the point estimation technique, and Latin hypercubic sampling.

Discharge Measurement - General Information

Available on the web at http://www.barringer1.com/drbob-bio.htm.

According to a handwritten note from Abernethy included in the PDF file, this standard was approved by unanimous Committee vote in 1987, and by world vote in 1988, (17 for, France and Italy against), but was never published because the French delegation controlled the ISO TC30 Secretariat.  Dr. Abernethy believes this earlier 10^th draft was better than the 12^th draft, which was the one approved by vote.  It was apparently later published, as a 1998 version is for sale from ISO.  Abernethy says this was the best standard he ever wrote.

I have also seen references in some papers to a 2001 version.  We do not have the official ISO publication.

We do not have this publication

We do not have this publication

Streamflow - Indirect Estimation Methods

The authors present equations and graphs to evaluate the accuracy of low flow measurements at ungaged sites.  The method is based on the regression equation and the length of record used.  There is an appendix at the end where E.J. Gilroy outlines the derivations of the equations.

Moss presents a model to estimate the serial-correlation structure of discretized streamflows on a monthly and annual basis.  The model accounts only for that component of the correlation that is caused by baseflow.  Direct runoff is treated as random noise.  Not particularly useful.

The authors outline a technique for assigning uncertainty to the estimation of the cumulative probability distribution functions (flow-duration curves) for river discharge at ungaged sites.  Not particularly useful.

Current-Meter Discharge Measurements

This article concentrates on the uncertainty introduced due to the assumed velocity profile (mean velocity in a vertical).

This chapter presents methods for estimating the uncertainty of current-meter discharge measurements, weirs and flumes, and stage-discharge relations.  This is very similar to the ISO stuff.

This is a short summary of the uncertainty stuff from Herschy's Hydrometry book.

ISO's take on estimating the uncertainty of a current-meter discharge measurement.  Similar to the approach taken by Pelletier (1998) and Sauer and Meyer (1992).

The author presents a method for estimating the uncertainty of a current-meter discharge measurement.  It is not nearly as useful and Sauer and Meyer's paper.

This publication presents a summary of over 140 publications on determining the uncertainty of current-meter discharge measurements.  It is set up similar to Sauer and Meyer's paper, but it is not as good.  Still, it's much better than Lintrup's paper.

The authors present a very good procedure for estimating the uncertainty of a current-meter discharge measurement.

This paper presents a methodology for determining the accuracy of measured flow rates and computed water volumes for surface water flows measured with current meters, as is typical in rivers and large canals.

This is an article from the UK.  It presents some qualitative, rather than quantitative, aspects of current-meter discharge measurements.  It concentrates on calibration.

Stage-Discharge Relationships

This is a comparison of stage-discharge relations developed from current-meter discharge measurements and from step-backwater analysis.  The agreement between the two methods is good (i.e., within ± 20%).

An error analysis shows that three types of errors influence the random error of a single discharge measurement determined from a rating curve.  They are rating curve error, water level measurement error, and an error caused by ignoring all physical parameters, other than water level, that affect discharge.  Methods in the literature for evaluating the first two types of errors are reviewed and a method for evaluating the third type is given.  The error of average discharge for an arbitrary period is also considered.

The authors analyzed the uncertainty associated with stage-discharge relations for more than 100 sites in the U.S.

The authors evaluated the uncertainty of 116 gage locations.  The uncertainty was broken into three parts:  1) natural, 2) measurement, and 3) modeling.  They also developed relationships to estimate the uncertainty of the stage-discharge relationship for ungaged streams and for the estimation of Manning's n values.

This is a very short paper that estimates the uncertainty of a stage-discharge relation.  It essentially is a procedure outlining how to determine the uncertainty of a curve fit.

This standard does not talk about uncertainty, but it's related to Part 2, which does.

This standard has a brief section on estimating the uncertainty of the stage-discharge relation as well as the uncertainty of the daily mean discharge, the monthly mean discharge, and the annual mean discharge.  This methodology is very similar to the one outlined by Herschy.

I'm waiting for Schmidt to send me his entire dissertation.  Right now, I only have chapter 3, which has a section summarizing previous efforts to describe the uncertainty in stage-discharge relationships.  His description is very thorough.  I also have his bibliography which pointed out many other references we should try to obtain.

The uncertainty was broken into three parts:  1) natural, 2) measurement, and 3) modeling.  They also developed relationships to estimate the uncertainty of the stage-discharge relationship for ungaged streams and for the estimation of Manning's n values.

The author estimates the uncertainty of the parameters of a stage-discharge relationship using regression analysis.

Weirs and Flumes

A large set of experiments were conducted on small Parshall flumes (2-ft throat width or less) to develop a procedure for adjusting measurements to correct for effects of submergence and differential settlement which produces lateral and/or longitudinal slope of the flume.  Several years ago Dr. Abt provided me with a computer program to perform the calculations.  I have noted some inconsistencies in the data tables of this paper, and was told by Dr. Abt that a 1998 version of the paper corrects the problems, but I do not have a copy of it and do not know the exact reference.

Field installations of Parshall flumes were evaluated, and serious problems were found with more than half of the installations.

Authors give some basic uncertainty concepts related to flow measurement flumes.

This laboratory study of 6- and 9-inch Parshall flumes investigated approach flow problems with installations in small mountain streams.  The study developed upstream pool modifications to improve approach flow conditions and measurement accuracy.  The focus of the study is on eliminating bias errors and the flow conditions that contribute to excessive random uncertainty.

Jones assumes that the uncertainty of critical depth flow meters (e.g., weirs and flumes) is well defined, and he uses them as a bench mark to compare the performance of velocity-area flow meters.  This work only applies to rectangular channels. 

This laboratory study of 1-ft Parshall flumes revealed a discontinuity in the discharge-head relationship originally developed by Parshall.  Parshall's original data was too sparse to reveal the problem.  For a range of submergences, two different discharges are possible.  Errors as large as 12 percent can occur between actual discharge and that calculated from Parshall's rating equation.  Improved submergence corrections were developed, and a submergence limit of 86 percent was recommended to avoid the zone of discontinuity.

This is a very old work that attempts to identify sources of error in Parshall flumes and weirs.  It points out many possible sources of error (e.g., sloping weir crest, ignoring the velocity of approach, rounding of the crest of the weir, etc.) and gives a quantitative approximation of the error for each identified source.

Annual Discharge

Qualitative outline of procedure used to estimate the standard error of annual discharges at various sites along the Lower Colorado River. 

Updated version of Moss and Gilroy's method.  Applied to selected sites on the Lower Colorado River.

The authors computed the uncertainty of a computed discharge at a site by randomly choosing a group of discharge measurements for use in a rating analysis and using the remaining elements as a control group.  The variance of the computed discharge was obtained by subtracting the variance of between the measured and true discharge from the variance between the measured and computed discharge.  I don't like this approach very much.

The author assumes the stage-discharge relation can be described as Q = g(h + a)^b.  He takes into account the uncertainty in the parameter a.  The annual maximum discharges are all estimated from the same stage-discharge relations, so they are correlated.  This correlation is not usually accounted for.  Clarke presents a method to estimate the uncertainty of the mean annual flood flow that takes into account this correlation as well as the uncertainties in the stage-discharge relationship.

The authors assume that the sequence of mean annual discharges is a stationary time series with uncertainty expressed by the standard deviation of annual discharge.  They use the uncertainty of the curve-fit stage-discharge relationship and the correlation between estimated mean annual discharges to obtain the uncertainty of the mean annual discharge.

Moss and Gilroy's uncertainty analysis applied to rivers in Maine.

A more detailed account of Fontaine's earlier publication.

Application of Moss and Gilroy's uncertainty analysis to the Lower Colorado River Basin.

Moss and Gilroy's uncertainty analysis applied to rivers in Hawaii.

Meyer uses flood-discharge data, current-meter discharge measurements, indirect measurements, and stage-discharge relations to estimate the uncertainty of annual peak floods.

This is a very interesting paper.  Moss and Gilroy assume that the uncertainty in the annual mean discharge at a stream gage is a function of the number of current-meter discharge measurements that are performed at that site.  Using Kalman filtering techniques, the authors estimate the uncertainty in the annual mean discharge as a function of the number of current-meter discharge measurements that were performed.  The mathematics in this paper are quite complex and some of the concepts are not explained very well.  A couple of years ago, I talked to Moss and even he couldn't clear up some of my questions.  As he was looking over his paper, he commented on how poorly he had written it.  Moss and Gilroy wrote a computer program to perform the calculations outlined in this Open-File Report.  Moss gave me a printed copy of the code (he wasn't sure if what he gave me was the complete code).  This methodology has not been used very widely, but it is still very interesting. 

Description of the computer program used in Moss and Gilroy's uncertainty analysis.

My attempt at estimating the uncertainty of the annual volume.  Simpler than Moss and Gilroy's method but not as accurate.  Considers both random and systematic uncertainties (Moss and Gilroy only consider random uncertainties).  Heavily dependent on the user's ability to identify and quantify uncertainties.  I am working on a new version of this paper that is easier to understand and follow.

Lower Colorado River Accounting System

Statistical analysis of the LCRAS procedure.  Uses standard statistical analysis techniques to estimate the uncertainty of various components of a mass-balance equation for the Colorado River.  It has a good overview of uncertainty with a strong emphasis on systematic errors.  Some of his uncertainty analysis is presented in a slightly different manner than what I have normally seen.  It also discusses digitization and linearization errors, which aren't typically mentioned.

An example of the uncertainty analysis described in Lane's 1998 report.

Closed-Conduit Flow Meters

This book has a detailed section of how to properly install an orifice meter to minimize the error.  It covers things like how to tap the pipe, how much straight approach length is needed, etc.

This paper examines the effect of flow disturbances on the performance of several varieties of closed-conduit flow meters, including propeller meters, pitot-tube devices, paddle wheel meters, and a Doppler acoustic meter.  The effect of straightening vanes was also investigated.

http://www.emersonprocess.com/daniel/Products/Gas/Orifice/Senior/AppNotes/Theoretical Uncertainty of Orifice Flow Measurement 172KB.pdf

Discusses the factors that can cause measurement errors and influence uncertainty.  This report appears to be from a commercial flow meter company or maybe a consulting firm specializing in orifice meter applications..

There is a small section on uncertainty in this standard.

This paper presents some equations for the discharge coefficients of Venturi meters operating with water and high-pressure gas.  It also gives estimates of the uncertainty of the discharge coefficients.  There is a section on the error introduced from the pressure tap.

This paper presents results of laboratory testing to evaluate the effect of distorted velocity distributions on the accuracy of closed-conduit and open channel flow meters and flumes.  The majority of the paper talks about closed-conduit devices.  The influence of straightening vanes and orifice-type flow conditioners is evaluated.  Research on this topic is ongoing.

Ultrasonic Flow Meters

This paper investigates accuracy of USFM's at various distances downstream from flow disturbances commonly encountered in irrigation systems.  Error correction multipliers were developed to overcome bias errors when flow meters were installed close to flow disturbing devices.

Acoustic Doppler Current Profilers

The authors identify several sources of error in turbulence measurements made with ADCPs including:  inaccuracy of Doppler-shift measurements, poor temporal and spatial measurement resolution, and inaccuracy of multi-dimensional velocities resolved from one-dimensional velocities measured at separate locations.

This Open-File Report has a section which describes the possible sources of error in an acoustic Doppler discharge measurement.

Laser Doppler Velocimetry

This paper presents a detailed uncertainty analysis of the beam alignment and vortex measurement technique used in LDV measurements.  Sources of uncertainty are identified with optics calibration, data acquisition, and data reduction.

Environmental Measurements

This is a qualitative report designed to introduce people to the concepts of risk and uncertainty.  It has a pretty good description of the difference between risk and uncertainty.

A very detailed annotated bibliography on risk and uncertainty analysis.  I need to take a closer look at this.

Sediment Measurements

The uncertainties implicit in suspended sediment transport curves are examined and the practice of using sediment load versus discharge is shown to be misleading, since the goodness of fit implied by this relation is spurious.

In estimating reservoir sedimentation, a number of uncertainties arise.  These are related to annual streamflow, sediment load, sediment particle size, trap efficiency, and reservoir operation.  Monte Carlo simulation is used to quantify the uncertainty of reservoir sedimentation.

Pressure Measurements

This is a new report on the error introduced in pressure measurements from the pressure tap.  This study indicates that the correction term for the pressure tap continues to increase as the hole Reynolds number increases, which is contrary to previous studies.

This paper determines the error in a pressure measurement due to the size and shape of the pressure tap.

Numerical Computations and Modeling

This paper reviews the formulation of methods for quantifying numerical uncertainty in simulations.  The authors present a five-step approach for the estimation of numerical uncertainty based on Richardson Extrapolation and the Grid Convergence Index (whatever those are).

This paper develops a general methodology for estimating the total uncertainty in computational simulations that deal with the numerical solution of a system of partial differential equations.  This methodology has six phases:  1) conceptual modeling of the physical system, 2) mathematical modeling of the conceptual model, 3) discretization of the model, 4) computer programming of the model, 5) numerical solution of the computer model, and 6) representation of the numerical solution.

This is a short article that has two simple examples that illustrate the importance of using enough computational cross-sections in hydraulic simulations.  In some instances, using too few computational cross-sections led to errors as large at 2 feet.

This document describes the results of an investigation of the effects of using survey and mapping technology for determining cross-sectional coordinate geometry and the reliability of Manning's roughness coefficient on the accuracy of computed water surface profiles.

This document presents information that can be used to select the appropriate method of data collection for the development of water surface profiles.

Wow!  Now that's a title.  In a previous work, the effects of discharging treated wastewater into two different rivers were simulated.  In this work, Wesolowski performs a first order uncertainty analysis of the simulated constituent concentrations and property values using the Enhanced Stream Water Quality Model-Uncertainty Analysis, whatever that is.

Last reviewed: 10/26/04