Workshop on Novel Approaches to Uncertainty Quantification

SUBSCRIBE GIT-UQ

listmanager@listman.lanl.gov

This one-day workshop provided a forum for the LANL research community and members of Sandia's Epistemic Uncertainty Project to share and discuss their technical approaches in this area.

Workshop Organizer:

• Cliff Joslyn, joslyn@lanl.gov, 505-667-9096

Additional Support, Questions:

• Alyson Wilson, agw@lanl.gov, 505-667-9167
• Francois Hemez, hemez@lanl.gov, 505-663-5204

We present a synoptic discussion of many of the components of Generalized Information Theory (GIT) in the context of their historical and mathematical development. We present fuzzy sets and logic as multivalued extensions of their Boolean counterparts; interval analysis and crisp possibility as a representation of strict nondeterminism; Choquet capacities as non-additive generalizations of probability measures; their further generalization to fuzzy measures; Belief and Plausibility measures as infinite-order Choquet capacities; Possibility measures and distributions as an extreme case of Plausibility; Dempster-Shafer evidence theory as an interpretation of these measures; random sets as an isomorphic representation of Dempster-Shafer in the context of set-valued probabilities; random intervals as their restriction to the line; and finally Probability Bounds as approximations of random intervals.

During the last two years the ASCI Verification and Validation Program Element at Sandia has begun two research projects focusing on epistemic uncertainty in the predicted response of engineering systems. Epistemic uncertainty is also referred to as subjective uncertainty, reducible uncertainty, or uncertainty due to lack of data. These two projects assume different starting points and use different approaches to modeling epistemic uncertainty. The first project, which is the topic of this presentation, takes an approach based on Dempster-Shafer theory, whereas the second project takes the approach of random fields. This presentation will briefly discuss why we believe non-traditional uncertainty estimation approaches are needed to improve the representation of the response of engineering systems with large epistemic uncertainty. An overview will be given of the various research thrusts, open issues, and activities of the project. Closing remarks will be made concerning how potential results of the project might be used in the ASCI program.

Dempster-Shafer theory offers an alternative to traditional probabilistic theory for the mathematical representation of uncertainty.  The significant innovation of this framework is that it allows for the allocation of a probability mass to sets or intervals.  Dempster-Shafer theory does not require an assumption regarding the probability of the individual constituents of the set or interval.  This is a potentially valuable tool for the evaluation of risk and reliability in engineering applications when it is not possible to obtain a precise measurement from experiments, or when knowledge is obtained from expert elicitation.  An important aspect of this theory is the combination of evidence obtained from multiple sources and the modeling of conflict between them.  This presentation surveys a number of possible combination rules for Dempster-Shafer structures and discusses the issue of conflict in rule selection.

Probability boxes are interval bounds on cumulative distribution functions that generalize both probability distributions and intervals.  These objects are closely related to Dempster-Shafer structures and random intervals of the real line.  Once we have probability boxes, they can be used as inputs in uncertainty propagation problems, but a fundamental question is where do these probability boxes come from? There is, of course, a vast literature on estimating probability distributions.  The approaches we are interested in generalize these methods.  There are five approaches to obtain probability boxes.  These are (1) direct assumption, (2) disaggregative modeling, (3) elicitation of expert opinion, (4) propagation of theoretical constraints, and (5) collection of empirical observations.  We describe a variety of methods for the fourth approach based on (4.i) limitations on risks (percentiles), (4.ii) bounds on density, (4.iii) limitations on range, (4.iv) limitations on variance, and (4.v) other shape information.  We also describe methods for the fifth approach that take account of empirical measurements involving interval uncertainty and various kinds of censoring.  The relationships with sampling theory are explored.

To many, the term uncertainty means an absence of knowledge. To others the term embodies multiple sources including variability and errors.  Regardless of the source, a fundamental question arises in how to characterize the various kinds of uncertainty and then combine them for a given problem in light of decision making. Examples of such complex problems include computer model verification and validation and reliability prediction applications with little or sparse data. Ever since the introduction of fuzzy set theory in 1965 by Zadeh, probability and statistics are sometimes considered inadequate for dealing with certain kinds of uncertainty (even if data are available), and probability models only a certain type of uncertainty. Since it is quite possible that different types of uncertainty can be present in the same problem, Zadeh (1995) has claimed that “probability must be used in concert with fuzzy logic to enhance its effectiveness.  In this perspective, probability theory and fuzzy logic are complementary rather than competitive.” This presentation explores how probability theory and fuzzy set theory can work together, so that uncertainty of outcomes of experiments and imprecision can be treated in a unified and coherent manner.  Both the theoretical and application of a linkage between the two theories will be presented.  The linkage involves the use of fuzzy membership functions, likelihoods, and the use of Bayes Theorem.  An example from reliability will illustrate the two theories working in concert, but within a probability framework.

In the Sandia Epistemic Uncertainty Project we are interested in improving risk and reliability analysis of complex systems where our knowledge of systems performance is provided by large simulation codes, and where moreover input parameters are known only imprecisely. Such imprecision lends itself to interval representations of parameter values, and thence to quantifying our uncertinay through Dempster-Shafer or Probability Bounds representations on the input space. In this context, the simulation code acts as a large "black box" function f, transforming one input Dempster-Shafer structure on the line (also known as a random interval A) into an output random interval f(A). Our quantification of output uncertainty is then based on this output random interval. If some properties of f (perhaps monotonicity or other analytical properties) are known, then some information about f(A) can be determined. But when f is a pure black box, we must resort to sampling approaches. In this talk, we first present the basic formalism of this Monte Carlo approach to sampling to approximate the belief and plausibility measures of the simulation output. We then discuss the algorithmics and computer simulations of the approach on some test problems.

Much of our work involves the construction of decision models. The systems for which these models are designed frequently have a common set of characteristics:

• The underlying phenomenology is complex and poorly understood
• The knowledge about the system is sparse, primarily quantitative and resides with a group of subject matter experts, no one person possesses all or most of it
• There is seldom any a priori agreement about what metrics should be used in the decision process nor how these metrics ought to be inferred

In this talk, we present several examples of our recent work to illustrate how LED works on “real” decision application problems. Depending upon the time available we will discuss the following problems:

• Prioritization of spare parts for nuclear weapon component manufacturing
• Conceptual design of a scientific/industrial complex
• A hybrid AR/Bayesian approach to estimating occurrence frequency
• Information loss path analysis for classified information

We describe a new arithmetic for computer simulations that uses a probability distribution variable (PDV) as the basic data type.  We have verified the effectiveness of the PDV arithmetic rules by extending Fortran to account for the creation and propagation of uncertainties in a computer program due to uncertainties in the data.  To compute reasonable and accurate bounds on these uncertainties, many of the functional dependencies developing between the variables during execution are monitored automatically and used by the PDV arithmetic algorithms.  The final uncertainty bounds are much smaller than they would have been had the dependencies been ignored.  These dependencies are also a measure of the sensitivity of the output of a program as a function of the uncertainty in the data.

The approach to be discussed is an extension of classical interval analysis, i.e. "Given bounds on the input parameters to a simulation code, what is the possible extent of the range of output function?" Since the computational models of interest are blackboxes, only those interval techniques that are not intrusive to the analysis code. The interval analysis problem is especially difficult in the case of blackboxes as the minimum and maximum of the output(s) often occur at interior points of the ranges of the input variables, e.g. resonance phenomena. The propagation of the uncertainty in the case of interval uncertain input variables is accomplished by the formulation and solution of two global optimization problems with bounded constraints. It can be shown that this formulation is also applicable if there is enough information to specify some input parameters as random variables and others as intervals or a combination of these. Along with bounds on the outputs, probability estimate that the output lies in this pertinent region, e.g. an interval or hyper-box, is calculated using Frechet bounds. It can be shown that these probability estimates are lower bounds, thus always conservative in nature. Several examples will be given.