CONFUSION MATRIX
Function	To evaluate and quantify operator errors.
Procedure	Probabilities can be derived experimentally (1) or using expert judgments. For example, experts judge the likelihood that operators misdiagnose specific critical events. Judgments can be solicited in a structured and systematic fashion at different times during event sequences. (1) Example of usage of the confusion matrix (see VOLPE National Transportation Systems Center (1998). Evaluation of Retroreflective Markings to Increase Rail Car Conspicuity. Cambridge, MA 02142-1093)   Stimulus Response Yellow Red R|W Yellow 2 4 24 Red 0 11 9 R|W 33 20 11 Total 35 35 43
Advantages	Easy way to identify critical situations.
Disadvantages	If confusion matrix is constructed based on experimental manipulation (1): External validity of method: are errors-rates that were observed in experimental setting transferrable to naturalistic environments? If confusion matrix is constructed based on expert judgments (2): Derived probabilities are disputable. Uncertainty is associated with the application of this technique itself.
References	VOLPE National Transportation Systems Center (1998). Evaluation of Retroreflective Markings to Increase Rail Car Conspicuity. Cambridge, MA 02142-1093. Potash, L. M., Stewart, M., Dietz, P. E., Lewis, D. M. and Dougherty, E. M. (1981), “Experience in Integrating the Operator Contributions in the PRA of Actual Operating Plants” in Proceedings of the ANS/ENS Topical Meeting on Probabilistic Risk Assessment, Port Chester, N. Y., American Nuclear Society: La Grange Park, Ill.
Links	General Information about Confusion Matrices Application of Confusion Matrices

Tools

Risk Assessment

Confusion Matrix
                 Product Type:  Tool             Product Class:  FAA Template
                               Year:  2004                       Author:  Moertl, Peter
          Lifecycle Phase(s):  Mission Analysis

            Lifecycle Task(s):  Establish

     Lifecycle Keyword(s):