Arthur Blanchard
Westinghouse Savannah River Company
Aiken, SC 29808

Kevin R. O’Kula, Jackie M. East, and Danielle R. Marx
Westinghouse Safety Management Solutions LLC
Aiken, SC 29804-5388

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

This report has been reproduced directly from the best available copy.

Available for sale to the public, in paper, from:  U.S. Department of Commerce, National Technical Information Service, 5285 Port Royal Road, Springfield, VA 22161,  phone: (800) 553-6847,  fax: (703) 605-6900,  email:  orders@ntis.fedworld.gov   online ordering:  http://www.ntis.gov/support/ordering.htm

Available electronically at  http://www.osti.gov/bridge/

Available for a processing fee to U.S. Department of Energy and its contractors, in paper, from: U.S. Department of Energy, Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831-0062,  phone: (865 ) 576-8401,  fax: (865) 576-5728,  email:  reports@adonis.osti.gov

An American National Standards Institute (ANSI) a posteriori (backfit) process, available to provide software quality assurance (SQA) for assurance that ssoftware developed outside of required qualification protocol, has been applied to a the special-purpose, versatile tritium dispersion and consequence model, UFOTRI,. A a computer model developed at the German Karlsruhe laboratory, UFOTRI, was chosen to benchmark the limited verification and validation process. This because of UFOTRI’s was chosen because of its strengths in initial tritium-related consequence analyses, and its potential for application in a Department of Energy accident analysis context. The six-task process met key ANSI requirements and was performed during a several-month level of effort. Included project deliverables were Assessment, Test Plan, Configuration Procedure, Error Notification Procedure, Comprehensive Technical Report, and SQA Qualification Report documentation. Comparison to acute release conditions is still in progress, but results to date indicate satisfactory, bounding predictions can be achieved with UFOTRI relative to measurements. Results of this compact effort appear to identify UFOTRI as a suitable candidate for a software "toolkit," i.e., minimum verification and validation (V & V) requirements are satisfied, and a configuration controlled version is deemed appropriate for use in a DOE accident analysis context.

The accident analysis sections of Safety Analysis Reports (SARs) and Basis for Interim Operation (BIO) reports for Department of Energy (DOE) nuclear facilities usually contain the methods and quantification of radiological exposures due to postulated accident conditions. In principle, SAR-related doses are calculated applying a qualified spreadsheet or other software package using conservative inputs and user assumptions. In practice, many of the computer-based options available to DOE accident analysts are not supported with adequate software quality assurance (SQA), and little validation evidence that the model is fit for the specific applications exists.

The analyses supporting SARs and BIOs for DOE tritium storage and processing facility operation face the same SQA issues. Unless sufficient justification can be provided demonstrating the pedigree of the software, including quality assurance and fitness for application to accident conditions, it is difficult for independent review of the analyses to draw any conclusion on the associated consequences calculated with the model. This may be especially problematic with software developed without a formal Software Development Plan (SDP) that is tailored to meets certain customer’s requirements, and then made available for specific users. However, an American National Standards Institute (ANSI) standard exists that details a posteriori (backfit) process, applicable to many accident analysis software models used for DOE nuclear facilities. The standard, ANSI/ANS-10.4-1987, provides assurance that software developed outside a defined SDP qualification protocol, is applied within the proper context and meets minimum Verification and Validation (V & V) standards [(ANSI/ANS, 1987)].

The ANSI backfit process has been applied to UFOTRI, a special-purpose, tritium dispersion and consequence model. The process demonstrating UFOTRI as "fit" for consequence analysis for Savannah River Site (SRS) and other DOE facilities is described in this paper. It may be of particular interest since the ANSI standard is applied to a computer model that is foreign-developed.

The Defense Nuclear Facilities Safety Board staff recently reviewed the application of safety analysis software and its associated quality assurance in the DOE Complex [(DNFSB 2000]). Technical Report 25 recommended improvements in the SQA practices, processes, and procedures used for accident analysis and process control of DOE nuclear facilities. Of particular significance in the report was the class of computer models for dispersion and consequence analysis, and the suggestion to establish a "toolkit" of verified and validated models with identified regimes of applicability. In principle, the accident analysis toolkit would be a repository of verified and validated computer models that are configuration-controlled and made available for specific applications. Because of UFOTRIs strengths in tritium-related consequence analysis, the SQA programThe UFOTRI model for UFOTRI Version 93/4.20P for Personal Computers was chosen to be upgraded with the ANSI standard process because of its strengths in tritium-related consequence analysis. Equivalent goals were to benchmark the resource commitment to conduct the a posteriori V & V (or Design) Review and to ultimately establish UFOTRI as a primary software model for the toolkit.

The UFOTRI (Unfallfolgenmodell fur Tritiumfreisetzungen) computer model was developed by the German national laboratory, Karlsruhe (Kernforschungszentrum Karlsruhe Gmbh, or KfK) to assess the radiological consequences due to postulated accidental atmospheric releases from nuclear facilities ([Raskob 1990, 1992, 1993]). The first version of UFOTRI was released in mid-1991 [Raskob, 1990]. Version 4.0An update was released in late 1993 incorporating several significant improvements to plant/soil exchange, soil/atmosphere exchange, plant, and photosynthesis (OBT formation) models [Raskob, 1992 and 1993].

UFOTRI describes the behavior of tritium in the biosphere and calculates the radiological impact on individual receptors and populations due to inhalation, skin absorption and uptake of contaminated foodstuffs. Although individual and population doses can be calculated, UFOTRI does not calculate health effects. Time-dependent processes modeled include dispersion, deposition, reemission, conversion of tritium gas (HT) into tritiated water vapor (HTO) by the soil, and conversion of HTO into organically bound tritium (OBT) (Figure 1). The source term model accounts for release duration, release height, tritium species being released, and thermal energy released. A Gaussian module is applied for the initial release of HT/HTO and reemission up to seven days after the release event. The reemission model addresses evaporation from soil and transpiration from vegetation. UFOTRI considers all relevant transfer processes in the environment (atmosphere, soil, plant, and animal), and is unique in that the initial plume passage model is integrated with the reemission (area source) model.

UFOTRI features an acute atmospheric model of transfer processes from the postulated event (~ first one hundred hours), coupled to a first-order compartment module for the longer-term behavior of two different chemical forms of tritium species in the food chain. The long-term model accounts for food ingestion doses from contaminated foodstuffs, but does not include ingestion doses from potable water consumption. Typically, for individual dose calculations for SARs, the food ingestion doses are not calculated. The Radiological Dispersion/Consequence Working Group of the Accident Progression and Consequence (APAC) Evaluation Project recommended UFOTRI over other software packages for deterministic safety analyses (SARs and BIOs), Environmental Impacts Statements, and probabilistic safety assessments of tritium nuclear facilities ([O’Kula et al., 1996)].

The code can execute with meteorological data treated either in a deterministic manner, using prescribed meteorological conditions, or it can run in a probabilistic manner, using a stratified random sampling algorithm. When meteorological data is treated in a probabilistic manner, Ultimately, a set of "start times" is selected from consequence bins (sampled from meteorological data representative of the site). The consequences for the respective source terms are weighed by the probability of the consequence category, such that the output is in the form of a complementary cumulative distribution function (CCDF) table, providing median, 95^th percentile, and other consequence levels.

UFOTRI has been applied primarily for fusion safety and design projects. The major applications have been for fusion studies such as the Next European Tokamak (NET) and the International Thermonuclear Experimental Reactor (ITER). Experimental studies in Canada indicate good agreement for use of UFOTRI for tritiated water vapor release [BIOMOVS, 1996]. Testing with HT species in Canada has shown good agreement with field measurements, despite data collection and modeling uncertainties [Raskob, 1990].

The model used for most consequence analyses of postulated accident conditions for SRS safety documentation is MACCS version 1.5.11.1 [Chanin et al., 1990, 1993; Jow et al., 1990; Rollstin et al., 1990]. Despite completion of the UFOTRI V & V process, MACCS will continue to be applied for most SRS tritium consequence analyses. UFOTRI will be applied in a supplemental and confirmatory context. Table 1 compares the two models, UFOTRI and MACCS.

In order for UFOTRI to be used with confidence for the calculation of doses associated with authorization basis (AB) documentation, the code must have a graded V & V effort that defines the qualification of the software. The ANSI and IEEE software quality assurance standards [ANSI/ANS, 1987 and IEEE, 1984] both contain sections that deal with software that is developed outside the respective standards.

A "backfit" process may make sense for a computer model that is the end product of a research program [ANSI/ANS, 1987]. In this context, a parallel V & V program may be impracticable until after the development process is completed. The ANSI standard details a backfit or a posteriori process that recovers some measure of V & V previously missing by taking advantage of available computer model products and collective user experience. Its purpose is to determine whether the computer model produces valid responses within a specific domain, and to document that an appropriate level of V & V has been conducted.

Six tasks defining the a posteriori V & V program were performed for UFOTRI. The tasks meet the ANSI requirements and define de minimis steps completed before the UFOTRI code was deemed suitable for Authorization Basis applications. Included are:

1. An Assessment Document containing listing of, or reference to, technical data pertaining to existing software requirements and documentation, code source listing, test data, validation experiments, and technical literature search.
2. A Test Plan defining tests performed and expected results, evaluation criteria, and technical bases for test data.
3. A Configuration Procedure defining code installation, a maintenance procedure for software/operating system modifications, test set contents, and a routine testing procedure.
4. An Error Notification Procedure that defines user qualifications and approvals, maintenance of user and software error lists, and indicates the process for user notification.
5. A Comprehensive Technical Report containing results of testing program, identifying areas of code applicability and potential limitations.
6. SQA Qualification Report containing completed SQA checklist, supplemental material and explanation, including software management and inventory program interfaces.

A modest effort was undertaken to complete these tasks. A flowchart of phases is discussed in the next section.

A compact process was executed to ensure that key V & V tasks were completed to bring the UFOTRI software into compliance with major requirements of the ANSI and IEEE standards. Figure 2 represents a process flowsheet for configuration control and to establish requisite attributes of an acceptable that was followed for the UFOTRI software. Phases and activities include:

Independent review of Phases 2 through 6 was periodically performed. It was useful check to ascertain if any aspect of work needed to be revised or redone. Based on our experience with the UFOTRI program, by far the most resource-intensive phase was the third one (approximately 70% of the overall effort).

The effort to perform the six tasks was conducted over two periods. The overall level of effort, including technical report generation, technical and management, and quality assurance review was approximately 3 person-months.

As an independent check of the SQA performed under the ANSI posteriori process, the V & V qualification process sheet was assessed for compliance against the relevant IEEE standard [IEEE, 1984]. The UFOTRI project was performed at a level where many requirements were consolidated, but in a manner that still complied with most of the applicable IEEE Standard requirements.

Table 2 is a list and description of the necessary documents needed for a complete SQA package as taken from these standards. The phase in which the IEEE standard was met is listed ahead ofnext to or before each document or activity. While it is designed to be implemented in a software development activity, a project assessment demonstrated that the UFOTRI qualification process met the intent of these items.

Several prediction-to-measurement comparisons of UFOTRI to field data are contained in the User’s Manual. A comparison to a predominantly tritium gas release was performed based on an event occurring at the Savannah River Site (SRS) on May 2, 1974.

The release was caused by a valve failure and resulted in a stack release of approximately 49 g from the central area of SRS [Murphy and Wortham, 19987]. The release occurred over a 4-minute period beginning at 0755 hours Eastern Standard Time (EST). The tritium form was estimated to be primarily tritiated hydrogen gas with less than 1% tritiated water. At the time of the release, light winds carried the tritium in a northeasterly direction at 6.4 to 9.7 km/h. Cloud cover recorded nearby in Augusta, GA was 90% - 100%. The atmospheric stability was predominantly neutral. The trajectory of the release carried the tritium north of Columbia, SC, beyond which point it was difficult to predict because of complex weather patterns.

Table 3 compares several environmental measurements of tritium concentrations with quantities calculated by the UFOTRI model. In this case, meteorology at the time of the release is used along with a 80-km grid specific to SRS and surrounding areas. The computer estimates are considerably above ground measurements for most quantities compared, including air, vegetable tritiated water (HTO), surface water, and milk concentrations. From the standpoint of safety analysis, this is preferred to under-predicting measurements. This trend is consistent with measured-to-observed data reported from an intended field release of HT conducted in Canada [Raskob, 1990].

The major factors leading to the over-prediction of tritium concentrations by the UFOTRI model are primarily

• One-hour averaging of meteorology applied to a short-duration release (<five minutes) - UFOTRI interprets changes in weather, including windspeed and wind direction, on an hourly frequency basis. The HT release modeled here occurred over less than five minutes, thus Briggs dispersion puff parameterization can be subject to appreciable error in defining peak maxima when one-hour meteorology periods are averaged.
• Sampling location error – The release was accidental as opposed to a planned experiment. Thus, positioning of instrumentation was not planned beforehand. In effect, the plume was being "chased" and assayed, based on the weather forecasting performed at SRS. It is highly improbable that computer model peaks coincided with recorded data from environmental sampling.

• Spatial uniformity assumed by UFOTRI – While UFOTRI provides concentrations at using 5-degree arc resolution, all aspects of the compartment model considering air, soil, vegetation, grass, and consumption by humans and animals, assumes uptake at the peak sectors. In reality, grass, pine tree, crop, and dairy locations are spatially dependent. It is very unlikely that particular sampling locations were in the path of the plume centerline.

Version 93/4.20P of the UFOTRI code has been verified and validated applying ANSI standard 10.4-1987 in an a posteriori (backfit) manner. The effort was maintained at a low resource commitment level (~ three person-months). The V & V process provides sufficient assurance that UFOTRI can be one of a group of software models that is available for tritium accident analysis for DOE facilities, as part of a toolkit of configuration-controlled models for specific applications.

The major observations and lessons learned through the V & V process for UFOTRI are as follows:

• Overall duration of the six-phase process, including: (1) UFOTRI Status Definition; (2) Remedial Action Plan; (3) Completion of all Remedial Actions; (4) Establishing New Baseline; (5) Development and Review of Formal Test Plan; and (6) Complete Testing and Evaluation phases required 3 person-months.
• These tasks met applicable ANSI software quality assurance documentation requirements, and resulted in an Assessment Document, a Test Plan, a Configuration Procedure, an Error Notification Procedure, a Comprehensive Technical Report, and a SQA Qualification Report.
• Although not as rigorous as a V and V process integrated into software development, the a posteriori process still could be shown to comply favorably with the most critical items from the IEEE software standard.
• Comparison to field data from actual accidental release events show that the UFOTRI model can be exercised in a bounding, conservative manner appropriate for accident analysis applications.

The UFOTRI V & V process discussed in this paper required extensive input and recommendations from many sources. While the process team is appreciative of all contributions and suggestions, it is especially indebted to Dr. Wolfgang Raskob of KfK (UFOTRI developer and maintainer) and Mr. Eric Hope of Westinghouse Safety Management Solutions (Software Quality Assurance issues).