U.S. DEPARTMENT OF ENERGY

OFFICE OF CIVILIAN RADIOACTIVE WASTE MANAGEMENT

OFFICE OF QUALITY ASSURANCE

 

AUDIT REPORT LBNL-ARP-98-12

OF

LAWRENCE BERKELEY NATIONAL LABORATORY

AT

BERKELEY, CALIFORNIA

MAY 18 THROUGH 22, 1998

 

 

 

 

 

Prepared by: ______________________________ Date: ____________________

Edward P. Opelski

Audit Team Leader

Office of Quality Assurance

 

 

 

Approved by: ______________________________ Date:____________________

Donald G. Horton

Director

Office of Quality Assurance

 

1.0 EXECUTIVE SUMMARY

This performance-based Quality Assurance (QA) audit was conducted at Lawrence Berkeley National Laboratory (LBNL), Berkeley, California, May 18-22, 1998, to evaluate Milestone Report SP33PLM4, "Testing and Modeling of Seepage into Drift, Input of Exploratory Study Facility Seepage Test Results to Unsaturated Zone Models"; and Milestone Report SP33PBM4, "Draft Report Fracture Flow and Seepage Testing in the ESF." The audit team determined that, with the exception of those areas where conditions adverse to quality were identified, LBNL has effectively implemented critical process steps for the preparation of the Milestone Reports. In addition, the technical evaluation of the two milestones determined that the scientific work on both was of good technical quality.

Three conditions adverse to quality were identified as a result of the audit. Technical procedures were documented in scientific notebooks instead of in Technical Implementing Procedures (TIP), scientific notebooks used for the studies were not reviewed during the technical review of the milestone report, and there was no verification of experience for scientists or technical reviewers associated with the studies. No new Office of Civilian Radioactive Waste Management (OCRWM) deficiency documents will be issued for these conditions. Corrective actions will be addressed in the responses to recently issued OCRWM Deficiency Reports (DR) LBNL-98-D-029, and LBNL-98-D-031. These deficiency documents identify other examples of conditions adverse to quality that are identical to those documented above. One additional condition adverse to quality, concerning the submittal of cited references to the Technical Information Center (TIC), required only remedial action and was corrected during the audit. Details of the conditions adverse to quality are presented in Section 5.5 of this report. The audit team also identified two recommendations during the audit. These recommendations are detailed in Section 6.0 of the report.

 

2.0 SCOPE

The audit was conducted to evaluate the technical adequacy of the following LBNL deliverables (Milestone Reports) and the effectiveness of critical process steps implemented during the preparation of these deliverables:

The audit team conducted interviews and reviews of documentation in accordance with the approved audit plan to evaluate the adequacy of deliverables and effectiveness of critical process steps.

2.1 Process Steps/Products/Documentation

The performance-based evaluation was based upon the following:

    1. Satisfactory completion of critical process steps.

2. Acceptable results and quality of the end product.

3. Documentation that substantiates the quality of products.

4. Performance of trained and qualified personnel.

5. Implementation of applicable QA program elements.

The following critical process steps were considered during the evaluation of Milestone Reports:

1. Scientific investigation planning

2. Identification, traceability and control of data

    1. Data analysis and review
    2. Status of data (Q – non Q)

5. Control of software

    1. Control of references
    2. Control of scientific notebooks
    3. Control of assumptions
    4. Control of measuring and test equipment
    5. Independent review of results
    6. Data input to Genesis
    1. Study planning
    2. Data identification and traceability
    3. Status of milestone

 

2.2 Technical Areas

The audit included a technical evaluation of the adequacy of Milestone Reports. Details of the technical evaluation are documented in Section 5.4 of this report.

3.0 AUDIT TEAM AND OBSERVERS

Name/Title/Organization

Edward P. Opelski Audit Team Leader, Office of Quality Assurance (OQA)

James Blaylock Auditor, OQA

Kenneth T. McFall Auditor, OQA

Keith M. Kersch Technical Specialist, CRWMS M&O

William Belke Observer, Nuclear Regulatory Commission (NRC)

T. H. Carter Observer, NRC

Jeffrey Ciocco Observer, NRC

4.0 AUDIT MEETINGS AND PERSONNEL CONTACTED

A pre-audit meeting was conducted at LBNL on May 18, 1998. Daily debriefings were held to apprise LBNL management and staff of the progress of the audit and of any identified conditions adverse to quality. A post-audit meeting was conducted at LBNL on May 22, 1998.

Personnel contacted during the audit, including those that attended pre- and post-audit meetings, are listed in Attachment 1.

5.0 SUMMARY OF RESULTS

5.1 Program Effectiveness

With the exception of the areas where conditions adverse to quality were identified, the audit team concluded that critical process steps applicable to the preparation of milestone reports were effectively implemented.

5.2 Stop Work or Immediate Corrective Actions Taken

There were no stop work actions or immediate corrective actions taken as a result of the audit.

 

5.3 QA Program Activities

A summary table of audit results is provided in Attachment 2. Details of the audit, including the objective evidence reviewed, are documented in the audit checklist. The checklist is maintained as a QA record.

5.4 Technical Audit Activities

Joe Wang is the Principal Investigator (PI) for the two milestones audited. During the interviews the PI and other LBNL staff, besides having a good grasp of the QA program requirements, were very cooperative and helpful.

The Milestone Reports audited support WBS 1.2.3.3.1.2.4, "Percolation in the Unsaturated Zone-Exploratory Studies Facility (ESF)." The objective of this element is to conduct in-situ ambient cross-hole pneumatic and liquid-release niche seepage studies and alcove surface infiltration studies to evaluate fluid flow within the unsaturated zone (UZ) at Yucca Mountain. The purpose of the niche drift scale seepage tests is to better understand the drift seepage process, percolation fluxes, and capillary barrier system.

Milestone SP33PLM4, "Testing and Modeling of Seepage into Drift, Input of Exploratory Study Facility Seepage Test Results to Unsaturated Zone Models," summarizes the drift seepage test results which have been successfully interpreted with the UZ drift scale models and site scale models to produce the flow fields for inputs to the Total System Performance Assessment for the Viability Assessment.

Although a completed milestone, the report appears to be more like an interim report on work being done on ESF seepage. The audit team determined the work to be technically sound with few flaws. The planning process for this work is not very well defined. The statement of work associated with this milestone consists of two sentences in the Planning and Control System (PACS) Participant Planning Sheet (PPS). The work evolved as the result of a series of negotiations between LBNL and CRWMS M&O management. The Master Planning Notebook, YMP-LBNL-JSW-6.0, contains a general discussion of the ESF seepage work that was planned, but did not reference this specific milestone. Only through interviews with the PI and others could the audit team identify the specific planning documents and their applicability, as well as the various steps in the planning process. Yvonne Tsang, Acting Laboratory Lead, indicated in the response, which was negotiated during this audit, to previously issued DR LBNL-98-D-029, that the planning process for future milestone deliverables will be detailed in a controlled document. Recommendation #1 addresses planning weaknesses.

The report for Milestone SP33PLM4 contained two references that could not be traced. One was a personal communication and the other was an unpublished technical paper. These deficiencies required only remedial action and were corrected during the audit. The personal communication was documented in a memorandum and along with the unpublished technical paper submitted to the TIC.

During the production of this milestone, some simple, but unqualified, modifications were made to software program ITOUGH2. Although this was noted in the milestone report, the audit team recommended the use of unqualified software, even though flagged, should cease at this stage of the project (see Recommendation #2).

Milestone SP33PBM4, "Draft Report on Fracture Flow and Seepage Testing in the ESF," is work in progress. There was not much documented work on this milestone, which is not due for several months. However, this milestone suffers from the same planning weaknesses as Milestone SP33PLM4.

Both milestones audited were of good technical quality. The preliminary testing and modeling results in drift seepage and alcove studies should lead to a better understanding of the seepage process and capillary barrier system important to the potential repository performance. These milestones are considered work in progress that provides the foundation for future seepage analysis and modeling.

5.5 Summary of Conditions Adverse to Quality

The audit team identified three conditions adverse to quality during the audit. Corrective action for these conditions will be addressed in the response to the recently issued OCRWM deficiency documents identified in Section 5.5.2 of the report.

      1. Corrective Action Requests

None.

5.5.2 Deficiency Reports

LBNL-98-D-029

This DR was issued to LBNL on March 3, 1998. The DR documents that technical procedures were documented in scientific notebooks instead of in TIPs and that scientific notebooks used for studies were not reviewed during technical reviews of milestone reports. During the audit, it was identified that three procedures that were used to control specific field activities associated with Milestone SP33PLM4 were documented in scientific notebook YMP-LBNL-JSW-6.0 instead of in TIPS. In addition, the audit identified that the scientific notebooks used in the production of this milestone were not reviewed during the technical reviews of the report.

LBNL-98-D-031

This DR was issued to LBNL on March 3, 1998. The DR documents that the applicable manager or the Human Resources Department had not verified experience for YMP LBNL personnel. During the audit, it was identified that there had been no verification of experience for scientists or technical reviewers associated with the Milestone Reports.

5.5.3 Performance Reports

None.

5.5.4 Conditions Adverse to Quality Corrected During the Audit

The report for Milestone SP33PLM4 contained two references that could not be traced. One was a citation of a personal communication and the other was cited as submitted unpublished technical paper. These deficiencies required only remedial action and were corrected during the audit. The personal communication was documented in a memorandum and along with the unpublished technical paper submitted to the TIC.

      1. Follow-up of Previously Issued Deficiency Documents

None.

 

6.0 RECOMMENDATIONS

The following recommendations resulted from the audit and are presented for CRWMS M&O and LBNL management consideration:

    1. LBNL should implement the response to DR LBNL-98-D-029 and develop a controlled document to capture and control planning activities.
    2. The use of unqualified software should cease at this stage of the project.

7.0 LIST OF ATTACHMENTS

Attachment 1: Personnel Contacted During the Audit

Attachment 2: Summary Table of Audit Results

 

 

ATTACHMENT 1

PERSONNEL CONTACTED DURING THE AUDIT

 

 

Name

 

Organization/Title

Pre-audit

Meeting

Contacted

During

Audit

Post-audit

Meeting

Aden-Gleason, Nancy

LBNL Engineering Assurance (EA) Manager

X

X

X

Ahers, Rick

LBNL Principal Research Associate

X

X

 

Benson, Sally

LBNL Earth Science Department Division Director

X

X

 

Birkholzer, Jens

LBNL Geological Scientist

X

X

 

Doughty, Christine

LBNL Staff Scientist

   

X

Fairley, Jerry

LBNL Hydrologist

   

X

Fissekidou, V. A.

LBNL EA Specialist

X

X

X

Finsterle, Stephan

LBNL Staff Scientist

 

X

X

Harris, Stephen

OQA Representative (LBNL)

X

X

 

Hastings, Cheryl

LBNL Program Administrator

X

X

X

Hayes, Larry

CRWMS M&O Operations Manager

X

X

 

Hinds, Jennifer

LBNL Hydrologist

   

X

Horton, Donald

Director, OQA

 

X

X

James, April

LBNL Senior Research Associate

 

X

X

Link, Suzanne

LBNL Technical Data Manager

X

McClung, Ivelina

LBNL Administrative Assistant

X

X

Mangold, Donald

LBNL Audit Lead

X

X

X

Oldenburg, Curt

LBNL Staff Scientist

X

Patterson, Russ

DOE/YMP Hydrology Technical Lead

X

X

Pelletier, John

OQA Representative (LLNL)

X

X

Trautz, Robert

LBNL Principal Research Associate

X

X

Tsang, Chin-Fu

LBNL ESD Department Head

X

X

X

Tsang, Yvonne

LBNL ESD Laboratory Lead

X

X

X

Wang, Joe

LBNL Staff Scientist PI

X

X

X

 

 

 

ATTACHMENT 2

SUMMARY TABLE OF AUDIT RESULTS

 

 

 

Product

 

Process Steps

Details (Checklist)

 

Deficiencies

Recommend-ations

Process Effectiveness

Product Adequacy

 

Overall

Milestone

SP33PLM4

Scientific Investigation Planning

 

pgs. 1-2A and 10

 

 

 

 

 

 

 

1

SAT

SAT

 

 

 

 

 

 

 

 

 

SAT

 

Identification of Data

pg. 3

 

SAT

SAT

 
 

Data Traceability

pg. 3

   

SAT

SAT

 
 

Data Review and Analysis

pgs. 4-4A

   

SAT

SAT

 
 

AQ - non Q@ Status of Data

pg. 4

 

2

SAT

SAT

 

 

Control of Software

Pgs. 7-8

   

SAT

SAT

 
 

Control of References

pg. 17

CDA

 

SAT

SAT

 
 

Control of Scientific Notebooks

pgs. 6-7

LBNL-98-D-029

 

UNSAT

SAT

SAT

 

Control of Assumptions

Pgs. 11-16

   

SAT

SAT

 
 

Control of Measuring and Test Equipment

Pgs. 9-9B

   

SAT

SAT

 
 

Independent Review of Results

pgs. 4-4A

LBNL-98-D-029 and LBNL-98-D-031

 

UNSAT

SAT

 

Data Input to Genesis

pg. 5

SAT

SAT

 

Legend:

SAT............Satisfactory

UNSAT......Unsatisfactory

 

 

 

 

 

 

 

 

 

 

 

 

ATTACHMENT 2

SUMMARY TABLE OF AUDIT RESULTS

(Cont’d)

 

 

 

Product

 

Process Steps

Details (Checklist)

 

Deficiencies

Recommend-ations

Process Effectiveness

Product Adequacy

 

Overall

 

Milestone SP33PBM4

Study Planning

 

Pg. 1

 

1

SAT

SAT

 
 

Identification of Data

Pgs. 2-3

   

SAT

SAT

SAT

 

Data Traceability

Pgs. 2-3

   

SAT

SAT

 

 

Legend:

SAT.............Satisfactory

UNSAT........Unsatisfactory