U.S. DEPARTMENT OF ENERGY
OFFICE OF CIVILIAN RADIOACTIVE WASTE MANAGEMENT
OFFICE OF QUALITY ASSURANCE
AUDIT REPORT LBNL-ARP-98-12
OF
LAWRENCE BERKELEY NATIONAL LABORATORY
AT
BERKELEY, CALIFORNIA
MAY 18 THROUGH 22, 1998
Prepared by: ______________________________ Date: ____________________
Edward P. Opelski
Audit Team Leader
Office of Quality Assurance
Approved by: ______________________________ Date:____________________
Donald G. Horton
Director
Office of Quality Assurance
1.0 EXECUTIVE SUMMARY
This performance-based Quality Assurance (QA) audit was conducted at Lawrence Berkeley National Laboratory (LBNL), Berkeley, California, May 18-22, 1998, to evaluate Milestone Report SP33PLM4, "Testing and Modeling of Seepage into Drift, Input of Exploratory Study Facility Seepage Test Results to Unsaturated Zone Models"; and Milestone Report SP33PBM4, "Draft Report Fracture Flow and Seepage Testing in the ESF." The audit team determined that, with the exception of those areas where conditions adverse to quality were identified, LBNL has effectively implemented critical process steps for the preparation of the Milestone Reports. In addition, the technical evaluation of the two milestones determined that the scientific work on both was of good technical quality.
Three conditions adverse to quality were identified as a result of the audit. Technical procedures were documented in scientific notebooks instead of in Technical Implementing Procedures (TIP), scientific notebooks used for the studies were not reviewed during the technical review of the milestone report, and there was no verification of experience for scientists or technical reviewers associated with the studies. No new Office of Civilian Radioactive Waste Management (OCRWM) deficiency documents will be issued for these conditions. Corrective actions will be addressed in the responses to recently issued OCRWM Deficiency Reports (DR) LBNL-98-D-029, and LBNL-98-D-031. These deficiency documents identify other examples of conditions adverse to quality that are identical to those documented above. One additional condition adverse to quality, concerning the submittal of cited references to the Technical Information Center (TIC), required only remedial action and was corrected during the audit. Details of the conditions adverse to quality are presented in Section 5.5 of this report. The audit team also identified two recommendations during the audit. These recommendations are detailed in Section 6.0 of the report.
2.0 SCOPE
The audit was conducted to evaluate the technical adequacy of the following LBNL deliverables (Milestone Reports) and the effectiveness of critical process steps implemented during the preparation of these deliverables:
The audit team conducted interviews and reviews of documentation in accordance with the approved audit plan to evaluate the adequacy of deliverables and effectiveness of critical process steps.
2.1 Process Steps/Products/Documentation
The performance-based evaluation was based upon the following:
2. Acceptable results and quality of the end product.
3. Documentation that substantiates the quality of products.
4. Performance of trained and qualified personnel.
5. Implementation of applicable QA program elements.
The following critical process steps were considered during the evaluation of Milestone Reports:
1. Scientific investigation planning
2. Identification, traceability and control of data
5. Control of software
2.2 Technical Areas
The audit included a technical evaluation of the adequacy of Milestone Reports. Details of the technical evaluation are documented in Section 5.4 of this report.
3.0 AUDIT TEAM AND OBSERVERS
Name/Title/Organization
Edward P. Opelski Audit Team Leader, Office of Quality Assurance (OQA)
James Blaylock Auditor, OQA
Kenneth T. McFall Auditor, OQA
Keith M. Kersch Technical Specialist, CRWMS M&O
William Belke Observer, Nuclear Regulatory Commission (NRC)
T. H. Carter Observer, NRC
Jeffrey Ciocco Observer, NRC
4.0 AUDIT MEETINGS AND PERSONNEL CONTACTED
A pre-audit meeting was conducted at LBNL on May 18, 1998. Daily debriefings were held to apprise LBNL management and staff of the progress of the audit and of any identified conditions adverse to quality. A post-audit meeting was conducted at LBNL on May 22, 1998.
Personnel contacted during the audit, including those that attended pre- and post-audit meetings, are listed in Attachment 1.
5.0 SUMMARY OF RESULTS
5.1 Program Effectiveness
With the exception of the areas where conditions adverse to quality were identified, the audit team concluded that critical process steps applicable to the preparation of milestone reports were effectively implemented.
5.2 Stop Work or Immediate Corrective Actions Taken
There were no stop work actions or immediate corrective actions taken as a result of the audit.
5.3 QA Program Activities
A summary table of audit results is provided in Attachment 2. Details of the audit, including the objective evidence reviewed, are documented in the audit checklist. The checklist is maintained as a QA record.
5.4 Technical Audit Activities
Joe Wang is the Principal Investigator (PI) for the two milestones audited. During the interviews the PI and other LBNL staff, besides having a good grasp of the QA program requirements, were very cooperative and helpful.
The Milestone Reports audited support WBS 1.2.3.3.1.2.4, "Percolation in the Unsaturated Zone-Exploratory Studies Facility (ESF)." The objective of this element is to conduct in-situ ambient cross-hole pneumatic and liquid-release niche seepage studies and alcove surface infiltration studies to evaluate fluid flow within the unsaturated zone (UZ) at Yucca Mountain. The purpose of the niche drift scale seepage tests is to better understand the drift seepage process, percolation fluxes, and capillary barrier system.
Milestone SP33PLM4, "Testing and Modeling of Seepage into Drift, Input of Exploratory Study Facility Seepage Test Results to Unsaturated Zone Models," summarizes the drift seepage test results which have been successfully interpreted with the UZ drift scale models and site scale models to produce the flow fields for inputs to the Total System Performance Assessment for the Viability Assessment.
Although a completed milestone, the report appears to be more like an interim report on work being done on ESF seepage. The audit team determined the work to be technically sound with few flaws. The planning process for this work is not very well defined. The statement of work associated with this milestone consists of two sentences in the Planning and Control System (PACS) Participant Planning Sheet (PPS). The work evolved as the result of a series of negotiations between LBNL and CRWMS M&O management. The Master Planning Notebook, YMP-LBNL-JSW-6.0, contains a general discussion of the ESF seepage work that was planned, but did not reference this specific milestone. Only through interviews with the PI and others could the audit team identify the specific planning documents and their applicability, as well as the various steps in the planning process. Yvonne Tsang, Acting Laboratory Lead, indicated in the response, which was negotiated during this audit, to previously issued DR LBNL-98-D-029, that the planning process for future milestone deliverables will be detailed in a controlled document. Recommendation #1 addresses planning weaknesses.
The report for Milestone SP33PLM4 contained two references that could not be traced. One was a personal communication and the other was an unpublished technical paper. These deficiencies required only remedial action and were corrected during the audit. The personal communication was documented in a memorandum and along with the unpublished technical paper submitted to the TIC.
During the production of this milestone, some simple, but unqualified, modifications were made to software program ITOUGH2. Although this was noted in the milestone report, the audit team recommended the use of unqualified software, even though flagged, should cease at this stage of the project (see Recommendation #2).
Milestone SP33PBM4, "Draft Report on Fracture Flow and Seepage Testing in the ESF," is work in progress. There was not much documented work on this milestone, which is not due for several months. However, this milestone suffers from the same planning weaknesses as Milestone SP33PLM4.
Both milestones audited were of good technical quality. The preliminary testing and modeling results in drift seepage and alcove studies should lead to a better understanding of the seepage process and capillary barrier system important to the potential repository performance. These milestones are considered work in progress that provides the foundation for future seepage analysis and modeling.
5.5 Summary of Conditions Adverse to Quality
The audit team identified three conditions adverse to quality during the audit. Corrective action for these conditions will be addressed in the response to the recently issued OCRWM deficiency documents identified in Section 5.5.2 of the report.
None.
5.5.2 Deficiency Reports
LBNL-98-D-029
This DR was issued to LBNL on March 3, 1998. The DR documents that technical procedures were documented in scientific notebooks instead of in TIPs and that scientific notebooks used for studies were not reviewed during technical reviews of milestone reports. During the audit, it was identified that three procedures that were used to control specific field activities associated with Milestone SP33PLM4 were documented in scientific notebook YMP-LBNL-JSW-6.0 instead of in TIPS. In addition, the audit identified that the scientific notebooks used in the production of this milestone were not reviewed during the technical reviews of the report.
LBNL-98-D-031
This DR was issued to LBNL on March 3, 1998. The DR documents that the applicable manager or the Human Resources Department had not verified experience for YMP LBNL personnel. During the audit, it was identified that there had been no verification of experience for scientists or technical reviewers associated with the Milestone Reports.
5.5.3 Performance Reports
None.
5.5.4 Conditions Adverse to Quality Corrected During the Audit
The report for Milestone SP33PLM4 contained two references that could not be traced. One was a citation of a personal communication and the other was cited as submitted unpublished technical paper. These deficiencies required only remedial action and were corrected during the audit. The personal communication was documented in a memorandum and along with the unpublished technical paper submitted to the TIC.
None.
6.0 RECOMMENDATIONS
The following recommendations resulted from the audit and are presented for CRWMS M&O and LBNL management consideration:
7.0 LIST OF ATTACHMENTS
Attachment 1: Personnel Contacted During the Audit
Attachment 2: Summary Table of Audit Results
ATTACHMENT 1
PERSONNEL CONTACTED DURING THE AUDIT
Name |
Organization/Title |
Pre-audit Meeting |
Contacted During Audit |
Post-audit Meeting |
Aden-Gleason, Nancy |
LBNL Engineering Assurance (EA) Manager |
X |
X |
X |
Ahers, Rick |
LBNL Principal Research Associate |
X |
X |
|
Benson, Sally |
LBNL Earth Science Department Division Director |
X |
X |
|
Birkholzer, Jens |
LBNL Geological Scientist |
X |
X |
|
Doughty, Christine |
LBNL Staff Scientist |
X |
||
Fairley, Jerry |
LBNL Hydrologist |
X |
||
Fissekidou, V. A. |
LBNL EA Specialist |
X |
X |
X |
Finsterle, Stephan |
LBNL Staff Scientist |
X |
X |
|
Harris, Stephen |
OQA Representative (LBNL) |
X |
X |
|
Hastings, Cheryl |
LBNL Program Administrator |
X |
X |
X |
Hayes, Larry |
CRWMS M&O Operations Manager |
X |
X |
|
Hinds, Jennifer |
LBNL Hydrologist |
X |
||
Horton, Donald |
Director, OQA |
X |
X |
|
James, April |
LBNL Senior Research Associate |
X |
X |
|
Link, Suzanne |
LBNL Technical Data Manager |
X |
||
McClung, Ivelina |
LBNL Administrative Assistant |
X |
X |
|
Mangold, Donald |
LBNL Audit Lead |
X |
X |
X |
Oldenburg, Curt |
LBNL Staff Scientist |
X |
||
Patterson, Russ |
DOE/YMP Hydrology Technical Lead |
X |
X |
|
Pelletier, John |
OQA Representative (LLNL) |
X |
X |
|
Trautz, Robert |
LBNL Principal Research Associate |
X |
X |
|
Tsang, Chin-Fu |
LBNL ESD Department Head |
X |
X |
X |
Tsang, Yvonne |
LBNL ESD Laboratory Lead |
X |
X |
X |
Wang, Joe |
LBNL Staff Scientist PI |
X |
X |
X |
ATTACHMENT 2
SUMMARY TABLE OF AUDIT RESULTS
Product |
Process Steps |
Details (Checklist) |
Deficiencies |
Recommend-ations |
Process Effectiveness |
Product Adequacy |
Overall |
Milestone SP33PLM4 |
Scientific Investigation Planning |
pgs. 1-2A and 10
|
|
1 |
SAT |
SAT |
SAT |
Identification of Data |
pg. 3 |
|
SAT |
SAT |
|||
Data Traceability |
pg. 3 |
SAT |
SAT |
||||
Data Review and Analysis |
pgs. 4-4A |
SAT |
SAT |
||||
A Q - non Q@ Status of Data |
pg. 4 |
2 |
SAT |
SAT |
|
||
Control of Software |
Pgs. 7-8 |
SAT |
SAT |
||||
Control of References |
pg. 17 |
CDA |
SAT |
SAT |
|||
Control of Scientific Notebooks |
pgs. 6-7 |
LBNL-98-D-029 |
UNSAT |
SAT |
SAT |
||
Control of Assumptions |
Pgs. 11-16 |
SAT |
SAT |
||||
Control of Measuring and Test Equipment |
Pgs. 9-9B |
SAT |
SAT |
||||
Independent Review of Results |
pgs. 4-4A |
LBNL-98-D-029 and LBNL-98-D-031 |
UNSAT |
SAT |
|||
Data Input to Genesis |
pg. 5 |
SAT |
SAT |
Legend: SAT............Satisfactory UNSAT......Unsatisfactory
|
ATTACHMENT 2
SUMMARY TABLE OF AUDIT RESULTS
(Cont’d)
Product |
Process Steps |
Details (Checklist) |
Deficiencies |
Recommend-ations |
Process Effectiveness |
Product Adequacy |
Overall |
Milestone SP33PBM4 |
Study Planning |
Pg. 1 |
1 |
SAT |
SAT |
||
Identification of Data |
Pgs. 2-3 |
SAT |
SAT |
SAT |
|||
Data Traceability |
Pgs. 2-3 |
SAT |
SAT |
Legend:
SAT.............Satisfactory
UNSAT........Unsatisfactory