Protecting People and the EnvironmentUNITED STATES NUCLEAR REGULATORY COMMISSION
1
1 UNITED STATES OF AMERICA
2 NUCLEAR REGULATORY COMMISSION
3 OFFICE OF THE SECRETARY
4 ***
5 MEETING WITH ACRS ON RISK INFORMING PART 50
6
7 ***
8 PUBLIC MEETING
9 Nuclear Regulatory Commission
10 One White Flint North
11 Building 1, Room 1F-16
12 11555 Rockville Pike
13 Rockville, Maryland
14 Thursday, March 2, 2000
15 The Commission met in open session, pursuant to
16 notice, at 9:29 a.m., the Honorable RICHARD A. MESERVE,
17 Chairman of the Commission, presiding.
18 COMMISSIONERS PRESENT:
19 RICHARD A. MESERVE, Chairman of the Commission
20 GRETA J. DICUS, Member of the Commission
21 NILS J. DIAZ, Member of the Commission
22 EDWARD McGAFFIGAN, JR., Member of the Commission
23 JEFFREY S. MERRIFIELD, Member of the Commission
24
25
2
1 STAFF AND PRESENTERS SEATED AT THE COMMISSION TABLE:
2 KAREN D. CYR, General Counsel
3 ANNETTE L. VIETTI-COOK, Assistant Secretary
4 JOHN SEIBER
5 GRAHAM WALLIS
6 ROBERT UHRIG
7 WILLIAM SHACK
8 JOHN BARTON
9 THOMAS KRESS
10 DANA POWERS
11 GEORGE A. APOSTOLAKIS
12 MARIO BONACA
13 ROBERT SEALE
14
15
16
17
18
19
20
21
22
23
24
25
3
1 P R O C E E D I N G S
2 [9:29 a.m.]
3 CHAIRMAN MESERVE: Good morning. I would like to
4 welcome you all to our session this morning with the
5 Advisory Committee on Reactor Safeguards. As I think all of
6 you know, we had a meeting with you last November. I
7 remember it well because it was my very first Commission
8 meeting. At that occasion we had the opportunity -- because
9 of the substance as well, I should add. We had a discussion
10 at that meeting about the various NRC initiatives to
11 risk-inform our approach to our regulatory activities.
12 After that meeting, we did present a number of
13 specific questions to the ACRS to consider, about
14 risk-informing our regulations, and, also, some questions
15 about our oversight program. Our session today will deal
16 with both of those matters.
17 So I am pleased to welcome Dr. Powers and his
18 colleagues to discuss these matters with us. Let me suggest
19 that in order to allow us to proceed efficiently, what we
20 would like to do is to go through all of the briefings with
21 regard to risk-informing 10 CFR Part 50, and we will then
22 open to questions, because they are linked with each other.
23 It is very important to the Commission that we
24 have ample time for questions, that the interaction back and
25 forth is extraordinarily valuable to us, and so we would
4
1 urge you to proceed through the charts, which, of course, we
2 have had an opportunity to review before the session this
3 morning, with dispatch so as to enable us to have ample time
4 for interaction.
5 After we complete that, we will then turn to the
6 presentation on the oversight.
7 If that is acceptable, let me turn to my
8 colleagues and see if any of them have any opening comments?
9 And, if not, Dr. Powers, you may proceed.
10 DR. POWERS: Thank you, Dick. This morning we are
11 going to begin on discussions on almost a philosophical note
12 as we discuss the technical strategies that can be
13 considered as we move toward risk-informing the reactor
14 regionals. Following that immediate discussion, Dr. Kress
15 will review for you some of the social and technical
16 impediments that we thing exist to the greater use of risk
17 information in reactor regulations.
18 Professor Apostolakis will discuss the roles of
19 importance measures in the risk-informed regulatory system,
20 and some of the limitations that we think exist on these
21 important measures that get derived from probabilistic risk
22 assessments. I think at that point you want to interrupt
23 the presentations and allow for discussions of collective
24 topics.
25 Once we have completed those discussions, we will
5
1 move toward a more applied and less theoretical aspect of
2 the whole issue of risk-informing the regulations and
3 consider the performance indicators that are to be used in
4 the new plant oversight and assessment process.
5 You catch us here at a point where looking at the
6 performance indicators is still very much a work in
7 progress. I have included in your package a quote from John
8 Ahern's panel on the ACRS effectiveness, and I tell you that
9 you have caught us at a point where we are still working
10 with the difficulties, the dilemmas, the uncertainties and
11 the contrasting opinions. We certainly haven't reached a
12 consensus on these performance indicators.
13 Mr. Barton is going to try to describe for you
14 areas where there is a general agreement within the ACRS,
15 and areas where, to put it politely, discussions will have
16 to continue. And, in fact, they are continuing. As soon as
17 we are done here, we will be meeting again with the staff to
18 discuss the performance indicators.
19 A theme that I think will emerge throughout our
20 discussions here is the question of what types of analytic
21 capabilities the NRC is going to have to have if it is going
22 to sustain a risk-informed regulatory system. In that
23 regard, it is unfortunate we are not going to have time to
24 delve into the questions of the significance determination
25 process the staff has constructed for the new oversight and
6
1 performance process. It is my own feeling that when we get
2 to the significance determination process, we will see most
3 clearly the kinds of analytic support that it would be
4 desirable for the staff to have available as it carries out
5 a risk-informed regulatory system.
6 With that introduction, I propose to move directly
7 to the topic of a technical road map for risk-informing 10
8 CFR Part 50. Let me begin by saying that for as long as I
9 can remember, the ACRS has been enthusiastic about the idea
10 of bringing greater use of risk information into the
11 regulatory process. My association with the committee
12 probably extends over 24 years, as both a supplicant and a
13 member, and throughout that period I saw the ACRS asking for
14 more quantification, more use of risk in defining the
15 regulatory process.
16 This current incarnation of the ACRS is no less
17 enthusiastic about the use of risk information and
18 regulatory regulation. Quite frankly, we may be more
19 enthusiastic about that. Many of us have matured
20 technically along with the abilities to do probabilistic
21 risk assessment. Many of us have actually been part of the
22 maturation process.
23 What I remember well is about two years ago
24 Commissioner Diaz visited with the ACRS and he said to us,
25 he challenged us, he said, "Why can't we just go ahead and
7
1 risk-inform the entirely of the reactor regulations as a
2 holistic body?" My reaction to that was, "Wow." Now I know
3 why Commissioners get the big bucks. This is the kind of
4 bold thinking and leadership that you would like to see
5 coming from the Commission. And today maybe we are in the
6 process where we can start discussing some of the approaches
7 that will be taken in doing this.
8 But I am going to have to admit to you that
9 defining a technical road map for risk-informing 10 CFR Part
10 50 just has not been a priority activity for the ACRS over
11 the last year. In the last year we have been quite busy
12 handling license renewal and some of the other initiatives
13 that the Commission has undertaken in response to Congress.
14 Staff, on the other hand, has moved aggressively and
15 developed a three option approach that the Commission has
16 approved.
17 Quite frankly, the staff has a problem here.
18 There is no guidance for them available. Risk-informing the
19 regulations is a pioneering activity that I place akin to
20 first of a kind engineering -- not much to tell you how to
21 go about doing it. We can expect that there will be blind
22 ends and stumbles along the way, because it is so new. ACRS
23 has tried to be supportive of the staff's effort, and at the
24 Commission's behest, we have tried to identify potential
25 pitfalls and potential barriers to risk-informed regulation.
8
1 On the next slide I show you some of those, I
2 think you are familiar with those. We have written you
3 reports on those. Dr. Kress will elaborate on some of those
4 in his presentation.
5 The staff has advanced a approach. There is no
6 question other approaches could have been advanced. At the
7 one extreme, one could imagine an approach that says the
8 regulations are risk-informed. People were definitely
9 thinking about risk when they came up with the regulations.
10 They were thinking about risk in perhaps less quantitative
11 terms than we do now, but risk nevertheless, and we don't
12 need to change the regulations, we need to change the
13 Regulatory Guides that implement those regulations.
14 At the other extreme is what I call the clean
15 sheet and the holistic approach. I think that is what
16 Commissioner Diaz had in mind when he came over and
17 challenged the ACRS, and I think that is the approach that
18 the ACRS would be most enthusiastic about.
19 We see two possibilities for doing a clean sheet
20 or holistic approach. One of the possibilities would be to
21 say let's design the regulations for an arbitrary reactor
22 and not think about the reactors we have in mind, but have
23 some regulations that would be applicable to any reactor.
24 The other approach is to say, no, we have some
25 existing reactors, we have some 3,000 man reactor years of
9
1 operating experience, let's build upon that. It is just too
2 big of a step to try to develop regulations for an arbitrary
3 reactor.
4 I think the staff and the ACRS are more
5 comfortable in thinking about a holistic approach and
6 thinking about it in connection with the existing reactors.
7 As I have indicated, developing a road map, a
8 technical road map to the risk-informing of the regulations
9 just has not been a priority activity. We have not
10 attempted to develop a report to you on that subject. And
11 so in preparation for this discussion, I have gone through
12 the minutes of ACRS meetings and our past letters, and my
13 memory of our past discussions to try to distill out for you
14 some key elements of philosophy that I think the ACRS would
15 have written in a letter if they had reported to you on
16 this.
17 I note that many of the discussions that the ACRS
18 has had on risk-informing the reactor regulations have
19 concentrated on the issues of focusing the resources of both
20 the regulatory body and the licensees on the areas of
21 greatest risk. On the other hand, when I look at what the
22 staff has brought before us, they frequently speak of the
23 issues of burden reduction. That language, "burden
24 reduction," never seems to appear in the internal
25 discussions at the ACRS. I only point this out as an item
10
1 of interest because I found the discrepancies striking. I
2 suspect that these don't reflect a difference in attitude,
3 but rather a difference in language.
4 The area where the ACRS has been consistently
5 concerned is in the area of coherence of the regulations. I
6 find concern about coherency of the regulation in ACRS
7 reports extending back at least 15 years. What I also
8 understand is that coherency in the regulations means a lot
9 of things to a lot of people. The one consistent aspect of
10 coherency in the regulations, I have tried to depict on this
11 slide, a diagrammatic slide, and that is a hierarchy where
12 regulations flow directly from the enabling act, the Atomic
13 Energy Act.
14 The Commission has established risk-informed
15 safety goals. If we could also have with that a definition
16 of adequate protection that is called for in the Atomic
17 Energy Act that also has the language of risk in it, we
18 would be well on the well on the way to establishing what we
19 have called the three region approach that was pioneered in
20 the definition of Reg. Guide 1.174, that is, in the spectrum
21 of activities that could be undertaken, there are those that
22 are clearly unacceptable. There is a region of activities
23 where one would have regulatory attention perhaps graded by
24 the magnitude of risk associated within this allowable
25 regime. And, finally, you would have activities that pose
11
1 so little risk that they would be -- they could proceed with
2 no prior approval of the regulatory body.
3 This definition of three regions of the spectrum
4 of activities might not be enough to assure safety. One
5 could easily imagine, and at least in a hypothetical
6 situation, that one could satisfy the risk requirement
7 simply by focusing all this attention on mitigation of
8 accident consequences. It is our suspicion that one could
9 never reach the safety goals by focusing just on mitigation
10 of accident consequences without also considering prevention
11 or intercession. But one might be able to meet the minimum
12 acceptable, provide adequate protection by focusing just on
13 mitigation of accident consequences.
14 Such a highly unbalanced approach towards
15 satisfying the regulations might not be satisfactory. It
16 might be that a balance was sought to assure safety. That
17 we feel is one of the two incarnations of defense-in-depth.
18 Defense-in-depth is a balance between accident
19 initiators, accident intervention and accident mitigation.
20 We see defense-in-depth as a policy, it is one that would
21 not disappear as risk information becomes more reliable and
22 more available.
23 The second incarnation of defense-in-depth is more
24 focused on compensation for uncertainty and our capabilities
25 to assess risk.
12
1 Once one has defined the three regions of activity
2 space and how regulations are applied, then risk information
3 comes in. The ACRS is consistent in its belief that
4 regulation should be where there is risk. And I have listed
5 down some of the areas that have been identified in risk
6 under various modes of operations, full power and shutdown
7 operations, fire initiators and seismic. I simply comment
8 that we see the staff treating now fire in a unique way,
9 rather than as part of the overall initiators that can
10 affect plants, and it is a curiosity to us.
11 Staff has elected not to pursue this top-down
12 approach, however, we have become aware that the Department
13 of Energy is sponsoring an effort at risk-informing the
14 regulations following a top-down process. Their focus is
15 geared on a future generation of reactors, what they call
16 the Generation 4 reactors, and the ACRS members are
17 following this effort by the Department of Energy.
18 Staff is attacking the process of risk-informing
19 the regulations in what I have called a piece-wise approach.
20 I don't infer any pejorative to that piece-wise, it is the
21 way they have attacked it. And when you attack the
22 risk-informing of the regulations in a piece-wise fashion,
23 you face some challenges. The most obvious of those
24 challenges, the one that comes most to mind is if you
25 risk-inform regulation, you are liable to find yourself in
13
1 conflict with another regulation.
2 I think we are all very familiar with this from
3 the graded quality assurance where we had successfully
4 risk-informed a graded quality assurance process, but it is
5 not possible to apply because it conflicts with another set
6 of regulations.
7 Another challenge one faces has to do with the
8 language of the regulations. They were written oftentimes
9 in an era when our ability to define and measure risk was
10 much more qualitative than it is now. And today we have a
11 great deal more precision when we speak of risk. This
12 conflict that arises, I call the 10 CFR 50.59 phenomenon
13 because I think we encountered it first there. But A
14 Appendix A provides us a good example.
15 I quote here some languages out Appendix A.
16 Appendix A is, of course, the general design criteria. If
17 we look perhaps at GDC4, it has to do with environmental
18 effects and accidents, we see it has language that says the
19 "probability of fluid system piping rupture is extremely
20 low." Now, extremely low at one time was something where it
21 was small enough to be negligible. Today we would interpret
22 this as so small that it falls outside the cut sets used in
23 our probabilistic risk assessment. This is probably much
24 lower than the architects of this language had in mind when
25 they wrote the general design criteria.
14
1 If you look at GDC12, it has to do with power
2 oscillations, it says "not possible." This is very
3 difficult language for engineers to use because there is
4 always some possibility. Perhaps it is as low as the
5 possibility of meteorites striking your home, but there is
6 always some possibility. And so when you say "not
7 possible," you are going to create a language that is going
8 to conflict with other regulations.
9 And I have quoted several others, "extremely low
10 probability." Without some quantification, that is going to
11 be interpreted as falling out the cut sets in a
12 probabilistic risk assessment, again, probably well below
13 the probabilities that were in mind of those who originated
14 it.
15 What I conclude from this is, because the general
16 design criteria have corresponding regulations in the
17 general body of regulations, it would be surprising to me if
18 the regulations could be made risk-informed without also
19 risk-informing the general design criteria. In fact, if you
20 don't do it, you are going to run into the same problem that
21 we encountered with graded quality assurance. You are going
22 to have well risk-informed one regulation, and it is not
23 going to make one whit of difference on the licensee's or
24 the regulator's course of action because he is going to be
25 constrained by the general design criteria.
15
1 With that, of course, having come to that
2 conclusion on one appendix, one asks, what about the other
3 appendix, the famous Appendix B that deals with quality
4 assurance? My view is Appendix B is a codification of the
5 best practices for quality insurance. We have found that it
6 is really not possible to quantify the risk worth of these
7 quality assurance requirements.
8 The quality assurance requirements are widely
9 viewed as burdensome, and they may be even a distraction of
10 focus. But I think we have discovered that a graded
11 approach is possible, that it is possible to go into the
12 systems, the components, and the structures within a reactor
13 and assign to them a risk worth or a risk significance, and
14 to grade the quality assurance according to that risk worth
15 or risk significance.
16 My conclusion is that we probably can risk-inform
17 the rest of the regulations without paying attention to
18 Appendix B, but I hasten to note that there is a value in
19 risk-informing Appendix B, a visibility, because it, like
20 Appendix K, risk-informing those two appendices would be a
21 very visible and very desirable demonstration of the
22 Commission's commitment to moving toward risk-informed
23 regulation.
24 I wanted to conclude by bringing other challenging
25 areas I think the staff is going to encounter as they
16
1 proceed on their efforts to risk-inform the regulation. One
2 area that the ACRS continues to struggle with is the area of
3 performance-based regulations. If one establishes
4 performance standards, those standards must come from
5 something. If they come from risk, of course, then we would
6 have risk-based regulations.
7 If they don't come from risk assessments, where do
8 they come from? And will these other sources cause the
9 performance standards to degenerate into prescriptive
10 regulations that we already have?
11 Another issue is what to do with design basis
12 accidents. Quite frankly, it is not clear to me whether
13 these are useful entities in a risk-informed regulatory
14 world. They may well be vestiges of an era when design and
15 construction were the predominant issues faced by the
16 industry and by the regulatory body. That is different from
17 the current era where operations and maintenance are the
18 focus of attention.
19 On the other hand, design basis accidents do have
20 a value, they provide a design to standard that makes it
21 easier for designs. I wonder if this design tool needs to
22 be codified in the regulations themselves.
23 The final challenge that I think is going to come
24 up repeatedly is the regulator is going to feel a need to
25 have some understanding of the probabilistic risk assessment
17
1 tools that licensees use. Having an understanding could
2 easily lead to the imposition of requirements and
3 regulations that would have the tendency to ossify the
4 methods that exist today and pose a barrier to the continued
5 development and refinement, and improvement of risk
6 assessment methods. This is a topic the ACRS is especially
7 concerned about, and I think you have seen in some of our
8 letters in discussing PRA standards.
9 At that point, I have outlined for you the kinds
10 of thinking we would have on the technical road map to
11 risk-informing the regulations. Now, I will turn to Dr.
12 Kress to discuss with you some of the impediments and the
13 barriers and possible pitfalls that we see in the greater
14 use of risk in the regulatory process.
15 Dr. Kress.
16 DR. KRESS: Thank you, Dr. Powers.
17 We were requested to give some examples of what we
18 considered impediments. Before I do that, I want to make it
19 clear that we don't want this to be interpreted to mean that
20 these impediments of such nature and degree that the
21 Commission cannot proceed with risk-informing the
22 regulations. What it really means is these are things that
23 have to be recognized and perhaps accommodated with a bit
24 more conservative risk-informing of the regulations or a bit
25 more conservative in the decision-making process when one
18
1 comes to interpret the regulations in terms of
2 plant-specific issues. So these are impediments, but we
3 don't think they are roadblocks that stop the process.
4 We also noted in one of our reports that such
5 impediments tend to have two different classifications, two
6 different characteristics. We chose to label those, one,
7 cultural and institutional, the other one technical.
8 The cultural and institutional ones are
9 characterized by attributes, such things as attitudes;
10 impressions; organizational type barriers, like this is the
11 way we have always done it and we continue to do it;
12 resource limits and things like that. The common theme with
13 those is they are people problems.
14 And as I review those, why those things exist,
15 that if we do the job right of actually technically
16 defensible process of risk-informing regulations, those will
17 just cure themselves in time. That people will begin to
18 recognize the benefits and the good parts of risk-informing
19 regulations and these attitudes and things will change. So
20 we chose not to focus on this type of impediment, although
21 there are a lot of those around.
22 Instead, the other type, the technical impediments
23 relate to what we consider technical shortcomings in risk
24 assessment and its application. We don't believe these will
25 just go away by themselves. Some overt action on the part
19
1 of the Commission will be required to fix these.
2 What we did was list a number of these that we
3 think are the more important ones. What I plan to do is
4 touch a little on each of these except the Item 4, which is
5 the use of importance measures. George Apostolakis will
6 talk in some detail to that one.
7 The first one on our list was PRA inadequacies and
8 incompleteness. We do feel that there are some deficiencies
9 in PRAs and these are the ones that we think are the more
10 significant ones. As Chairman Powers mentioned, that fires
11 are treated in sort of a unique way, and they are not really
12 part of the PRA. We do not have good phenomenological
13 models for how fires progress and spread, and the damage
14 they do to equipment and instrumentation. Nor do we have
15 such models for the smoke associated with them. So we think
16 that is an area that PRAs are very weak in.
17 It is generally recognized by most PRA
18 practitioners that the human performance element in PRAs is
19 the weakest part, particularly in errors of commission or
20 when one thinks about unproceduralized activities that might
21 come about. These are just not well treated in PRAs at all.
22 Organizational and safety culture factors are
23 often thought to be a large contributor of risk to safety,
24 but we just have no way of treating those at all PRAs. They
25 are just not part of PRA. Unless they reflected in
20
1 equipment performance and things like that, they are just
2 not treated directly.
3 The second bullet, it is our opinion that most
4 PRAs are actually incapable of assessing the risk
5 contributions from low-power and shutdown conditions. And
6 this is because -- and when risk-informing the regulations,
7 what one needs is a projection of the average lifetime risk
8 due to these conditions. And the nature of low-power and
9 shutdown risk is that they are dynamic, they are always
10 changing in time. And the PRAs are not dynamic, they are
11 not built to handle that sort of situation. So that that is
12 a problem we see that exists in how you assess the risk in
13 low-power and shutdown conditions.
14 Now, that is to differentiate itself from the risk
15 management activities of the licensees and the industry.
16 They have good ways to manage the risk if they have a
17 planned shutdown and know what the configurations are going
18 to be and how to control those. That is a different
19 situation and that is not what we are talking about.
20 And we think we need to be vigilant in looking at
21 the reliability database because it has tended to focus in
22 the past on what we would call safety significant systems
23 and components. Well, what we are finding out is that
24 doesn't capture all the things that are really safety
25 important, so that we need to be sure that the database
21
1 includes other systems, as well as passive components, which
2 we probably haven't developed a database for much at all.
3 The second one has to do with risk-acceptance
4 criteria. This is addressing the slide that Dr. Powers had
5 on the three region approach. If you do have such an
6 approach in risk-informing the regulations, you need some
7 sort of quantitative description of what these boundaries
8 are, the two boundaries, the upper and lower one.
9 The lower boundary is probably what I would call
10 the safety goals. How safe is safe enough? Below which you
11 don't need to pay much regulatory attention. It is the
12 upper boundary that is disturbing. It is the one above
13 which you are unacceptable. And these boundaries, in our
14 risk language, are boundaries of CDF and LERF, for example.
15 And this upper boundary needs to be quantified we think. It
16 would be an additional quantification that would go into the
17 definition of adequate protection in addition to the
18 definition that you already have.
19 And when one does this quantification in terms of
20 CDF and LERF, we shouldn't forget that there are other
21 regulatory objectives, and I have listed some of those
22 possible ones, societal risk, land interdiction, worker
23 exposure. Those are all things we deal with the regulations
24 as they are now. It is not clear to us that LERF, for
25 example, as it is presently incarnated, deals appropriately
22
1 with those. It may have to be defined differently. You may
2 have to have different limits for it if you are going to
3 deal with these other regulatory objectives. The idea is we
4 just shouldn't forget about those when we risk-inform the
5 regulations and some thought should be given to them.
6 One way to be conservative in your regulations and
7 risk-acceptance criteria is to use defense-in-depth. We
8 happen to like very much the White Paper's definition of
9 defense-in-depth in terms of successive compensatory
10 measures to prevent and to mitigate. What we see as a
11 problem is when the staff gets ready to implement that
12 definition, they really need some criteria or guidance on
13 just how many compensatory measures are necessary and how
14 good do these have to be. They will have to make those
15 decisions.
16 And we have written at least one letter on the
17 subject where we are addressing, or at least exploring
18 putting limits like this on defense-in-depth, and we will
19 have another one coming out shortly from the joint
20 subcommittee with the ACNW which also addresses that
21 subject. And I won't dwell on it now, but we think we are
22 making some progress on how to put limits on it.
23 And, finally, the thing that comes up all the time
24 is the variation in PRA quality and scope. We recognize
25 that there is a great deal of difference in the scope and
23
1 quality of the IPEs, and we are very pleased that the agency
2 is involved in an activity to develop standards with the
3 ASME and the ANS, and we think this activity can go a long
4 way towards solving this particular difficulty. We are
5 looking forward to reviewing the next incarnation of these
6 standards when they get ready to come to us again.
7 Our concern, one of our concerns has been is at
8 least the opinion of one member of the ACRS that quality of
9 a PRA is measured by its uncertainty. If the uncertainty is
10 done correctly, that is a measure, a metric you can use to
11 say how good is this PRA. And, so, we will want to see,
12 when the ASME and ANS and staff comes to us with the
13 standards, how they are treating uncertainties, how they
14 intend to deal with them in the standards.
15 And, in addition to that, we think once
16 uncertainties are appropriately dealt with in a PRA, the
17 staff itself needs guidance on how to consistently use these
18 uncertainties in their decision-making process.
19 So those are the two areas that we think are
20 things we will tend to focus on. With that, I will turn it
21 back to you, Dr. Powers.
22 DR. POWERS: Professor Apostolakis will now
23 discuss importance measures.
24 DR. APOSTOLAKIS: The first slide gives us an
25 opportunity to look at the issue of importance measures in a
24
1 broader context. If we look at the two boxes at the bottom
2 called "expert panel deliberation" and "risk-informed
3 decision," we can say that the way decisions were being made
4 before PRA was developed were exactly this way. The
5 decisions were based on the judgment of people or groups of
6 people and they were to some extent risk-informed, as Dr.
7 Powers said earlier, but that risk was unquantified, that
8 was not a risk assessment the way we understand it now as a
9 PRA.
10 And I believe that today when we say risk-informed
11 regulation, we really mean a regulatory action that is
12 utilizing some insight, some results from a PRA, not this
13 just unquantified risk-informed decision that we used to
14 make and that we used to have.
15 Now, this has become more clear and concrete by
16 adding the two boxes on the left at the bottom of the figure
17 where, especially after the publication of Regulatory Guide
18 1.174, it became very clear, very formal, that when one
19 considers a number of decision options, one has to assess
20 the impact of each option on two metrics, the core damage
21 frequency and the large early release frequency, and then
22 based on these results will be forwarded again to the expert
23 panel, which will make the ultimate decision by taking into
24 account other considerations as appropriate. So this now a
25 truly risk-informed decision-making process as we understand
25
1 it.
2 Then we realize that, unfortunately, we cannot
3 always assess the impact on CDF and LERF. There are several
4 important situations where this cannot be done, and this
5 includes the special treatment requirements. We simply
6 don't have models that will tell us how the CDF will be
7 affected if we relax certainly quality assurance
8 requirements, for example. We can do sensitivity studies
9 and "what if" studies, but we really don't have them in the
10 sense that, say, 1.174 requires.
11 Then we go to the top box, and we come up with a
12 better -- with a different idea, not better, a different
13 idea. We realize that we can develop categories of systems,
14 structures and components that tell us how risk significant
15 these SSCs are. And we do this by using some information
16 from the PRA, most often importance measures, to define
17 these categories.
18 And then, as you see, the two arrows, we go
19 straight to the expert panel. We are giving them now
20 information regarding the risk significance of the SSCs and,
21 of course, the decision options, and they will have to make
22 a decision that will be, again, risk-informed, but it will
23 not have the benefit of the information or possible
24 information regarding the impact of these decision options
25 on CDF and LERF
26
1 Now, this diagram I think makes it very clear that
2 one has to talk about the various methods for categorizing
3 the components, like importance measures, which I am
4 supposed to do today, but also other things like the impact
5 on CDF, delta CDF and so on. In the context of this
6 integrated decision-making process, one cannot just look at
7 importance measures as a mathematical quantity and start
8 saying, you know, they are good, they are not good. It is
9 the integrated process that counts.
10 This is very good because it lifts a lot of the
11 burden from the PRA analyst. We don't have to be perfect
12 now, which is very good.
13 DR. POWERS: But I thought you were.
14 [Laughter.]
15 DR. APOSTOLAKIS: The generic analyst. On the
16 other hand, we are beginning to see now something that also
17 came up in the context of importance measures, and I think
18 we will see more of it. There is this trend -- not trend,
19 but maybe point of view that, well, since you have the
20 expert panel there, you don't really have to do a very good
21 job on the left, on categorizing the SSCs or assessing the
22 impact on CDF and LERF, because the expert panel will take
23 care of it. Your methods can be imperfect, the expert panel
24 will see that and the decision will be the correct one.
25 Well, the big question before us I think will be,
27
1 how far can you push this argument? In fact, sometimes you
2 hear that non-PRA methods can be used in risk-informed
3 regulation. Well, that takes us back 30 years ago when risk
4 was not quantified. So, in this context, we have to look at
5 importance measures.
6 Now, there is another issue here, that I think
7 rigor in our analytical methods is important. That doesn't
8 mean that the method has to be exact, it can be an
9 approximate method, but at least we have to demonstrate that
10 we understand the limitations, all the approximations have
11 been listed clearly. And I think there are important
12 stakeholder groups out there that are usually I don't think
13 included in the term when we say stakeholders, and these are
14 the technical communities out there which have to be
15 satisfied that the methods we are using are, in fact,
16 appropriate.
17 So, let's come now to the way these categories of
18 SSCs are developed using importance measures. And these
19 importance measures most commonly used are the
20 Fussell-Vesely and risk achievement worth,
21 Several people have commented in the literature,
22 including our own staff, on the limitations of these
23 methods. One limitation that appears to be universally
24 accepted as an important one is that the SSCs are
25 categorized individually and not as groups, yet the decision
28
1 options affect groups of components. You will never decide
2 to relax the QA requirements on a specific SSC, you will
3 probably do it for a class. And we will come back to this
4 issue a little bit later.
5 These measures are global measures. In other
6 words, they are based on the totality of information that is
7 in the PRA. Now, what happens many times is that we have to
8 analyze a particular risk. The models may not be very good
9 and so on, so we are conservative when we do that, and it is
10 fine. That is what we should do when we analyze this
11 particular type of risk. However, if that is added to the
12 PRA, and then you calculate the global measure, that measure
13 is distorted by the fact that you were conservative in this
14 particular assessment. Okay. And to what degree and so on,
15 we don't know. It depends, obviously, on what we are
16 assessing.
17 But even with the absence of these anomalies, with
18 full scope, good quality PRA, there are limitations to
19 importance measures, and we listed a number of them in one
20 of our letters a few months ago.
21 Now, what I am going to do next is show that there
22 is a certain degree of arbitrariness. Again, this word, I
23 don't want it to be taken as a criticism of what is
24 happening, it is just that it is a fact that these methods
25 are evolving right now. People are coming up with different
29
1 ideas how to handle these things, and I think that clearly
2 demonstrates that we need to understand them better.
3 NUMARC 93-01 recommended, some time ago, that
4 systems, structures and components that have a risk
5 reduction worth measure, which is related to Fussell-Vesely,
6 greater than 1.05 or risk achievement worth greater than 2
7 would be risk significant.
8 Now, again, this is an integrated process. They
9 go on and tell you that you also have to look at the top 90
10 percent of minimal cut sets, gain more insights and so on,
11 so it is really unfair to just talk about the numbers.
12 Okay.
13 But then we go out to the practice, current
14 practice, and we see that people are doing different things.
15 And the next slide shows how South Texas, for example, is
16 handling these things. Now, what is important to the
17 present discussion is the righthand side column. Here we
18 see a much finer categorization. They just don't go with
19 RAW greater than 2 and Fussell-Vesely greater than 00.005.
20 For example, we see the red boxes where they will
21 apply the full quality assurance requirements, and it says
22 this is defined by a number of combinations. If RAW is
23 greater than 2 and Fussell-Vesely greater than .005, or if
24 RAW by itself is greater than a hundred, or if
25 Fussell-Vesely by itself is greater than .1. And,
30
1 similarly, we see the medium category, low and then the
2 others, there are five categories.
3 Now, if you go and look at other practices by
4 other utilities, you find that they are going also to a
5 finer categorization without necessarily using the same
6 numbers that this particular utility is using. It is not a
7 question of right or wrong here -- it is not a question of
8 right or wrong, but it does at least convince me that we
9 need to understand a little better what these things are,
10 and maybe have better insights and give guidance to people.
11 But they are certainly not doing what NUMARC 93-01
12 recommended.
13 And then in our meeting of February 4, we had
14 another licensee who came in with an entirely different
15 approach, or at least it appeared to be that way, and this
16 is Consumers Energy. They come with what they call top
17 event prevention analysis. And, of course, they claim that
18 it is better than the standard importance measures.
19 Now, what is that? Well, they are not looking at
20 the probabilities of the accident sequences and so on, they
21 are looking at the sequences themselves. And they are
22 saying, well, we define what we call prevention sets. We
23 will make sure that no accident sequence can occur by going
24 to all the accident sequences and taking two events from
25 each. And they say, we will make sure that this new set
31
1 that we develop, we will maintain appropriately, we will do
2 everything we can so that these things will not fail,
3 therefore, no accident sequence can occur, or actually the
4 probability will be very low.
5 And then they bring into the process some
6 probability evaluations, too. Of course, their
7 manipulations here are very huge. I mean one application on
8 check valves, for example, they came up with 55,000 such
9 sequences, what they call prevention sets, each one
10 consisting of several hundred events, but they have the
11 computer tools to do it, and they did it. And some of the
12 results you see in the next slide where they are also
13 showing the risk achievement worth on the vertical access
14 and the Fussell-Vesely measure on the horizontal access,
15 comparing their results to those that one would have
16 obtained by using the standard techniques of importance
17 measures.
18 Now, they claim that the major advantage of what
19 they are doing is that it addresses what I said earlier,
20 that systems, structures and components in this new method
21 now are categorized individually. You are looking at the
22 whole context of accident sequences. And there are check
23 valves, some of the check valves that you see in the lower
24 lefthand side quadrant, that become important under certain
25 conditions where other things have failed. In the standard
32
1 Fussell-Vesely and risk achievement worth approach, you are
2 looking at one component and you assume that all others have
3 their nominal failure characteristics. Whereas, now, you
4 may have other failures as well, in which case, this
5 component now may become important.
6 Is this better? We don't know. I am not going to
7 argue that it is better, I am still trying to understand it
8 myself. The staff, as far as I could tell last February, or
9 most of them anyway, it was the first time that they saw
10 this.
11 So I think this discussion on the South Texas
12 project and NUMARC and Consumers Energy clearly demonstrates
13 that the methods for categorizing the systems, structures or
14 components are still evolving.
15 So what are the recommendations then that the
16 committee has come up with? Yes, we agree that mathematical
17 methods involving, in this case, importance measures have to
18 be evaluated in the context of the integrated
19 decision-making process. There is no question about it.
20 But we believe that we also have to clearly understand the
21 limitations of each approach and make recommendations as to
22 which approach is best for what application. And, also, all
23 these limitations and so on should be provided to the expert
24 panel so that the expert panel will have a better
25 appreciation of what kind of information they are getting
33
1 from the risk assessment or from the analysts.
2 Now, if we go back to the figure, the very first
3 figure, you see I have no arrow going down to the impact on
4 CDF and LERF. The committee feels -- I mean this is the way
5 things are now, the committee feels that even when we have
6 to resort to the risk significant categories and we proceed
7 with those, it would still be very useful to try to evaluate
8 the impact on CDF and LERF of whatever decision we are
9 considering. We admit that this is not easy to do with the
10 current models. There may be a way in the future, but I
11 don't think we should just, well, we don't think that we
12 should just settle on this approach that bypasses completely
13 the assessment of the impact of the decision options on CDF
14 and LERF.
15 And on a happy note, back to you, Mr. Chairman.
16 DR. POWERS: We should allow some time for the
17 Commission to ask what questions they want. This
18 constitutes a body.
19 CHAIRMAN MESERVE: Good. Thank you very much. I
20 very much appreciate it. A very informative briefing.
21 Dr. Powers, I would like to first address a
22 question to you, and it is a rather fundamental one, I
23 think. That you had indicated, as I understood you, that
24 the ACRS, if it had its preference, would adopt a what they
25 call holistic or a clean sheet approach, which as I
34
1 understood that to mean is that we throw out all of Part 50
2 and we start all over and focus on existing reactors. And I
3 just sort of -- and it seems to me that that is disconnected
4 from every other presentation I have heard this morning,
5 which is that we have all these inadequacies of PRAs. We
6 don't understand exactly the role of defense-in-depth. We
7 are not exactly sure of the role of performance-based
8 regulation. We have these problems, George has indicated,
9 with importance measures, and there is a lot of things are
10 evolving.
11 I really wonder that is a feasible thing to do
12 given the fact that some of the underpinnings that you would
13 want to have for a truly risk-informed approach really are
14 something that are still a work in progress, and given that
15 isn't really the most practical approach, what we are doing,
16 which is what you have characterized as a piece-wise
17 approach, we learn as we are going, do what we can.
18 DR. POWERS: I think you catch us in a mode of a
19 peer reviewer. We are looking at a superb body of work that
20 exists and asked to review it and, of course, the review
21 only focuses on the bad things and neglects to say, gee,
22 what great strides have made? And they are monumental.
23 They are impressive.
24 Could one sit down and take a holistic approach?
25 I think the ACRS says yes. And that despite these
35
1 impediments, despite these questions of exactly how you
2 proceed, you could make tremendous progress. On the other
3 hand, I don't want to very critical of the staff's approach.
4 They, too, have taken an approach, one they want to pursue.
5 It seems to be feasible. I mean we, after all, have sent
6 you a letter that says, gee, this looks like a fine way to
7 proceed here.
8 I think we are interested in looking at the
9 holistic approach as a comparison to where they stand, to
10 where they go, and what kinds of things, because, in the
11 end, I think you want a body of regulations that looks like
12 you came from a holistic approach.
13 The challenges that are ahead of us can't be
14 underestimated, but I don't think any of the speakers have
15 said these are debilitating. I think that Dr. Kress
16 indicated some things that have to be done, and Professor
17 Apostolakis indicated some things where there are
18 alternatives coming before us because we are unleashing the
19 imagination of the licensee community to figure out ways to
20 do things. And we have to accommodate those different
21 approaches toward achieving the same end.
22 Well, I think I will stop there.
23 CHAIRMAN MESERVE: Let me just say, I would be
24 very concerned if we were to try to just start all over with
25 a clean sheet. I mean we have a job to do and we have a job
36
1 to do now. We have an immense project which will take an
2 enormously long to do, with all kinds of uncertainty as we
3 approach it. And I just think we, as a practical matter,
4 have no choice but to do what we are doing.
5 DR. POWERS: I can certainly be sympathetic to
6 that, but I think one has to recognize that when you proceed
7 that way you have a tendency to say, gee, I have got a
8 regulation here on offsite power for liability, as an
9 example, and I am always going to have that regulation. I
10 may put some risk words in it, but, in fact, the holistic
11 approach might find that there was no need to have that
12 there. The danger is that you will retain in the regulatory
13 body aspects of the current regulation that a holistic view
14 would say probably weren't necessary. That is the danger
15 you are facing.
16 CHAIRMAN MESERVE: Well, I appreciate it, and you
17 have pointed out there is the danger about inconsistencies
18 and so forth. And we are very conscious of that and,
19 obviously, we try to fix those as we are going forward and
20 with your help.
21 DR. POWERS: Another challenge you are going to
22 face, and this one is going to be more difficult, I think,
23 is that a dispassionate view of the risk structure, that you
24 might well say there is a need for another regulation, one
25 that there is no counterpart in the existing body of
37
1 regulations. I think it is much more difficult to inject a
2 new regulation into an existing system than it is to take
3 one out, or to preserve one. I think that is a challenge.
4 I don't know I have any good examples of that
5 right now, but, clearly, they do exist, because, I mean,
6 certainly the ATWS rule and certainly the station blackout
7 rule were products of risk information, and it would not
8 surprise me if others would come along like that. You want
9 to make sure that you don't create barriers in the
10 piece-wise approach toward injecting regulations when they
11 are necessary.
12 CHAIRMAN MESERVE: We recently received a letter
13 from the NEI. I don't know whether you have seen it, but it
14 was -- they made an effort to give us an array of the
15 priorities that they thought we should hold in terms of
16 approaching the regulatory problems and risk-informing the
17 regulations. I recognize this is somewhat outside the scope
18 of your presentations, but I am curious as to whether you
19 have looked at that letter and have any views on how we
20 should approach the prioritization.
21 DR. POWERS: I am familiar with the letter in its
22 draft from, but I will admit that was sometime -- it was
23 several months ago since I looked at it. My view on it was
24 that the prioritization was based on magnitude of licensee
25 effort, and it was less clear to me how it was tied to the
38
1 risk significance of the items. Maybe that is all I would
2 really like to say about that.
3 CHAIRMAN MESERVE: Yes, we may seek some further
4 views from you on that point.
5 Dr. Kress, I have just one question for you. On
6 your Slide 29, you point out some of the PRA inadequacies
7 and incompleteness, and you make the point, which I am sure
8 is true, that one of the difficulties is that the
9 reliability database for non-safety-related systems is weak.
10 I mean one of the aspects of our risk-informed effort was to
11 enable us to look at non-safety-related systems that turn
12 out to have high safety significance.
13 That bullet seems to suggest that we are on --
14 maybe it is too extreme to say -- but sort of a fool's
15 errand and that we are not going to be able to detect those.
16 DR. KRESS: I didn't mean to have it interpreted
17 that way. There is a database on those. We do, the plants
18 do keep records of how often those things fail and they have
19 those records. They are just not centralized in what I
20 would call the PRA community's database. They need to be
21 assessed and brought into the same level of review and
22 appreciation that the safety system and components have. So
23 it will take an effort to go out and get this data that
24 exists out there.
25 CHAIRMAN MESERVE: It does exist.
39
1 DR. KRESS: Yes. In fact, INPO has a great deal
2 of information on that. But George might want to comment on
3 this, too.
4 DR. APOSTOLAKIS: Yes. I think that it is
5 literally for passive components. For those I don't think
6 we have anything.
7 DR. KRESS: Yes, we don't have anything on
8 passive.
9 DR. APOSTOLAKIS: For the others, I agree with Dr.
10 Kress.
11 CHAIRMAN MESERVE: Okay. Let me give some of my
12 colleagues an opportunity. Commissioner Dicus.
13 COMMISSIONER DICUS: Let me follow up another
14 aspect of the Chairman's first question regarding holistic
15 approach. We are aware that some of the licensees will not
16 use a risk-informed regulation for a variety of reasons,
17 generally because it is resource-intensive to go that route
18 and they are probably not going to stay in operation long
19 enough to do it. So when you are talking about the value of
20 going to a holistic approach, you still recognize, even if
21 we did that, we have another set of regulations for that set
22 of licensees.
23 DR. POWERS: We have always -- it has always been
24 in our mind that there would be, at least for some
25 substantial period of time, two sets of regulations, the
40
1 existing ones and the risk-informed ones.
2 COMMISSIONER DICUS: Okay.
3 DR. POWERS: I mean that, quite frankly, that has
4 more to do with lawyers than it does to do with us.
5 COMMISSIONER DICUS: Understood. On the slide
6 that you had on pitfalls and barriers, the second bullet,
7 there is a comment -- incompleteness as one of the pitfalls
8 or barriers. Incompleteness -- I think Slide 6 -- and the
9 analytic capabilities to support a risk-informed regulatory
10 system. I am curious about whose analytic capabilities you
11 are referring to, the NRC's, the industry's, or both?
12 DR. POWERS: I personally have questions about
13 many of our capabilities. I think the capabilities for
14 doing risk assessments during power operations has undergone
15 the kind of technical development of good science, that is,
16 there has been intensive peer review, many discussions, many
17 papers written. Conflicting approaches have been debated
18 and we are coming down to a set of practices for doing risk
19 assessment under power operations that can be standardized,
20 that is, we can have an ASME standard in that area.
21 I think when you come to other areas, it is a
22 little more questionable. One of my current concerns is the
23 area of risk assessment from fire initiators. It is an
24 aspect of probabilistic risk assessment that has not
25 undergone much development since it was first initiated
41
1 perhaps 15 -- 17 years ago.
2 DR. APOSTOLAKIS: 1979.
3 DR. POWERS: By one of my esteemed colleagues.
4 And it hasn't had the kind of development that has been
5 accorded risk assessments for power operations. They have
6 not been as intensively debated.
7 We do know that we have phenomenological
8 difficulties in that area, particularly, sets of papers were
9 presented in a conference held by the International Atomic
10 Energy Agency, a very good list of What is wrong with the
11 methods that I use for fire risk assessment? It is a
12 confessional by the risk analysts.
13 And they have identified a number of areas where I
14 think substantial conservatisms are still built into the
15 process. And one gets very nervous about using risk
16 analysis techniques with bounding and conservative
17 phenomenological models in them. And I think we see
18 controversies developing between the staff and the licensees
19 with respect to these conservatisms.
20 That is just one example, and it is as a
21 community. It is not regulator versus licensee. This is a
22 weakness that exists in this. Our entire treatment of fire
23 has been, as I call it, the stepchild. It clearly is an
24 internal initiator, but it is already treated in the
25 external events PRAs. It has not had the kind of
42
1 phenomenological research that has been done for severe
2 reactor accidents, for instance, or aerosol transport. We
3 know a lot about radioactive aerosol transport. We don't
4 know so much about smoke transport.
5 So, my concern in this bullet is as a community
6 and not an individual. Now, there are probably other
7 individual areas where I think we are going to have to look
8 closely and say, what kind of technical support is it
9 necessary for the operational arms of the NRC to have
10 available to them?
11 And that brings up an issue that I think you are
12 going to have to confront on a policy basis, and that is, on
13 these technical areas, where is it you want the NRC staff to
14 do independent assessments? And where is that you think it
15 is satisfactory for them to review the submission of
16 licensees? That will dictate what kinds of technical
17 capabilities and tools, analysis tools they have to have
18 once we have a good understanding of that. Right now there
19 are no criteria.
20 COMMISSIONER DICUS: Okay. One final quick
21 question, it goes to your comment or this so-called wish
22 list that the industry has sent us of where to start. You
23 made the comment that you didn't think it was particularly
24 based on risk, but rather on effort. Can you expand a bit
25 on that?
43
1 DR. POWERS: I may speak out of poor memory, but
2 my recollection was that I think the list reflected careful
3 attention to the areas where the industry thought an
4 enormous expenditure of effort was taking place, perhaps
5 with little risk significance. That is my memory. And,
6 quite frankly, it has been long enough that I could be in
7 some error.
8 COMMISSIONER DICUS: Okay.
9 DR. POWERS: But that is my memory.
10 COMMISSIONER DICUS: Okay. Thank you, Mr.
11 Chairman.
12 CHAIRMAN MESERVE: Commissioner Diaz.
13 DR. POWERS: Of course, he is the godfather of the
14 risk-informing of Part 50.
15 COMMISSIONER DIAZ: Not really, I refuse to have
16 that title. It has all kinds of bad connotations.
17 [Laughter.]
18 DR. POWERS: I am sorry, sir.
19 COMMISSIONER DICUS: He was willing to do it
20 himself, take a year off.
21 COMMISSIONER DIAZ: I would love to do that. I
22 might still do that.
23 COMMISSIONER MERRIFIELD: Of course, Commissioner,
24 you always know on the Hill we always got nervous when
25 people complimented us that much.
44
1 COMMISSIONER DIAZ: I am very worried. You know,
2 my sensitivity has been raised.
3 Let me start by saying that I have been
4 three-and-a-half years and I would really like to compliment
5 the ACRS for a very clear presentation. I think what you
6 have done today is, in a very simple manner, expressed what
7 are the issues that need to be faced. In fact, I was even
8 able to understand Professor Apostolakis.
9 [Laughter.]
10 COMMISSIONER DIAZ: Which, at my present reduced
11 brain power due to the flue, it is a credit to the way that
12 he expressed things.
13 COMMISSIONER DICUS: Slide Number 1.
14 COMMISSIONER DIAZ: Right. The first thing that I
15 come out of this is that I have so many questions that I
16 believe are important that I would publicly tell you that I
17 would like to get a re-engagement with you in a little room,
18 because I have, practically on every point I have something,
19 and I don't think this is the right place to do it. But I
20 am going to take a couple of cracks at a couple of issues,
21 including the holistic approach, of course.
22 First, you know, let me go to one of the first
23 statements of Dr. Powers, which I think addresses on of the
24 things that we are really having to grapple with. It is,
25 you know, the second part of the presentation is going to be
45
1 more applied and less theoretical, and that might go to the
2 heart of some of the problems that we are having. There is
3 a very applied feeling here that needs to have a high degree
4 of acceptability to both the licensees and the NRC to be
5 able to progress into the areas which are more theoretical.
6 It is a reinforcing function, and we all need to realize how
7 these two things interact.
8 I have seen in the last almost two years, from the
9 first time that NEI came and said let's go ahead and change
10 Appendix A, and they have all this book, to the last letter,
11 kind of a reduction of the approach. And that reduction of
12 the approach comes up from the human interactions. You
13 know, if somebody says, I cannot do this, then the other guy
14 says, well, I think you can, but let me take the best
15 position on it. And that is the main advantage of not doing
16 it holistically, but taking a holistic approach to it.
17 And it brings out the difference between
18 risk-informed regulation and deterministic regulation. If
19 you bring a body of regulations to become risk-informed, you
20 are not limiting to the present state of the art. You are
21 actually embodying into that set of regulations the
22 capability to improve as things are improved. They are no
23 longer set values, but it is the capability to analyze, you
24 know, and implement measures to reduce risk. And that is
25 the real value of risk-informing our regulations, is that it
46
1 is not static, that it is dynamic in itself. That it is not
2 constrained, that it doesn't put you into a corner. That it
3 frees you to do what is best as things are developed. And
4 that is really where the things are. That was my first
5 question.
6 COMMISSIONER McGAFFIGAN: That sounded like a
7 statement.
8 COMMISSIONER DIAZ: That was a statement, I didn't
9 realize.
10 [Laughter.]
11 DR. POWERS: But I think you raise a good point
12 and something that maybe speaks to the Chairman's question,
13 because it may well be that we can proceed along a step-wise
14 -- I think my wording "piece-wise" has too much of a
15 pejorative nature to it -- step-wise process until we grow
16 comfortable with what we are doing, and then it is possible
17 to move to a more holistic step. And I think you can have
18 the best of both worlds there.
19 COMMISSIONER DIAZ: I think you are absolutely
20 right, if we gain confidence with it. But if we abandon
21 from the beginning the idea that there could be a holistic
22 approach that can come to be effective at any one time, then
23 we are already reducing our capabilities. Do you have any
24 comments on that?
25 DR. POWERS: I think that is why it was worthwhile
47
1 for me to try to go back and distill out and do this
2 exercise of, what is it we would have done if we had
3 undertaken this ourselves? Now, understand, the ACRS is
4 four square behind the staff's three option approach, and I
5 understand the Commission is as well. But if we tried to do
6 it, I think Commissioner Diaz is absolutely right, we need
7 to think what is the capability that we want to have
8 eventually and not lose sight of that as we go through
9 looking at 50.44 and then 50.46, and then 50.48.
10 DR. APOSTOLAKIS: That's correct.
11 DR. POWERS: It is very easy to get into a trap
12 that you lose sight of what you are trying to achieve as you
13 try to work these, oh, so frustrating communications between
14 one aspect of the regulation and the other.
15 COMMISSIONER DIAZ: There is one aspect of the
16 holistic versus the step-wise approach, which is addressing
17 really those parts of the regulations that are the crux,
18 that really have connections to most everything else. And I
19 think you highlighted very clearly Appendix A and B. I
20 think that somehow we are concerned that when we address
21 these two major fundamental safety components, okay, of our
22 regulatory body, that we might be going too far or too fast.
23 I had a document that came from Europe, it was a
24 fascinating document. Some people independently analyzed
25 Part 50, and they concluded that the two most relevant and
48
1 fundamental components of Part 50 were Appendix A and
2 Appendix B.
3 DR. POWERS: If you undertake to read Part 50, you
4 are well advised to read Appendix A first, and then, as you
5 go through Part 50, you have a better understanding of why
6 the other what I would call technical elements are in there.
7 What that translates into is if you change those technical
8 elements, you haven't gained anything. You are still
9 constrained by the general design criteria because they
10 speak exactly to the same issue that is spoken to in
11 regulatory report.
12 For instance, the staff is very interested in
13 50.44, but there is a general design criteria that asks for
14 exactly the same thing. And you get into what I call the
15 graded quality assurance problem. Yes, I have graded, I
16 have put risk into this, but I haven't done anything to the
17 licensee because he says I am still controlled over here.
18 COMMISSIONER DIAZ: I sincerely believe that
19 somehow leaving Appendix A as an incomplete piece of work
20 has done a disservice to this body. If it had been, you
21 know, at certain time increments, really brought up to date,
22 we might have really got something that we would not be in
23 this dilemma.
24 Let me -- this is, you know, I say there is a
25 limited amount of time, I am looking at my clock in here.
49
1 Let me go back to the issue of acceptability. The issue of
2 putting resources where it should be, and how can we do the
3 most good.
4 I am convinced that there is a tremendous body of
5 work that needs to be done to increase the reliability of
6 the tools that we use. I think sometimes, theoretically, we
7 overemphasize the issue of uncertainty reduction versus
8 reliability of the data, and that, Dr. Apostolakis and I
9 will need to get into a dark room and has that out.
10 But if you look at your Slide 29, this issue of
11 acceptability versus what is really, you know, important and
12 relevant in the short-term probably comes to mind. You look
13 at the things you correctly address as being inadequate, you
14 know, fire, human performance, organizational and safety
15 culture factors. And then you look at your second bullet,
16 which is probably the only thing that I very much disagree
17 with is, you know, making low-power and shutdown a front
18 runner.
19 And the reason that I don't agree with that is not
20 because I don't believe that we should not quantify it. I
21 agree that we should. It is that reducing the uncertainty
22 in that area, when we have uncertainties that are in the
23 other areas that, to me, are more important fundamentally,
24 you know, shutting down for full power and coming down to
25 low-power, which is where I think the real high risk is.
50
1 Then we start having this, you know, what comes
2 first? What do we do first? And I think that the
3 identification of where risk are, rather than the
4 quantification of the uncertainty in the calculation, is far
5 more important in this area than anything else. Would you
6 care to comment on that?
7 DR. APOSTOLAKIS: Did you just say, Commissioner,
8 that you think that the transition risk is very important,
9 is that what you said?
10 COMMISSIONER DIAZ: I think that identification of
11 where the transition risk is, and approximately how much it
12 is, is extremely important rather than trying to reduce the
13 uncertainty in the calculation of how much it is.
14 DR. APOSTOLAKIS: Oh, yes.
15 DR. POWERS: I think there is a great community of
16 agreement here. We are in violent agreement, sir.
17 COMMISSIONER DIAZ: Okay. That is the case then.
18 DR. POWERS: I think that there is technical
19 support for your point of view. That when we look at the
20 risk assessment done for Sizewell, we find that these
21 transition risks play a fairly important role. When we look
22 at the risks during full shutdown, where we may be working
23 with the vessel open, the containment open, that we
24 understand the difficulties the risk assessment tools have
25 in confronting unproceduralized actions, which would be so
51
1 easy to do under those considerations. So we question their
2 accuracy there.
3 We question the accuracy of not focusing on these
4 transition states because they look like there is potential
5 for not only human error but equipment failure, especially
6 as the plants get older. So I think there is a raging
7 agreement here on that.
8 Now, on the other hand, let me assure you that it
9 is possible to overemphasize uncertainties, because
10 uncertainties here are things that physicists and engineers
11 are unfamiliar with, uncertainties that are equal to, and
12 sometimes larger than the magnitude of the quantity in
13 question. But the question is now how big the uncertainties
14 are, but how do they affect the decision-making process?
15 And I can assure you that that is a lesson that I am
16 reminded of regularly by Professor Apostolakis. When I make
17 errors in that, he insists that I come up and take his
18 probability course.
19 COMMISSIONER DIAZ: All right. Thank you very
20 much.
21 CHAIRMAN MESERVE: Commissioner McGaffigan.
22 COMMISSIONER McGAFFIGAN: I guess I am going to
23 start off by trying to figure out whether I did under Dr.
24 Apostolakis. The Consumers Energy slide that you used --
25 DR. APOSTOLAKIS: Which one?
52
1 COMMISSIONER McGAFFIGAN: The Consumers Energy
2 presentation to the ACRS.
3 DR. APOSTOLAKIS: Okay. Yes.
4 COMMISSIONER McGAFFIGAN: Are the ones, twos,
5 fours next to the boxes what their importance measure is and
6 they are comparing it in this chart to the normal table of
7 importance measures? In other words, is Number 1 down in
8 the lower right quadrant what they think is the most risk
9 significant based on their importance measures, yet it is
10 showing up in a region that would not be treated by South
11 Texas as important, or it would be medium important in South
12 Texas?
13 DR. APOSTOLAKIS: Yes. The intent of this was to
14 show what kind of results one would get by applying the --
15 COMMISSIONER McGAFFIGAN: Their importance
16 measures compared to the others.
17 DR. APOSTOLAKIS: Yes. Right.
18 COMMISSIONER McGAFFIGAN: So what the slide leads
19 me -- I mean I see 1 down in the lower right quadrant, I see
20 44, 45 up in this area where you know, you would apparently
21 think things are important, have high quality assurance. I
22 see low numbers, 11, 10, 19, down in the lower left area
23 where, you know, you would want a low -- you would want a
24 basic quality assurance program or whatever.
25 So my question that this raises, are we ready --
53
1 is 50.69, which I know, you know, Dr. Powers keeps saying,
2 you know, you endorse the staff approach, but are we really
3 ready for 50.69 or what is -- you know, if there is this
4 much variation in the ability to quantify it, you know, and
5 depending on the methodology used, you end up with something
6 either being important and needing high quality quality
7 assurance or something not, have you done any sensitivity
8 analysis to how often this is going to occur?
9 DR. APOSTOLAKIS: No, people have not done this.
10 This is the first time that we ourselves saw such a
11 comparison. But there is an important point here, though.
12 When I first saw this methodology, I thought it was
13 drastically different, dramatically different from what
14 other people are doing. The more I think about it, the more
15 I think that it is closer to what other people are doing.
16 For example, if you were to calculate the risk
17 achievement worth in Fussell-Vesely for all the check
18 valves, not just the ones above the limits that NUMARC has
19 given us, you would have identified these other things down
20 here. And South Texas will tell you, we will apply good
21 engineering practice to those.
22 COMMISSIONER McGAFFIGAN: Okay.
23 DR. APOSTOLAKIS: So the question in my own mind
24 is whether this is really a truly different approach.
25 COMMISSIONER McGAFFIGAN: Okay.
54
1 DR. APOSTOLAKIS: So, all I said was that things
2 are evolving.
3 COMMISSIONER McGAFFIGAN: Just the bare slide by
4 itself, compared to the previous slides, starts raising
5 questions.
6 DR. APOSTOLAKIS: Exactly.
7 COMMISSIONER McGAFFIGAN: Dr. Kress, I guess it
8 was -- on my slides it says Slide 30, but it is the slide
9 entitled "Need for Risk-Acceptance Criteria." You say in
10 that slide that the limits would differ from those in Reg.
11 Guide 1.174 for adequate protection. How would they differ?
12 I saw the same in your letter, they would differ. Would
13 they be higher, would it be 10 to the minus 3 CDF, as
14 opposed to the 10 to the minus 4? Where would -- give me a
15 guess.
16 DR. KRESS: I can give you some speculation.
17 1.174 really has only one set in there, that is the 10 to
18 the minus 4 and 10 to the minus 5. They don't have numbers
19 for an upper boundary.
20 COMMISSIONER McGAFFIGAN: Right.
21 DR. KRESS: If I look at the IPE results and the
22 IPEEE results, and look at what has been achieved by the
23 various plants for CDF and LERF, and if I make allowances
24 for the fact that the IPEs and the IPEEEs reflect things
25 that the licensees do that are not actually required by the
55
1 regulations, they go beyond the regulations, so that this
2 tends to lower their numbers. But if I make allowances for
3 that, I would have guessed that if I were to build a plant
4 just to meet the regulations and do none of the other
5 enhancements, just to build the regulations, I could build a
6 plant and be at about a level of a factor of 10 above the
7 CDF and LERF that is in the 1.174.
8 And, you know, I don't know exactly where you
9 would call an adequate protection level in terms of those
10 two, but it is not at the level of the safety goals.
11 COMMISSIONER McGAFFIGAN: Dr. Apostolakis wants to
12 comment.
13 DR. APOSTOLAKIS: I think an important point here
14 is also that instead of just talking about what is adequate
15 protection, to actually see how people act. And in my
16 experience, the staff and the industry act immediately when
17 they identify a contributor that is on the order of 10 to
18 the minus 3 or higher to core damage. So where exactly is
19 the line, we don't know. But, you know, the famous words
20 "increased management attention" that they use in 1.174,
21 well, if you want to see increased management attention,
22 tell them you found a contributor of 3 -- 10 times to the
23 minus.
24 COMMISSIONER McGAFFIGAN: Well, I don't want to --
25 again, maybe I will follow on in private like Dr. Diaz on
56
1 some of these issues. The whole notion of setting a safety
2 goal in terms of CDF and LERF, I will go back to Dr. Kress,
3 we have talked repeatedly in the past about delta CDFs and
4 delta LERFs having some meaning, but the absolute
5 quantities, given all the problems with PRAs and whatever
6 that you talk about not being that useful, so if we ever did
7 go and establish whatever the number is as the upper bound,
8 10 to the minus 3, 2 times 10 to the minus 3, whatever it
9 is, would we have -- would it be useful? I mean do we
10 really believe the sum total numbers that come out of these
11 PRAs? And would it be --
12 DR. POWERS: It would be pretty hard to calculate.
13 COMMISSIONER McGAFFIGAN: Yes, it would be pretty
14 darn had to calculate given, you know, that you say it
15 doesn't cover FAR, it doesn't human performance, it doesn't
16 do this, do that.
17 DR. KRESS: I, personally, believe that
18 establishing such numbers would be extremely useful in
19 dealing with crafting the regulations in such a way that you
20 have the coherence we need.
21 I do believe that that bright lines like a
22 specific number are hard to deal with, and one has to
23 incorporate uncertainties and there have to be fuzzy lines.
24 And you can't get away from defense-in-depth and true
25 regulatory judgment. I think those things are important.
57
1 So I think having a number which represents a value you
2 would like to achieve if you had perfect PRAs, or if you had
3 perfectly quantified uncertainties, so that you know what
4 the uncertainty is in the number, is of value. And how you
5 deal with it in the regulatory process is, I think,
6 something else.
7 COMMISSIONER McGAFFIGAN: It would be more a
8 hortatory statement than something that we would then try to
9 mathematically reflect in the regulation.
10 DR. KRESS: I think in practice it would end up
11 being something you would actually act on, the actual
12 numbers. You would actually act on them.
13 COMMISSIONER McGAFFIGAN: You would act on, if
14 anybody were above 10 to the minus 3, if there were a
15 number, we would do something about it.
16 DR. KRESS: You would do something, yes, Exactly.
17 COMMISSIONER McGAFFIGAN: But Dr. Apostolakis'
18 intervention was we would so something if they are above 10
19 to the minus 4, in practice.
20 DR. APOSTOLAKIS: Three.
21 DR. KRESS: Well, I think above 10 to the minus 4,
22 you use a lot more regulatory judgment, and you know, things
23 like --
24 COMMISSIONER McGAFFIGAN: Right. Okay.
25 DR. POWERS: Quite frankly, we have plants that
58
1 are above 10 to the minus 4 and they are fine.
2 COMMISSIONER McGAFFIGAN: Right.
3 DR. APOSTOLAKIS: They are licensed.
4 COMMISSIONER McGAFFIGAN: Just to now consume too
5 much time, I will go back to Dr. Powers. Your chart Number
6 11 talks about DOE sponsoring an effort on risk-informing
7 the reactor regulations following a top-down process. I
8 look at that and I wonder whether that will have any
9 credibility at all, and whether it is at all connected with
10 us. I mean you have a promotional agency sort of saying,
11 you know, I don't know what the product of this effort is,
12 but they say, you know, Dear NRC, here are the regulations
13 we respectfully request you think about applying to a future
14 generation of reactors. Sincerely, Bill Richardson, or
15 something.
16 DR. POWERS: I think you would have to ask the
17 Department of Energy what they intend to do with it. We are
18 simply aware of the effort being undertaken, and that they
19 profess that they are going to use this top-down holistic
20 type process. And I think it is interesting.
21 COMMISSIONER McGAFFIGAN: It is interesting, but
22 shouldn't -- I mean my notion is, if we are going to think
23 about future reactor regulation and having effort in that, I
24 know DOE has money and we don't, but it strikes me that if
25 somebody were funding this in a way that would have
59
1 credibility in the long run, it would be better for us to be
2 doing it than them.
3 DR. APOSTOLAKIS: I have a comment about it. This
4 is not a Department of Energy effort. It is sponsored by
5 the Department of Energy, but it is really part of the NERI
6 program, the Nuclear Energy Research Initiative. You know,
7 a group of organizations submitted a proposal, it was
8 approved. That does mean that this is DOE's position or
9 will be DOE's position. It is just a research project, so
10 let's not give it more importance than it has. And I doubt
11 very much you will get a letter from the secretary.
12 COMMISSIONER McGAFFIGAN: Okay. Thank you.
13 CHAIRMAN MESERVE: Commissioner Merrifield.
14 COMMISSIONER MERRIFIELD: I would like to join
15 Commissioner Diaz in completing the presentation. I thought
16 -- like he said, I think it was very clear and certainty
17 very helpful so far.
18 I am struck, and I think all of my fellow
19 Commissioners have talked going to the issue of Part 50 and
20 whether we went with a holistic approach, just took a blank
21 sheet, or whether we went along what we are not calling the
22 step-wise fashion, which we have decided to undertake.
23 And I was reminded of an analogy, and that was of
24 the difference between an artist and a house painter. And
25 it strikes me, and, again, I don't mean this in a pejorative
60
1 sense either, it strikes me that the ACRS has the ability to
2 step back and think big picture, out of the box, in a
3 theoretical way, and present to the Commission some of the
4 possibilities that are available to us. And the Commission,
5 like the house painter, has to work with what we have. And
6 what that is are limited budgets, limited staff resources, a
7 need to response to stakeholder concerns, a need to respond
8 to the concerns of Congress that we move forward and
9 expeditiously to reform the way in which we do our
10 regulations, to improve the safety, but at the same time
11 reduce unnecessary burden.
12 And so this conversation very much I think falls
13 in line with that. That if we had the luxury of time and
14 resources, certainly, doing this in a holistic manner and
15 moving forward in that way, from a blank sheet, would
16 probably be a great outcome. But, given what we have on our
17 plate, that may not be possible for us, and, indeed, I think
18 that is why the approach that we have taken makes sense.
19 To underscore this and to package it, I will
20 repeat, and I think I have gotten this right, the last words
21 that Dr. Powers said on this, ACRS is four square behind the
22 staff's approach to Part 50. And I think that is certainly
23 where I would want to leave that particular comment.
24 DR. POWERS: And it is absolutely true. We have
25 just had a briefing from the staff working on the Option 3.
61
1 They probably can attest to you they got a health and
2 in-depth interrogation. But I think there is a genuine
3 enthusiasm for what they are undertaking. I think there is
4 even greater enthusiasm for the special efforts under Option
5 2.
6 COMMISSIONER MERRIFIELD: In that regard, let me
7 ask you just a couple of quick questions. They relate to
8 Slide 6. You mentioned the incompleteness in our analytical
9 capabilities to support a risk-informed regulatory system.
10 How would you characterize the staff's response to this
11 assertion? And what do you think if being done to make
12 these capabilities complete?
13 DR. POWERS: I believe that the way to assess the
14 staff's response toward the assertions of incompleteness is
15 to look in two places. What have they done on the PRA
16 implementation plan? And what have they done in their
17 research programs? And I guess we get mixed messages there,
18 that we see a PRA implementation plan that is fairly
19 anachronistic. Its major elements were written before
20 Commissioner Diaz gave his sermon on the second floor of the
21 White Flint Building, and it doesn't have laid out for it
22 yet the kinds of analytic tool development that may be
23 identified as they go through the step-wise process.
24 In the research programs we see elements that I
25 think speak to many of the current deficiencies. Certainly
62
1 the research program includes a human performance program
2 plan and I don't know whether the Commission recalls, but
3 after many discussions with the ACRS, the staff brought
4 forth a plan in the human performance program that the poor
5 speaker was embarrassed by the Committee standing up
6 applauding virtually as they went through that, those plans.
7 Similarly, in the area of incorporating digital
8 I&C;, the staff is coming forward with plans in that area.
9 The staff has formulated a plan for looking at shutdown
10 risk. Staff has some efforts underway in fire protection
11 and the development of the risk tools in that area.
12 What we don't see is a coherency in these
13 activities that say, and here is how good it has to be, here
14 is what we want to accomplish. Here is what we are going to
15 do with this. Is this is a tool that is used by
16 researchers? Is this a tool used by the NRR, at
17 headquarters, or is this a tool that we want in common use
18 by the line organizations out meeting directly with the
19 licensees? That is the part we see missing right now when
20 we look at these, at the development of these programs.
21 But I believe, correct me if I am wrong, that
22 there are elements of the research program, as at least it
23 has been proposed, that address every one of the
24 deficiencies that we have called out here today, and have
25 addressed the issue of uncertainty analysis.
63
1 COMMISSIONER MERRIFIELD: Just to follow up, do
2 you think it is understandable, given the fact that we are
3 still in the early stages of this effort, that that would --
4 it is not unexpected that there would be that particular
5 difficulty? I mean I am not trying to rationalize it, but
6 it seemed to me that that is not to be unexpected.
7 DR. POWERS: I guess I don't want to speculate
8 right now. I think there is distinction drawn between the
9 research organization and the line organizations as far as
10 their familiarity with these things.
11 The other problem, of course, is that we have got
12 limited resources and we have to put it into the most
13 important areas, and some of these deficiencies just are not
14 going to get addressed when you have a limitation of
15 resources.
16 COMMISSIONER McGAFFIGAN: Okay. Let me keep going
17 quickly, because I don't want to take up too much time. Dr.
18 Kress, on Slide 32, you had two issues you had raised at the
19 bottom. Will the standards include guidance on the
20 appropriate determination of uncertainties? And does the
21 NRC plan to develop guidance on how to consistently use
22 uncertainties in the decision-making process.
23 Obviously you talked a bit about the ANS effort
24 that is currently underway. Do you have the sense that
25 staff understands these two issues and that they are
64
1 addressing them?
2 DR. KRESS: Yes, I think the staff thoroughly
3 understands these two issues. Now, I don't see much effort
4 in this guidance on how to consistently incorporate
5 uncertainties in the decision-making process, but they
6 understand it has to be done. There is some vagueness about
7 how they intend to use uncertainties. But they are aware of
8 the issue.
9 COMMISSIONER MERRIFIELD: A final question for Dr.
10 Apostolakis. On Slide 39, I was wondering if you could help
11 me better understand the context of your third and fourth
12 observations. Now, given your presentation, I draw from the
13 discussion that the limitations and arbitrariness you talk
14 about, there is some degree of inevitability to that. What
15 I would better like to understand is what does this
16 inevitability mean in terms of our ability to risk-inform
17 Part 50?
18 DR. APOSTOLAKIS: I don't think that this a major
19 roadblock. I think so, I think all we have to do is
20 identify the limitations that we recommend later be
21 identified and understand better why there is this apparent
22 arbitrariness in the application of the methods, and write
23 the Regulatory Guides appropriately. And, frankly, the
24 Consumers Energy people are very anxious to see something in
25 the guide about their approach, I mean they have said so in
65
1 public to us. So I don't think this is a major --
2 COMMISSIONER MERRIFIELD: So you have confidence
3 in staff's ability?
4 DR. APOSTOLAKIS: Oh, I think, yes. Yes.
5 COMMISSIONER MERRIFIELD: Great. Thank you.
6 DR. APOSTOLAKIS: No problem at all.
7 COMMISSIONER MERRIFIELD: Mr. Chairman.
8 CHAIRMAN MESERVE: Thank you very much. I very
9 much appreciate what was really a very helpful presentation
10 on, obviously, an enormously important initiative for us and
11 for our licensees.
12 I would like to suggest that we now turn to the
13 final presentation having to do with performance indicators.
14 DR. POWERS: We will go from the theoretical to
15 the applied very quickly here. And Mr. Barton will walk us
16 through this area, which I can assure you is foremost on our
17 plates right now, and I remind you that that this is still a
18 work in progress for us.
19 MR. BARTON: Thank you, Dana. It is good to be
20 back in the real world.
21 [Laughter.]
22 MR. BARTON: The committee received the other day
23 the SECY paper 049, which we have had a chance to look at in
24 a cursory matter. We will be meeting with the staff this
25 afternoon to discuss the details of that paper. But a
66
1 cursory review of the document shows that several of the
2 committee's questions and concerns that we have had on this
3 process are being addressed by the staff. For example, the
4 initial implementation, rolling out for one year, continuing
5 to adjust the process during that year and doing a
6 self-assessment at the end of the year process, at the end
7 of that initial implementation process does answer several
8 concerns that we have had. Also, the handling of adverse
9 trends indicated by the substantial cross-cutting issues is
10 something that we were concerned about and see that the
11 staff is addressing that in the SECY.
12 Just to review the overall objectives of the
13 process, the process was intended to improve the
14 objectivity, improve scrutability, and to risk-inform the
15 regulatory process so that resources are focused on aspects
16 of performance that are important to safe operations.
17 On the next slide there are areas that the ACRS is
18 in full agreement with. In principle, the new inspection
19 assessment approach is better than the process it replaces.
20 I think we agree that the new oversight process makes
21 assessments and actions more objective, understandable,
22 predictable to both public and the industry.
23 The objective of the process is to assure the
24 plant performance is at an acceptable level. We have had
25 numerous discussions with the staff on the objectives that
67
1 the agency desires from this new process, and I think we
2 feel if the agency is satisfied with the overall objectives
3 that the staff has laid out, then the process that they have
4 put in place will meet those objectives.
5 However, we must recognize that there are some
6 potential downsides. One of them is the possibility of
7 losing an early warning signal that something is amiss with
8 licensee's performance, especially if one concentrates on
9 just the performance indicators.
10 Less regulatory burden could lead to bad
11 decision-making. And we don't see that there is incentives
12 for licensees to continue to improve performance. Now, the
13 SALP process, for all the faults it had, did present that
14 challenge. If you look at the new process, and the results
15 of the pilot program and the indicators that the licensees
16 have submitted for 1999, they are essentially all green, and
17 it is difficult to cross the threshold from green to white
18 to yellow.
19 The new process consists of performance indicators
20 and baseline inspections performed by the NRC. I think we
21 are in agreement that the glue that holds together this new
22 process is the inspection program. The residents must feel
23 comfortable with the inspection program and with using the
24 significance determination process. We feel they must be
25 provided with the proper resources to adequately perform the
68
1 inspection program and a look at the SDP has the potential
2 to bog down inspectors and take away from inspection time.
3 I mean these are the some of the concerns that the new
4 process appears to have.
5 The next slide. A pilot program should have been
6 longer. We addressed that, I am not going to spend more
7 time on that topic.
8 Performance indicators and their thresholds should
9 recognize plant- or design-specific characteristics. And
10 the current PIs, as we understand, don't seem to accomplish
11 this, and they weren't designed to do so. But without some
12 of these factors in the PIs, we question how much value the
13 PIs are going to have to the staff in the new process.
14 Performance indicators focus on equipment and only
15 indirectly reflect human performance and shutdown
16 operations. Some plant risks in certain shutdown
17 configurations is as high as during operating periods. We
18 think the staff needs to develop PIs for shutdown
19 conditions.
20 The staff should also continue to seek additional
21 indicators and review existing indicators for threshold
22 adjustments. And I think you need to really reflect on
23 where the thresholds are and make appropriate adjustments.
24 There is no demonstration of safety equivalence
25 for thresholds of different performing indicators. For
69
1 example, it is hard to figure out if you have the same
2 significance in white/yellow and emergency preparedness
3 area, as opposed to a white or yellow in initiating events
4 cornerstone.
5 Now, the next two slides cover areas of continuing
6 discussion both amongst members of the ACRS and with the
7 staff. Current PIs, the values are not plant-specific and,
8 thus, may be too high for some plants and too low for
9 others. We feel you are unable to identify trends in a
10 timely manner and values are disincentives to improve plant
11 performance and degraded -- degradation in performance can
12 be rapid and really not picked up when one focuses on the
13 PIs.
14 The values we feel that establish the PIs are
15 basically where the industry is operating and has operated
16 in the past. And the industry is really monitoring
17 performance at a much lower level than the thresholds that
18 are depicted in the PIs in the current process.
19 I can give you a recent observation where the PIs
20 were submitted for 1999 for a licensee that had all green
21 PIs and one white PI in security, which since turned to a
22 green since we changed the threshold in security since staff
23 has changed that. However, this licensee is monitoring its
24 performance against all the PIs on a monthly basis. In the
25 January 2000 PIs for the licensee there are two yellows.
70
1 When entering discussion and talking to the
2 licensee about, how can you have all green for '99, and all
3 of a sudden in one month this year, you have two yellows?
4 Well, the answer is we are really monitoring our performance
5 at a threshold much lower than the PIs and intend never to
6 show performance other than green by the way we are
7 monitoring performance. So is that bad? No, we don't think
8 so. But it doesn't seem to do anything to help the
9 assessment process or there is no incentive to improve plant
10 performance, and that speaks of the need for different
11 thresholds or plant-specific type indicators. Also, you
12 worry about complacency setting in when it seems that
13 performance is going to be green on all indicators.
14 The values that were chosen were arbitrary chosen,
15 95 percentile. We understand why the staff chose that. And
16 one of the suggestions by an ACRS member, and something that
17 is still under discussion is to use values based on grouping
18 of plants or individual plants.
19 The selection of performance indicators, based on
20 data that licensees were wiling to provide. No clear
21 correlation or interrelationship between performance
22 indicators and the baseline inspection program. I don't
23 think you will find any PIs that are driving inspectors in a
24 certain area.
25 Types of performance indicators such as human
71
1 performance are missing. I think human performance is just
2 an example of other indicators the staff may want to
3 consider as we roll out this process over the next year.
4 And some of these have been discussed and discounted, but
5 maybe we need to look at them again.
6 Backlogs, which also could be an indication of a
7 leading indicator, since all the indicators now are really
8 not -- we don't see any of them as leading indicators.
9 Industrial safety, reactivity events, safety
10 system actuations, and we have had discussions with the
11 staff on that, but that could be a leading indicator also.
12 So I think there is an opportunity to look at additional
13 indicators over this next period.
14 Performance indicators, not leading, we just
15 talked about that, and some members believe that we should
16 have some leading indicators in this process.
17 The next slide, the next two slides classify
18 examples of questionable indicators. These are -- we could
19 have taken others as examples also. And the reason I just
20 picked these is, if you look at the barrier integrity
21 cornerstone, many licensees are looking at performance and
22 monitoring performance much lower and they have
23 administrative limits that require them to take action even
24 before you trip the threshold in the current PI. And, also,
25 we may want to consider what may be more meaningful in this
72
1 cornerstone, is to look at unidentified reactor coolant
2 leakage as opposed to identify it.
3 An additional performance indicator for low-power
4 and shutdown operations should also be considered. We
5 believe the staff is looking at this and we will be
6 discussing that with them later this week, or this afternoon
7 I think it is.
8 Now, the next slide, emergency preparedness
9 cornerstone. Licensees only get one chance to make a proper
10 classification notification of a protection action
11 recommendation in a real event. And if you look at that,
12 the performance in the current process, and you look at what
13 occurred during the pilot program, you will find that there
14 misclassifications -- now, these are during drills and
15 practice, but yet there were misclassifications of events
16 and examples of untimely notifications, but yet the
17 indicator is green. So that is one to say, well, you only
18 get one chance in a real event, but if you can have three or
19 four hits against that indicator and still be green, one
20 questions, you know, is the indicator -- is the threshold
21 right, is the indicator tough enough to ensure that
22 licensees are maintaining excellent performance?
23 And, in closing, let me say that the committee I
24 think is in agreement with proceeding with the initial
25 implementation of the process as described in the SECY, and
73
1 ACRS individual members have strong feelings about
2 performance indicators because the PIs need to be a key part
3 of the new process. We currently have a diverse opinion on
4 what the final PIs should look like, and we are continuing
5 our deliberations on PIs with the staff. We recognize the
6 importance of having a proper set of PIs and that the
7 thresholds be meaningful.
8 CHAIRMAN MESERVE: Good. Thank you very much.
9 Perhaps we will allow some questions.
10 MR. BARTON: Sure.
11 CHAIRMAN MESERVE: Let me just make initially a
12 comment and emphasize a point that you had mentioned, is
13 that it is very clear that this program is in its infancy.
14 As you know from the SECY paper, there are a variety of
15 issues that we are continuing the address. The nature of
16 the performance indicators are subject to change. They are
17 only one component of the program. Of course, baseline
18 inspections are obviously important as well.
19 So this is very much a work in progress now, and I
20 think that you have indicated you are going to continue your
21 deliberations on it, and that would be very welcome, because
22 I am sure what we start off with, our initial implementation
23 is something that we are dedicated to continue to monitor
24 and to change as necessary, so that all of these issues that
25 you have raised are ones that are very much on our minds.
74
1 So that I don't want to have anyone in the audience in
2 particular to think that that oversight program is cast in
3 concrete and that your comments are not issues with which
4 the Commission is very much concerned and intends to address
5 as we move forward.
6 DR. POWERS: I think it is our understanding,
7 based on some preliminary information or information we just
8 got, that, in fact, the implementation is a little different
9 that it is an implementation in experiment -- in a
10 continuing experiment more than a cut-and-dried thing, and I
11 think that is something that we have always been concerned
12 about the duration of the piloting effort and the fact that
13 it didn't go through a complete fueling cycle. So
14 implementation in the form of an ongoing experiment seems
15 much more comfortable to us.
16 CHAIRMAN MESERVE: I would like to just ask one
17 question of you. You made the comment that with the
18 performance indicators that we are losing an early warning
19 signal and that there are no incentives for a licensee to
20 continue to improve performance. I think that is a question
21 of thresholds as much as anything else.
22 MR. BARTON: Exactly, that is the issue.
23 CHAIRMAN MESERVE: It is not so much that the
24 performance indicators are flawed.
25 MR. BARTON: No, that is true. It is not the
75
1 numbers or what has been chosen, it is just a threshold
2 issue, I think is the real danger and is what I would base
3 my comments on.
4 CHAIRMAN MESERVE: And I think beyond that, there
5 still are incentives in looking at the real numbers. They
6 still give a signal and that there are trends, that even if
7 something is green, there is an observable there that may
8 not rise to the level in which the NRC feels it is necessary
9 to yet intervene, but as I think you indicated, licensees
10 are monitoring these very same kinds of things and are
11 prepared to intervene at earlier stages, as appropriate.
12 MR. BARTON: I think the thing you have to careful
13 of is when you look at the indicator and threshold, and what
14 is it going to take to trip the threshold, and if you look
15 at inspection findings that may be tied to that indicator,
16 and you go through the SDP, you may also find that the
17 inspection findings are all green as well, because of the
18 threshold of the inspection findings, when you go through
19 the process.
20 So I think you really need to look at that and
21 say, does this program really, you know, glue together
22 properly? And does the SDP and the inspection piece of it
23 kind of confirm what is going on in PIs, or is it giving you
24 an indication of licensee performance regardless of what the
25 PI is doing, because you may not trip the threshold, but yet
76
1 you may be having inspection findings? I think you really
2 need to look at that together.
3 CHAIRMAN MESERVE: Okay. Let me turn to my
4 colleagues. Commissioner Dicus.
5 COMMISSIONER DICUS: Thank you. You listed
6 several issues and concerns which I think we have heard
7 before, and I have heard them expressed by others as well.
8 Is there one or two, or three that you consider to be the
9 most significant that we really should address prior to
10 implementation?
11 MR. BARTON: With respect to indicators or --
12 COMMISSIONER DICUS: Indicators or thresholds or
13 whatever of all of the concerns you have listed?
14 MR. BARTON: I think that the SECY is addressing
15 the major concerns that we have got. The staff has also
16 identified issues from the pilot program that they have
17 prioritized as to which ones need attention prior to rolling
18 this out for initial implementation, and we had that
19 discussion with the staff last month. I think we agreed
20 that the prioritization is correct.
21 COMMISSIONER DICUS: Okay. The second issue that
22 has come up, and this is in the area of cross-cutting issues
23 that surface. There has been an assumption made that the
24 oversight process itself, that these cross-cutting issues,
25 and there are several of them, would actually show up in the
77
1 cornerstones through other means, and we will be able to
2 capture them. Do you have a view on that assumption?
3 MR. BARTON: I think we would agree with the staff
4 that they will be captured, unless there is --
5 DR. POWERS: I think it is an area of continuing
6 dialogue.
7 COMMISSIONER DICUS: So do I.
8 DR. POWERS: I think, in all fairness, when the
9 staff appeared before us before, they said, gee, we have got
10 these cross-cutting issues and we have got to do something
11 about it, and they really gave us only some preliminary
12 thinking at that time, and that was fair because the
13 take-home message to us really was that they were going to
14 work the cross-cutting issues. And I think we agreed, yes,
15 you definitely have to work the cross-cutting issues because
16 they are not trivial.
17 COMMISSIONER DICUS: And they may not show up in
18 the cornerstones?
19 DR. POWERS: It was unclear to us that, short of
20 Talmudic scholarship, whether one could actually find them
21 within the cornerstones.
22 COMMISSIONER DICUS: All right. And one final
23 thing, it really bothered me on this one slide, Slide 51, I
24 think it is, and this is this, under the values not
25 plant-specific, et cetera, unable to identify trends. Do
78
1 you want to elaborate a little on more that, because that is
2 troublesome?
3 MR. BARTON: Well, the point there was that where
4 you have established thresholds, that it is difficult, if
5 you are just looking at PIs, to establish, say, a decreasing
6 trend in performance until it may be too late.
7 COMMISSIONER DICUS: Okay. And just one final
8 comment, I agree, as we all have, that we recognize this, of
9 course, is a work in progress, and it is going to have to be
10 modified. We are a learning organization. I know I will
11 get a rise out of Dr. --
12 COMMISSIONER DIAZ: We are not becoming, we are.
13 COMMISSIONER DICUS: We are a learning
14 organization. It is a work in progress. You wanted to say
15 something.
16 DR. APOSTOLAKIS: I really have to comment on
17 something. You asked the first question, what do you think
18 are the most important issues, and I have on in my own mind,
19 which we are discussing among ourselves that is important,
20 and I think it is underlies a lot of these problems. I
21 think the objective of the oversight process has not been
22 clearly defined. Are we trying to convince ourselves that
23 the risk profile of Plant X is the way we think it is? Or
24 are we looking at the population of plants and making a
25 judgment that the performance of each one is acceptable?
79
1 They are very different approaches leading to
2 different thresholds, different handling of the performance
3 indicators. Now, there are people who disagree with me of
4 my colleagues here, so this is something we are discussing.
5 But in my mind, unless we settle that, there will be a lot
6 of other issues that will come under different guises.
7 COMMISSIONER DICUS: Thank you.
8 COMMISSIONER DIAZ: I think I have been very nice
9 with you today.
10 MR. BARTON: You have been till now anyhow.
11 [Laughter.]
12 COMMISSIONER DIAZ: So I am going to change. I am
13 going to look at your Slide 48 and really disagree with the
14 statement that this new process consists of performance
15 indicators and baseline inspections. I think that if that
16 is what it looks out there, and then I think there is
17 something wrong. I think fundamentally there is one more
18 feature that is a practical ongoing feature of this process,
19 which is vital to it, which is the process of data
20 collection and incorporation into the corrective action
21 program. And that is, to me, a substantial, you know, and
22 fundamental part of the process.
23 And if we are just looking at whites or greens, we
24 are missing it. And if we don't have a good correlation
25 with baseline inspections, we are missing it. But, really,
80
1 from the beginning, this process started with the premise
2 that we are going to have data gathering that was going to
3 be open, transparent. It is going to be better, it is going
4 to be online, it is going to be state of the art. It is
5 going to be able to be scrutinized. And things were going
6 into the corrective action program. That is a fundamental
7 part of this process.
8 MR. BARTON: Yes, it is.
9 COMMISSIONER DIAZ: That is not, you know, as
10 glamorous as a performance indicator, it doesn't have
11 colors. But it is in this part of the process where the
12 strength of it actually will be. That is where it will be
13 developed. Because it will not only be in the absolute
14 values of what is happening in the corrective action part,
15 it is going to be in the differentiation or in the deltas
16 between components that goes in there.
17 And so I would really, you know, sincerely hold
18 that not only ACRS, but everybody realize there are three
19 major components of this, not two.
20 MR. BARTON: I wasn't trying to put light to the
21 fact that the key is the corrective action program, but I
22 think it tied in to my bullet on baseline inspections,
23 because a lot of the issues identified, both by licensees
24 and by the residents, are going to go into the corrective
25 action program. And how effective that is by each licensee
81
1 is the key. You know, how to prioritize timely correction
2 and, also, was the root cause right, did they really solve
3 the problem?
4 And I think one of the things that -- are we
5 really looking at that? Maybe that is an indicator, maybe
6 that is a leading indicator is the effectiveness of the
7 corrective action program. Maybe that is something we ought
8 to be looking at, because right now it is not that visible,
9 although I know inspectors are going to be looking at that,
10 but is it really that visible? Are we really focusing on
11 the effectiveness of the corrective action program?
12 COMMISSIONER DIAZ: Well, I agree, however, I do
13 want to insist that lost in the glamour of the colors, there
14 is something of tremendous strength and value that is the
15 basis for how all of this started, and it was that data
16 gathering, data analysis, incorporation into corrective
17 action. And that provides the program multiple dimensions
18 that are not obvious when you look at performance indicators
19 or baseline inspection.
20 Having said that, let me go back to something that
21 really concerned me, and that is your Slide 52 or 51, areas
22 of continuous -- let me see. Madame Secretary, it would be
23 worthwhile to ask the centers to put big number of pages in
24 the things so people blind like me can see. 52, bullet
25 Number 2. No clear correlation or interrelationship between
82
1 performance indicators and the baseline inspection program.
2 I think that is a major issue. I think that needs
3 to be strengthened. But I would like to get your comments
4 on it, some additional comments of why you believe this is
5 special.
6 MR. BARTON: Well, I think the reason for that is,
7 to date, we don't see a correlation, we don't see where PIs
8 have driven inspections. It is really, you have got the
9 baseline inspection program and its finding, you know, what
10 have we seen in the pilot program? Not significant
11 findings, as I recall. So it is down there finding the low
12 threshold violations, but when you try to look at those as
13 to compare, and you look at the performance indicators, I
14 don't see a real correlation between the inspection findings
15 that you have seen and how it impacts what is going on in
16 the performance indicators. That is --
17 COMMISSIONER DIAZ: Is something under
18 development, is that something that you think the Commission
19 should ask the staff? Is it not clear to you? Probably not
20 clear to me.
21 MR. BARTON: Well, I think that is something we
22 will discuss with the staff.
23 COMMISSIONER DIAZ: Okay.
24 DR. POWERS: I think the staff has been looking at
25 a clearer correlation between augmented inspections and
83
1 performance indicators and not baseline inspection and
2 performance indicators up till now.
3 MR. BARTON: Right.
4 COMMISSIONER DIAZ: Thank you.
5 DR. POWERS: If I could -- I am not going to let
6 you get away that easily.
7 COMMISSIONER DIAZ: Oh.
8 [Laughter.]
9 DR. POWERS: You bring up correctly the corrective
10 action program. Have you given thought to the kinds of
11 metrics that the corrective action program ought to be
12 communicating to us? You and I have been in the business of
13 inspecting other facilities and we ask questions nearly
14 always when we go to a facility. Gee, what is your
15 maintenance backlog? What is the lifetime of individual
16 issues in that backlog? We have metrics in our mind and we
17 use those in, at least in my case, a very qualitative sense.
18 Is it big, is it small? Are things old or young?
19 Do you think those kinds of metrics are useful for
20 characterizing the corrective action program, or should we
21 just let this corrective action program be the grail that
22 absorbs all the wisdom?
23 COMMISSIONER DIAZ: I think the data processing,
24 the way that it is set, have in it the factors to make those
25 things happen because it will be impossible for licensees to
84
1 maintain, you know, this open process without getting all of
2 those points in there.
3 DR. POWERS: Yes.
4 COMMISSIONER DIAZ: And I believe that eventually,
5 as we progress to it, we will get a better understanding of
6 what are all those things that should be required. I think
7 they are inherent to the process. They will not be able to
8 survive without --
9 DR. POWERS: It has to be done that way, and we
10 will get the metrics that we need.
11 DR. APOSTOLAKIS: Isn't the corrective action
12 program one of the cross-cutting issues?
13 COMMISSIONER DIAZ: If it not, --
14 DR. APOSTOLAKIS: Yes.
15 COMMISSIONER DIAZ: It is.
16 DR. APOSTOLAKIS: Which means we don't do anything
17 about it. Because if it is no good, we are going --
18 DR. POWERS: We haven't done anything anybody it.
19 COMMISSIONER DIAZ: It means we are not putting it
20 -- it is an indispensable component. We are not doing
21 anything about it, but it will be done, it is a natural
22 process. It is a natural process, George.
23 DR. APOSTOLAKIS: Evolution.
24 COMMISSIONER DIAZ: Okay.
25 COMMISSIONER McGAFFIGAN: Just to follow on to
85
1 that, my understanding is the corrective action program
2 under the baseline inspection program is a fairly inspected
3 area, because it is just like design areas that don't
4 normally -- aren't going to be indicated by indicators are
5 going to be relatively heavily inspected, and I would hope
6 that is the case.
7 MR. BARTON: Well, it has to be. It has to be an
8 integral part of the inspection program. Because if you are
9 going to rely on licensee self-identifying, or inspectors
10 identifying an issue, and it is going to go in the
11 corrective action program, so, therefore, you are not going
12 to cite it, the inspectors have got to follow up that
13 process that the licensees are applying to the issue.
14 COMMISSIONER McGAFFIGAN: And like Commissioner
15 Diaz, I think that is a strength of this new program. My
16 theme is going to be not having the perfect, the enemy of
17 the good enough, I want to bring you back to SALP. I mean I
18 look at these charts here and our old oversight process, not
19 just SALP. SALP was the assessment piece of the old
20 oversight process. Did it identify things in advance? I
21 mean Maine Yankee I think had fault 1.5, Crystal River was
22 about 1.5, D.C. Cook was 1.5. You know, and we get
23 surprised.
24 So why -- was SALP any good in terms of
25 identifying things in advance, or the old process?
86
1 DR. POWERS: Well, I think the biggest critique
2 that was laid down on the SALP process by Arthur Andersen
3 was that it was not predictive.
4 COMMISSIONER McGAFFIGAN: Right.
5 DR. POWERS: And I think the point that it is
6 attractive is that the SALP was rather good at encouraging
7 improvements in performance. And we saw that routinely.
8 COMMISSIONER McGAFFIGAN: But to different
9 thresholds. Didn't it? I mean you would have -- I mean one
10 of the problems you had with SALP is that you had one plant
11 being held to one standard in one region and not necessarily
12 -- a plant with identical indicators somewhere else not
13 getting held to the same standard.
14 DR. POWERS: Understand in no sense are we
15 defending the old process. In fact, the one area that I can
16 say the committee has a universal agreement is that this is
17 a better process than what we had before.
18 COMMISSIONER McGAFFIGAN: Okay. So we start with
19 -- that is a good place to start. I mean I think it is a
20 major improvement potentially, I agree. And I always
21 understood the staff recommendation that this was -- that
22 the first, you were going to go for six or nine months.
23 Then we were going to go for a year, and it was going to be
24 a continuing experiment. I mean I am glad that that is
25 fully understood now, but that was always, I think, our
87
1 understanding as to what, you know, full implementation
2 meant. Full implementation meant full experimental
3 implementation subject to improvement as we go.
4 MR. BARTON: I think our issue was that if they
5 had run it longer the first time, we would have been able to
6 correct some of the things that we are now going to have to
7 correct during this initial implementation program for all
8 the plants, that we could have done that with the pilots,
9 and that was our --
10 COMMISSIONER McGAFFIGAN: But the problem with
11 that, in all honesty, is that you have 13 plants in the
12 plant, 90 plants in limbo, and this brings all 103 into the
13 pilot, and we are going to learn at all 103. I think we
14 will get a larger database. I think there was a tradeoff
15 there, and I think it was a rational decision in all
16 honesty, because we really had suspended SALP and there are
17 no PIs. We were trying to do an annual meeting at these
18 plants to discuss something. What is the -- the PPR? The
19 PPR. That was basically all we had for the 90 was a plant
20 progress report which was sort of watered down mini-SALP.
21 So the question is -- there was a tradeoff there,
22 I believe.
23 DR. POWERS: I kind of like PPRs.
24 COMMISSIONER McGAFFIGAN: You like PPRs.
25 DR. POWERS: They give you a good insight on the
88
1 plant. I learned a lot from those.
2 DR. BONACA: I would like to just make a statement
3 on a personal basis. I mean one of the issues is that we
4 were asked a question regarding the technical adequacy of
5 the performance indicators, and I had trouble with that
6 because it is very hard to decouple the performance
7 indicators from the process. I personally believe the
8 process is a much -- is a high improvement on what we had
9 before.
10 But then when I look at the performance indicators
11 and I have to address them on a technical basis, then I have
12 trouble, because, again, that is an integral part of it.
13 And if I look at the individually without the consideration
14 that they should be, in fact, looked at together with the
15 program in general, then I begin to pick on specific issues
16 including the thresholds.
17 COMMISSIONER McGAFFIGAN: Well, we may not have
18 given you a broad enough mandate in that case, because, you
19 know, I agree with you, this is a significant improvement,
20 and it is a work in progress, and it will be even better a
21 year from now.
22 The significance determination process, have you
23 all -- I mean it isn't one, it is several, right, there is
24 different significance determination processes for some of
25 the softer areas? Are we in the significance -- and this is
89
1 not the question, I guess, but are we -- are you comfortable
2 with the implied thresholds and the significance
3 determination process?
4 DR. POWERS: I think we have major questions on
5 the significance determination process. And, in fairness,
6 we have been supplied some written material by the staff and
7 only the briefest of comments to the effect that, yes, we
8 are doing a significance determination process. That is
9 area we still need to go into with the staff. But the
10 written material has provoked a lot of questions. And it is
11 fair to say that there are areas that are soft and will
12 always be soft. And there are areas that are hard and
13 everybody accepts they are hard. There is some divine
14 middle ground where I think creative ways have been adopted
15 to get around some of the deficiencies and there are
16 available tools now. And that is the area where I
17 personally have a lot of difficulties.
18 COMMISSIONER McGAFFIGAN: One of the things that
19 was said to us at the start of the pilot program was if you
20 took all 103 plants, you know, this was likely to turn out,
21 there would be a hundred-or-so findings a year that would
22 have to enter the significance determination process and 10
23 would turn out, you know, approximately as being something
24 that would move a performance indicator into white or
25 yellow, or, God forbid, red.
90
1 And we didn't have -- I think, like you said
2 earlier, the database, you have, you know, 13 plants for
3 nine months, you know, your probability is you will get --
4 we might have got one that would get through the process.
5 We didn't, I guess. Over the next year we will.
6 It strikes me that having you follow the
7 significance determination process application over the
8 coming year, as we deal with real cases, you know, would be
9 useful, and then you could -- then we can determine after a
10 year's effort whether we have got the thresholds right in
11 the SDP so that we are getting these 10 big findings, or 15
12 or whatever proves to be a year out of it, and they truly
13 are significant, and we are not missing, you know, the next
14 15 that should have been, in your view, captured by the SDP
15 -- I should say processes, because it is multiple processes.
16 DR. POWERS: And done in three phases.
17 COMMISSIONER McGAFFIGAN: Right. The last
18 question on the trending, I don't totally understand the
19 stuff about, you know, the disincentive to improve plant
20 performance. It strikes me that people still will want to
21 be high green as opposed to low green. And you will have
22 the exact number, it may result in a green, but it strikes
23 me that, as you say, these administrative limits and all
24 that, and INPO, I mean insurance and all that drives you
25 towards -- because many of these indicators are WANO
91
1 indicators as well, as I understand it. That all drives you
2 towards wanting to be high green.
3 So there are incentives. Maybe they are not
4 entirely in our system, maybe they are imposed by the
5 licensee. Maybe they come from the INPO space. But it
6 strikes me that there are insurance costs, et cetera,
7 continuing incentives to want to be high green. And if I go
8 back to the infamous SALP process, we did have plants who
9 were quite comfortable limping along in SALP 2.5 space
10 forever. Never got on the watchlist. In fact, I can think
11 of one, it was SALP 2.5 forever, for, you know, 15 years or
12 so, 2.5, 2.25, 2.75, but never got watchlist and never --
13 and they weren't incentivized. You know, SALP might
14 incentivize if you wanted to. But why is it different?
15 Why, if the numbers are --
16 MR. BARTON: I just go from my performance
17 experience and know that I had a lot of pressure when I ran
18 a power plant to become a SALP I, and if I look today at
19 where most of the licensees are, and most of them are in
20 green, with most of their indicators in green, and a lot of
21 them in the high green, and I say, okay, I am doing fine at
22 this threshold, and as long as I continue at this threshold,
23 I am fine.
24 So, what is the bad side of that? Well, if you
25 don't improve, you are going to start sliding backwards.
92
1 But there is no carrot out there to get.
2 COMMISSIONER DIAZ: Money.
3 COMMISSIONER McGAFFIGAN: In our system, there
4 isn't. In the INPO system, there is, because if you are not
5 INPO I, your insurance costs are higher, right? I don't
6 know how big that is. Is that a few hundred thousand
7 dollars a year? It is not trivial.
8 MR. BARTON: I don't remember.
9 COMMISSIONER McGAFFIGAN: Okay.
10 DR. APOSTOLAKIS: I think there is a much more
11 fundamental issue here. It is not a matter of setting the
12 limited threshold at 95th percentile or 90th. What is the
13 agency trying to do with the oversight process? It comes
14 back to what I said earlier. Do you want to know that the
15 risk profile of Plant X is today the same as it was a year
16 ago, or are you looking at it as industry-wide? The rest is
17 just trivial applications. Unless we --
18 DR. POWERS: How true, Professor.
19 DR. APOSTOLAKIS: Let me give you the other side.
20 COMMISSIONER McGAFFIGAN: Make sure that they are
21 following that they are following the rules and regulations
22 of the NRC and that --
23 DR. APOSTOLAKIS: But these rules and regulations,
24 the plant was licensed as an individual plant, not as part
25 of a population of a hundred plants. So that implied a
93
1 certain profile. Look at the IPE insights report, each
2 profile is different. And all of a sudden, they want to
3 say, oh, look at the whole population, take the 95th
4 percentile and everything is fine. That is not -- I think
5 that has to be settled.
6 And then what do you with the poor five fellows
7 who are above the limit? What do you do with those? You
8 are declaring that they are already yellow, red.
9 DR. POWERS: Well, I think the problem is when you
10 across the threshold, through no fault of your own, it is a
11 peculiarity of your design and when your design was
12 submitted for approval, it included compensatory measures so
13 that any perceived deficiency there was corrected. Now, you
14 create a threshold, the poor guy is across it and he can
15 never get back, because it is part of his design and he is
16 not getting the appropriate credit for other features of his
17 plant that are not reflected.
18 DR. APOSTOLAKIS: Exactly. Exactly. It is a
19 unique profile.
20 COMMISSIONER DIAZ: There is a value to
21 statistics, right?
22 DR. APOSTOLAKIS: Oh, as a general statement,
23 Commissioner, there is great value.
24 [Laughter.]
25 COMMISSIONER DIAZ: Since I have this flu, you
94
1 know, my mind is not working well. And there is a value to
2 having a population being defined, right?
3 DR. APOSTOLAKIS: The statistics in this case
4 should be applied to each plant. Because I think what Dr.
5 Powers says is a key issue here. You may be very high on
6 one indicator, but you have other compensatory measures that
7 will appear in the PRA. But if you are dealing only with
8 that performance indicator, you are in trouble.
9 COMMISSIONER DIAZ: I agree.
10 DR. APOSTOLAKIS: So this is a key issue in my
11 view.
12 DR. KRESS: And there is a clear difference of
13 opinion.
14 COMMISSIONER McGAFFIGAN: My question, is this a
15 clear theoretical issue or is this a clear practical issue?
16 But that is --
17 DR. POWERS: I think we already have examples of
18 approximately five cases of where you are going to have a
19 plant that is going to have a white indicator, even though
20 -- I mean he can do nothing about it, not unless he wants to
21 rebuild his pressurizer or something like that.
22 DR. KRESS: That is one of the contentious areas,
23 that he can do nothing about it. There are some of us that
24 think he can.
25 COMMISSIONER DIAZ: Okay.
95
1 CHAIRMAN MESERVE: Commissioner Merrifield.
2 COMMISSIONER MERRIFIELD: I might return to my
3 analogy about the artist versus the house painter. I start
4 as a baseline that we have agreement -- had agreement,
5 continue to have agreement between a couple of data points,
6 one of them being NEI and its membership, and the other
7 being the Union of Concerned Scientists. Both of those
8 groups, which don't agree very often, agreed the SALP
9 program was not a very good mechanism for determining how
10 safe these plants were. Both of those groups also agree
11 that this new program is an improved mechanism for
12 evaluating the safety of these plants.
13 Now, a lot of the discussion today has been on the
14 performance indicators, and there are some valid concerns
15 that the committee raises, and, indeed, those are the very
16 same kind of concerns that members of this Commission on
17 this side of the table have raised at various points during
18 the course of the last year.
19 There is overall recognition that this is a work
20 in progress. In a perfect world perhaps, if we could have
21 stopped time in its place, we could have worked hard and
22 come up with a perfect set of indicators and a perfect way
23 of rolling those out. This is indeed not a perfect world,
24 and what the Commission was faced with was a need to, as
25 quickly as possible, fix a system that was not adequate, and
96
1 respond to our stakeholders and try to do the best we can.
2 We will need to continue to work with ACRS and the
3 staff to make sure that we can improve those performance
4 indicators. And I, and I think the other Commissioners
5 agree with me, are under no preconception that these
6 performance indicators will stay precisely the way they are.
7 They will evolve, we will add new indicators. We will
8 perhaps get rid of some of the current indicators, and we
9 will change the thresholds, and we will appropriately
10 balance the significance determination factors.
11 One thing that can't be lost in all of this is an
12 important component, and that important component, I
13 believe, is our inspectors. I have had the opportunity to
14 go out and visit a lot of plants this year, and the issue of
15 this new inspection and enforcement process is probably the
16 most important thing I discuss with our inspectors and with
17 our licensees. And there are a couple of observations that
18 I would make.
19 First of all, I think is some -- there has been
20 some fear, and I think our regional administrators are doing
21 a good job of trying to alleviate this, and that is that --
22 the fear was that we are going to so limit our inspectors in
23 terms of what they can look at, that they wouldn't be able
24 to share with the licensees their concerns, the things that
25 they see, that backsliding or the problems that perhaps
97
1 weren't picked up by the indicators coming forward.
2 As a result of my discussions, I believe we can do
3 a better job of encouraging our inspectors to understand
4 that, indeed, if they see problems that don't necessarily
5 fit on the matrix inspections, that those are still issues
6 that we expect, as a Commission, for them to raise with the
7 licensees. We hire them as inspectors because they are
8 bright, because they are able to find these issues, and we
9 want them to act in that particular manner.
10 Similarly, in the discussions with the licensees
11 to an individual, you know, the head, whether it is a senior
12 VP, president and CEO, all of them have said, I want to know
13 about those. I don't want to be limited to simply the
14 baseline inspection. If there are concerns that your
15 inspectors see at my plant, I want to know about it.
16 To a man, as well, they have also said, the
17 performance indicators are not a baseline for our
18 performance. As you mentioned, all of them, virtually all
19 the plants I visited, have a whole other set of indicators
20 which are far more stringent than the ones that we have, and
21 that is what they are managing themselves towards, not our
22 baseline. And the reason for that, as has been explained to
23 me by some of the industry folks, is that things have
24 changed in this industry. The economic pressures to make
25 sure that these plants are operating appropriately is a
98
1 driving force towards a greater level of safety, because if
2 they backslide in these facilities, a shutdown could indeed
3 be the end of that plant.
4 And I think there is a greater understanding of
5 that in a deregulated market among the plant owners at this
6 time.
7 Now, is that to say that at some point down the
8 line we are not going to have a plant out there that may be
9 trying to cut close to the margin? Well, that may very well
10 be. We may have that, and, indeed, we may have additional
11 plant shutdowns. But I think -- I don't believe -- I don't
12 believe, at least given all the conversations I have had,
13 and they have been many this year, that folks are going to
14 be managing themselves merely towards just staying at the
15 low part of green.
16 MR. BARTON: I think you made a good point with
17 respect to the inspectors rolling this new process. The one
18 concern I would have is that the agency, regional
19 administrators, down through the whole ranks in the regions,
20 be careful that the SDP does not bog down the inspectors and
21 take away from their time in the plant, which -- and there
22 is a possibility that could happen.
23 And, also, frustration on the part of inspectors
24 by finding issues, identifying them as violations, applying
25 the SDP and finding out it is only green, and I think that
99
1 is another caution I would throw out, because if I keep
2 doing that, and I keep finding green, I eventually say, you
3 know, what is the purpose of what I am finding, it doesn't
4 mean anything. I think you need to be conscious that that
5 -- that there is a potential there for that to happen.
6 COMMISSIONER MERRIFIELD: And for that very
7 reason, we will continue to monitor this as we go forward
8 and continue to assess and improve it as we work in the
9 future.
10 I don't have a question, I would just repeat as I
11 did in the last -- after the last panel, I will quote Dana
12 Powers again, there is universal agreement from the ACRS
13 that this is a better process. And after all these
14 questions we have had on this panel, I think that is the
15 appropriate place for us to end.
16 Thank you, Mr. Chairman.
17 CHAIRMAN MESERVE: Commissioner Diaz, do you have
18 an additional comment?
19 COMMISSIONER DIAZ: Yes. I will try to make it as
20 quick as possible. I am just trying to maybe whet the
21 appetites of statisticians and PRA people, and go back to my
22 statement on the data processing, the data gathering, the
23 data process and the corrective action. Years from now,
24 that information will contain all the statistically
25 significant data. Then when correlated in the corrective
100
1 action program and cross-correlated with the results of the
2 corrective action program will give you the information that
3 you need to determine when trends are changing.
4 DR. APOSTOLAKIS: I agree.
5 CHAIRMAN MESERVE: Wow. On that fine note, we
6 definitely should end this.
7 [Laughter.]
8 CHAIRMAN MESERVE: I would like to thank all of
9 you for some very helpful presentations this morning. This
10 has very helpful data for us. We also very much welcome
11 your continued involvement in areas that are of enormous
12 significance to the Commission and in which your assistance
13 is very much appreciated.
14 With that, we are adjourned.
15 [Whereupon, at 11:59 a.m., the briefing was
16 concluded.]
17
18
19
20
21
22
23
24
25
