The PDF version 
Display Related Directives to this directive.
Display Reference Documents to this directive.
U.S. Department of Energy GUIDE
Washington, D.C. DOE G 151.1-3
7-11-07
Programmatic Elements
Emergency Management Guide
[This Guide describes suggested nonmandatory approaches for
meeting requirements. Guides are not requirements documents and
are not to be construed as requirements in any audit or appraisal
for compliance with the parent Policy, Order, Notice, or Manual.]
1. PROGRAM ADMINISTRATION
1.1 Introduction
The purpose of this chapter is to assist Department of
Energy (DOE) and National Nuclear Security Administration
(NNSA) field elements in complying with the DOE O 151.1C
(the Order) requirement to provide effective organizational
management and administrative control of an emergency
management program by establishing and maintaining
authorities and resources necessary to plan, develop,
implement, and maintain a viable, integrated, and
coordinated program. Each manager or administrator of a DOE-
, NNSA- and/or DOE/NNSA contractor-operated facility/site or
activity subject to this Order shall designate an individual
to administer the emergency management program. This
individual shall develop and maintain the emergency plan,
develop the (Emergency Readiness Assurance Plan (ERAP) and
its annual updates, develop and conduct training and
exercise programs, coordinate assessment activities, develop
related documentation, develop a system to track and verify
correction of findings or lessons learned, and coordinate
emergency resources. Responsible administrators of emergency
management programs should use the guidance in this chapter
to define responsibilities and implement functions to ensure
and maintain effective emergency planning, preparedness,
readiness assurance, and response activities.
This chapter is designed primarily for facilities/sites or
activities that are required to implement an Operational
Emergency Hazardous Material Program and is directed at
operations and emergency management staff at Field Elements
and operating contractor organizations responsible for DOE
and NNSA facilities/sites or activities.
This guide and supersedes the following.
DOE G 151.1-1, Vol. 5, Ch. 1, Program Administration,
dated 8-21-97
DOE G 151.1-1, Vol. 5, Ch. 2, Standard Format and
Content for Emergency Plans, dated 8-21-97
DOE G 151.1-1, Vol. 5, Ch. 3, Emergency Readiness
Assurance Plans (ERAPs), dated 8-21-97
DOE G 151.1-1, Vol. 5, Ch. 4, Training and
Drills, dated 8-21-97
DOE G 151.1-1, Vol. 7, Ch. 1, Development and
Conduct of Exercises, dated 8-21-97
DOE G 151.1-1, Vol. 7, Ch. 3, Exercise
Controller and Evaluator Manual, dated 8-21-97
1.2 General Approach
Management and Operating (M&O) contractor
managers/administrators at each DOE/NNSA facility and site
retain overall authority and responsibility for emergency
management at their respective levels. However,
responsibility for and authority over the development and
day-to-day operation and maintenance of the program should
be delegated to a specifically designated emergency
management program administrator, with responsibility and
authority to ensure:
Development and maintenance of the Hazards Surveys and
Emergency Planning Hazards Assessments (EPHAs), emergency
plans and procedures, and related and supporting
documentation
Development and conduct of training and exercise
programs, and development, conduct, and coordination of
the readiness assurance program and activities
[e.g., evaluations (internal and external)]
Coordination of emergency resources by identifying
resource needs and ensuring the availability of adequate
resources
Development and submittal of the annual ERAP
Interface with State and local emergency response
elements, other Federal agencies, and private
institutions providing emergency medical and other
emergency support to the site
The designated administrator has authority and resources in
accordance with assigned responsibilities and has access to
top-level management. The administrator is responsible for
implementing a facility/site- or activity-specific
comprehensive emergency management program based upon a
graded approach that is commensurate with hazards.
The administration of programmatic activities
(i.e., planning, preparedness, readiness assurance, and
programmatic response element) activities is established and
maintained through rigorous adherence to a formal process.
Review and approval processes are established and documented
to ensure that the planning and development of components of
the emergency management program receive sufficient
oversight by staff, management, and DOE/NNSA elements. To
ensure that programmatic activities are initiated,
completed, and periodically repeated in a timely and
efficient manner, reasonable schedules are established for
planning (e.g., document submission, reviews, approvals),
preparedness (e.g., training), readiness assurance
(e.g., self-assessments), and programmatic response element
[e.g., maintaining Emergency Response Organization (ERO)
assignment roster] functions.
Adequate resources are identified and obtained to ensure
that the program is ready to respond. Financial resource
requirements are identified and budgeted. Response facility
needs are identified and locations established. Equipment
requirements are identified; supplies of required equipment
are monitored and acquired as needed. Personnel
requirements are identified and addressed.
An emergency management document control system that meets
industry standards for document review, approval,
distribution, and change control is established or emergency
management documents are controlled under an existing site-
wide document control system. An auditable administrative
program for ensuring the availability of vital records
(i.e., essential to the continued functioning or
reconstitution of an organization during or after an
emergency), regardless of media, is established and reliably
maintained (Cf. DOE O 243.2). If classified information or
materials are being used or generated, effective security
procedures and controls are implemented, and security
reviews are conducted.
Administration of an emergency management program can vary
considerably from site to site depending on characteristics
of the site and program, including, size, geographical
layout, hazards, administrative structure of the M&O
contractor, and structure and constituents of the ERO.
A small site with few facilities and hazards and a simple
response structure may have one program administrator
responsible for management and control of the program, who
may have direct responsibility for various aspects of the
detailed planning, implementation, and maintenance functions
and activities. At a larger site with many facilities, more
extensive hazards, and a more complicated ERO, the program
administrator may delegate detailed programmatic
responsibilities to site-level and facility-level
administrators, retaining overall responsibility for site-
wide program administration and control.
Information and data that the designated site administrator
can track and oversee depends primarily on the size of the
site and scope of the emergency management program. With
only a few facilities, the program administrator at a small
site is familiar with details of the site-wide program, as
well as each individual facility program. The administrator
at a large site may only be personally cognizant of the
larger aspects of each facility program (e.g., percentage of
each facility ERO trained versus the detailed data on who is
trained).
On a multiple-facility site, the site emergency management
program administrator is responsible for tasks similar to
those of the facility program administrator (or for all
tasks, if the sole administrator). In addition, the site
program administrator is responsible for review and
oversight of emergency management activities of facility
emergency management program administration. The site
program administrator must prepare guidance for facility
emergency planners to ensure an effective, integrated site
program is achieved when the facility capabilities are
activated for a coordinated response.
The program administrator's job is to ensure the emergency
management program is developed and maintained - not
necessarily to perform all these tasks or track/monitor all
activities personally. Emergency management authority may
be delegated to subordinate administrators responsible for
various aspects of the program (e.g., exercises, training,
plans and procedures). The designated emergency management
program administrator has ultimate responsibility for
ensuring that requirements of the Departmental emergency
management-related policies and Orders are met. The program
administrator coordinates with other site groups responsible
for implementing various aspects of emergency preparedness
and response (e.g., Health Physicists, Industrial
Hygienists, Medical, Public Affairs, Security, Operations,
and Engineering).
The general approach in this chapter focuses on functions or
activities, specific responsibilities, and documentation
that are expected to be accomplished at any DOE or NNSA site
in order to effectively manage and administer an emergency
management program. At a site with a single site
administrator or at one with multiple facility/area
administrators, the same management and administrative
control is required to assure that emergency response
capabilities are maintained and are ready to respond.
In the following sections, general responsibilities of
program administration are discussed in the context of the
key activities of an emergency management program: planning,
preparedness, readiness assurance, and response.
1.3 Planning Responsibilities
As indicated in the Order, emergency planning includes the
identification of hazards and threats, hazard mitigation,
the development and preparation of emergency plans and
procedures, and the identification of personnel and
resources needed for an effective response. In the
following sections, the responsibilities of program
administration with respect to emergency management planning
activities are divided according to the following topics:
technical planning basis, program implementation,
documentation, resource management, and policy issues.
1.3.1 Technical Planning Basis
The primary responsibility of the emergency management
program administrator(s) is the establishment and
implementation of the Comprehensive Emergency Management
System. This involves the establishment of an Operational
Emergency Base Program that coordinates and integrates the
emergency planning and preparedness requirements of
applicable Federal, Tribal, State, and local laws,
regulations, and ordinances, and other Orders and standards
of performance. As warranted, the Base Program is expanded
to implement additional emergency management requirements of
an Operational Emergency Hazardous Material Program, if
hazardous materials pose a major threat to the health and
safety of workers and the public.
The Hazards Survey and EPHA (if required) are the technical
planning basis for establishing the scope of the
facility/site- or activity-level emergency management
program. Development of the Hazards Surveys and EPHA is
often a complex and multi-disciplinary activity involving a
number of technical skills and facility/site or activity
organizations, coordinated by the emergency management
program administrator(s). The program administrator(s)
ensures that the proper technical staff are assigned to the
efforts [e.g., health physicists when radioactive materials
are involved; industrial hygienists to address toxic
chemicals; meteorologists; subject matter experts (SMEs) in
the transport and dispersion of hazardous materials;
operations personnel; security specialists; etc.].
A key responsibility of program administrator(s) is to
ensure that the technical planning basis (i.e., Hazards
Surveys and EPHAs) is regularly maintained and reflects the
current operations and hazards associated with the
facility/site or activity. The appropriate method is
dependent on the specific facility or site programs
established on the site [e.g., hazardous materials inventory
control systems, Integrated Safety Management Systems
(ISMS)] that can be utilized to achieve emergency management
objectives.
1.3.2 Program Implementation
Using the results and conclusions of the technical planning
basis, the program administrator(s) coordinates the
development of the emergency plan(s) and the implementing
procedures for the Base Program and, as required, the
Hazardous Material Program and ensures that they are
commensurate with the hazards on the facility/site. As with
the Hazards Survey and EPHA efforts, this activity may
require involvement of personnel from a variety of technical
areas and facility/site organizations.
The program administrator(s) ensures that emergency plans
and implementing procedures are coordinated with all
involved site and facility response elements, integrated for
site-wide consistency, and in accordance with Departmental
policies. Emergency management plans are developed for Base
Program Facilities, which must address the minimum Base
Program requirements, and for facilities requiring a
Hazardous Material Program, whose requirements are
seamlessly integrated with Base Program requirements.
Coordination and cooperation of tenant facilities (if any)
with the site organization in programmatic and response
activities should be described in the emergency plans.
The program administrator(s) ensures that emergency plans
and procedures have the following characteristics:
Document the emergency management program, including
provisions for response to an Operational Emergency (OE)
and procedures to describe how the emergency plan will be
implemented.
Clearly state roles, responsibilities, and requirements
associated with program administration, EROs, individual
positions, operations, and interfaces.
Describe the integration and coordination of the
emergency management program with the DOE/NNSA ISMS.
Are compliant with the requirements of the National
Response Plan (NRP) and the National Incident Management
System (NIMS).
1.3.3 Documentation
Documentation of the technical planning basis (i.e., Hazards
Surveys and EPHAs) is an essential component of an emergency
management program. It represents the technical information
related to hazards on the facility/site or activity, methods
and assumptions that form the foundation of the program, and
documented evidence that responsible emergency management
planners understand the facility/site- or activity-specific
hazards. Existing hazardous material databases and safety
documentation are monitored to ensure that Hazards Surveys
and EPHAs represent the current status of hazards and
operations at the facility/site or activity.
The program administrator(s) is responsible for ensuring
that emergency plans and procedures are developed, verified,
validated, reviewed periodically, updated as necessary, and
that the program receives an appropriate level of oversight.
This includes providing direction and guidance for
conducting and documenting reviews, assessments, and
approvals to ensure they are consistent, correct, up-to-
date, and complete. Program administrators ensure that
reasonable schedules are established and SMEs are made
available to provide competent reviews and evaluations.
Appendix A contains an outline and recommended content for
an emergency plan for a facility/site required to have a
Hazardous Material Program.
1.3.4 Resource Management
Emergency management programs require resources to function
effectively. Emergency management programs are developed
based on the technical planning basis, four programmatic and
ten response program elements. Each program element
requires financial, material, and human resources to develop
and maintain the program.
Financial resources. The program administrator(s) tracks
the financial resources allocated for their emergency
management programs, including costs of facilities,
equipment needed to respond to emergencies, training
programs, drills and exercises, and all related personnel
costs. Annual budgets are prepared, based on program
needs identified through the readiness assurance process.
The program administrator(s) provides justification for
budget requests and acts as an advocate for needed
resources. (Cf. DOE Order 226.1, Implementation of
Department of Energy Oversight Policy, for possible
additional sources of information regarding the
identification of program needs.)
Emergency facilities and equipment requirements. Changes
in Hazards Surveys and EPHAs and the results of program
and exercise evaluations and self-assessments may
identify needed modifications and improvements that
necessitate revising or updating facilities or equipment.
Personnel requirements. Program and exercise
evaluations, as well as the self-assessment process, may
identify additional personnel needs. These requirements
should also be documented so that additional resources
can be allocated.
In summary, the program administrator(s) is responsible for
ensuring: adequate resources are identified and obtained to
ensure that the program is ready to respond; financial
resource requirements are identified and budgeted;
facilities and equipment requirements are identified,
monitored, and acquired; and personnel requirements are
identified and addressed. Resource needs (including
personnel, facilities and equipment, and financial) are
identified and justified in the annual ERAP.
1.3.5 Policy Issues
The emergency plan and associated procedures, as well as
supporting planning documentation (e.g., EPHAs), must comply
with DOE/NNSA policy as contained in DOE O 151.1C,
Comprehensive Emergency Management System. A companion
Emergency Management Guide (EMG), DOE G 151.1-series,
provides guidance for implementing the Order requirements
and represents a source for interpreting the intent of the
requirements in the Order. The program administrator(s) is
responsible for ensuring that all of the emergency
management program elements are consistent with Order
requirements, including both prescriptive requirements as
well as broadly stated and general performance goals found
in the EMG. Plans must also be fully compliant with the
requirements of the NRP and NIMS.
Examples of policy issues that should be monitored include:
The Order contains no prescriptive requirements for
performing EPHAs. However, the EMG provides sufficient
guidance for developing an EPHA that satisfies the intent
of the Order (e.g., a spectrum of events analyzed,
including beyond-design-basis events, identification of
emergency recognition indicators).
The emergency plans and associated procedures for
consequence assessments and protective actions are
consistent with the Order requirements in terms of the
components (and phases) of the assessment process and
Protective Action Criteria (PAC) selected for triggering
protective actions.
Prescriptive times for emergency notifications are given
in the Order and must be followed in the facility/site
and activity procedures.
ERO training requirements (i.e., initial and refresher
training) are given in the Order.
1.4 Preparedness Responsibilities
As indicated in the Order, emergency preparedness includes
. . . the acquisition and maintenance of resources, and the
conduct of training, drills, and exercises. In the next
four sections, the responsibilities of program
administration with respect to emergency preparedness
activities are divided according to program plans, program
implementation, documentation, and policy issues.
1.4.1 Program Planning
Training and Drills. DOE/NNSA emergency management training
and drill programs ensure that personnel are prepared to
respond to, manage, mitigate, and recover from emergencies
associated with DOE/NNSA facilities/sites and activities.
Training programs can include a variety of instruction
methods, such as classroom instruction, computer-based or
web-based coursework, and hands-on training and drill
activities.
General training for employee response, required as part of
the Base Program, may be included as part of an employer's
General Employee Training (GET) program. This program may
include emergency awareness, warnings and alarms, evacuation
and accountability, and first aid. Hazardous Material
Programs have additional training requirements for
developing and maintaining specific emergency response
capabilities for all personnel identified as members of
their EROs. The training program should be commensurate
with the hazards identified in the EPHA.
The program administrator(s) must ensure the development and
coordination of training program activities to prevent
conflict with other activities and to ensure that resources
are available. In larger facility/site programs, there may
be an individual assigned to be the manager of the training
program(s). In smaller programs, the designated site
emergency management program administrator(s) may be
responsible for all aspects of the training program. The
program administrator should also ensure a formal training
plan [cf. DOE G 151.1-3, Chapter 2] is developed that
describes program goals and objectives, organizational
responsibilities, resources, and planned activities.
The administration of training and drills programs should
include functions to:
Ensure that a plan is developed and maintained describing
and documenting the training and drills program.
Ensure a comprehensive and coordinated program of
training and drills for the identified ERO, both primary
and alternate members.
Establish training requirements for each position in the
facility/site- or activity-specific ERO.
Ensure a coordinated program for all responders, both
initial training, and annual refresher training based on
the plan and procedures for that emergency management
program.
Identify and coordinate adequate resources for training
program implementation, including facilities, equipment,
budget, etc.
Ensure the integration of GET in the Base Program and
training programs necessitated by the DOE/NNSA Hazardous
Material Program.
Ensure the training program adequately addresses each
Response Element of the facility/site or activity
emergency management programs.
Ensure that the program plan provides for demonstrations
of proficiency following training for ERO positions.
Identify training needs and provide for development,
scheduling and delivery of training activities.
Establish qualifications for the training staff for each
training module.
Ensure that drills provide practical, hands-on training
and use realistic situations and scenarios; ensure they
are coordinated with site groups, such as Health
Physicists, Industrial Hygienists, Medical, Public
Affairs, Security, etc.
Exercises. Emergency management exercises are formal,
evaluated demonstrations of the integrated response
capabilities of an emergency management program. Exercises
are conducted to validate the response program elements of
an emergency management program. Exercises should be
realistic simulations of potential facility/site or activity
emergencies. They may vary significantly in size and
complexity to achieve their respective purposes. Department
of Homeland Security (DHS) exercise methodology refers to
these evaluated demonstrations as discussion-based or
operations-based exercises (cf. Chapter 3 of DOE G 151.1-3).
Exercise-specific objectives are used to establish the
exercise scope, specify the emergency response functions to
be demonstrated, identify the extent of organizations/
personnel participating, and identify the breadth and depth
of exercise activities to be accomplished or simulated.
Typically, not all emergency management program elements are
demonstrated in each exercise. The program administrator(s)
ensures that a systematic approach is used, with emphasis on
participation and coordination among the members of the
EROs, to develop an exercise plan to ensure that all
elements of facility/site and activity programs are
exercised and validated over a multi-year period.
Coordination of exercises is particularly important at sites
with multiple, integrated facility emergency management
programs, where response resources are shared, and
efficiency in scheduling and conducting exercises is
paramount.
The administration of exercise programs should include
functions to:
Ensure a formal exercise program to validate all response
elements over a five-year period, in accordance with
DOE O 151.1C.
Ensure each exercise has specific objectives keyed to the
emergency plan and procedures.
Ensure that exercises are evaluated, including a critique
process to gather and document observations of the
participants.
Ensure that a system is in place to track lessons-learned
and corrective actions resulting from the evaluation of
exercises.
Ensure, at a minimum, the conduct of building evacuation
exercises consistent with Federal regulations, local
ordinances, or National Fire Protection Association
(NFPA) Standards.
Ensure tests of communications systems annually, or as
often as needed, to ensure information can be efficiently
exchanged with response organizations off site, and at
DOE/NNSA field element and Headquarters (HQ)-levels.
1.4.2 Program Implementation
Training and Drills. The emergency management program
administrator(s) has the responsibility for the conduct of
the training program, including the scheduling of drills,
based on the plans and procedures developed for the specific
program. Administration of the training and drills program
implementation should include functions to:
Ensure training and periodic drills are scheduled,
conducted, monitored, and documented.
Ensure coordination of training and drills at sites with
multiple facilities.
Ensure auditable training and drill records are
developed, maintained, and updated.
Ensure a system is in place to track the development and
implementation of lessons-learned from training and
drills and promote program improvements.
Conduct periodic (training program) self-assessments,
including evaluating instruction and reviewing (training)
materials.
Ensure drills provide supervised, hands-on training for
members of EROs.
Ensure emergency preparedness training is provided to all
workers who may be required to take protective actions.
Ensure emergency-related training on facility/site
conditions and hazards is made available/offered to
offsite response organizations that may need to respond
onsite.
Maintain access to a qualified training staff.
Exercises. For effective conduct of the exercise program,
the administrator(s) must ensure that:
Each facility exercises its emergency response capability
annually in a facility operations-based exercise.
Each site exercises its site-level ERO elements and
resources, as well as its integrated emergency response
capability, at least annually in a site operations-based
exercise. For multiple-facility sites, this site-wide
exercise will be rotated among the facilities.
Offsite response organizations are invited to participate
in the annual site-wide exercise once every three years.
Site-wide exercises that include offsite participation
are referred to as full-participation operations-based
exercises.
The annual site-wide exercise is a full-participation
operations-based exercise at least every third year, if
offsite response organizations agree to participate.
The evaluation of exercises conducted by the sites and
facilities is accomplished by knowledgeable, independent
organization(s) whose staff displays familiarity with
responder organizations, functions, and procedures.
Auditable exercise records are developed, maintained, and
updated.
Corrective actions items, identified as a result of the
evaluation and critique process, are incorporated into
the emergency management program.
A system is in place to track the development and
implementation of lessons-learned from exercises and
promote program improvements.
1.4.3 Documentation
Training and Drills. The training program plan should be
documented and training materials archived. Training and
drill records enable the emergency management program
administrator(s) to determine the types of training to be
scheduled, the numbers of people to be trained, the specific
individuals to be trained, and the type and quantity of
resources that are needed to conduct the training and
drills. Scores on training validation tests and performance
during drills should also be recorded. The training records
provide a means for verifying qualification requirements for
ERO participation.
Lessons-learned from training and drill sessions are
recorded and correlated with exercise evaluations, and other
readiness assurance activities to determine additional
training program needs.
Exercises. A complete, documented operations-based exercise
package [i.e., an Exercise Plan (EXPLAN), as described in
DOE G 151.1-3, Chapter 3] should be produced for each annual
site-level exercise. Facility-level exercises can be
accomplished with an exercise package that contains only the
essential elements that are required to actually conduct the
exercise. Exercise participation records enable the
emergency management administrator(s) to ensure that
individual members of the ERO are given the opportunity to
demonstrate their proficiency annually. An exercise report
[i.e., After Action Report (AAR)] should be produced
following the exercise that provides an account of exercise
control, player performance, and self-assessment evaluation
findings.
1.4.4 Policy Issues
Training and Drills. Policy issues related to training and
drills will focus on the adequacy of the training to prepare
ERO members for their respective response tasks. The Order
requires a coordinated program of training and drills for
developing and maintaining ERO position skills, including
initial and refresher training. The administrator(s) are
responsible for ensuring that the training matches the
skills required for the specific ERO positions.
Exercises. In addition to frequency requirements for
conducting the exercises, the administrator must ensure that
the scopes of facility- and site-level operations-based
exercises match the intent of the Order. Also, the exercise
program should include facilities participating in the site-
level exercise on a rotating basis and all of the emergency
management program elements validated over a multi-year
basis.
1.5 Readiness Assurance Responsibilities
As indicated in the Order, emergency readiness assurance
includes . . . evaluations (assessments) and documentation
to ensure stated emergency response capabilities are
sufficient to implement emergency plans. In the next three
sections, the responsibilities of the program
administrator(s) with respect to emergency readiness
assurance activities will be divided according to the
following topics: program plan, program implementation, and
documentation.
1.5.1 Program Plan
Readiness assurance provides a framework and associated
tools to assure emergency plans, implementing procedures and
resources are sufficiently maintained, exercised and
evaluated; and appropriate, timely improvements are made in
response to identified needs. The framework consists of
evaluations, improvements, and documentation. Emergency
management administrator(s) should develop a readiness
assurance plan that consists of evaluations (e.g., internal
and external program evaluations, exercise evaluations,
performance tests of single response tasks, tabletop tests
of decision-making) and a real-time improvement system to
ensure that findings from all evaluations (including self-
assessments) result in corrective actions that are
implemented in the program and are verified and validated.
In addition, the administrator(s) should institute a lessons
learned program to take advantage of lessons, not only from
DOE/NNSA, but also from other Federal (e.g., DHS) and
commercial activities performing similar tasks.
1.5.2 Program Implementation
To assure a quality emergency management program, persons
with knowledge of the program or response activity being
assessed should conduct an internal assessment of all
aspects of a facility or site emergency management program
annually. These assessments will be the basis for
improvements, which should be integrated into the emergency
management program. The site emergency management program
administrator(s) should coordinate the scheduling of
evaluations and assessments by external organizations to
minimize impacts and maximize benefits. Evaluation
schedules shall be forwarded to the Program Office and HQ
Associate Administrator of Emergency Operations to ensure
maximum coordination. The emergency management program
administrator(s) should coordinate the response to emergency
management evaluation findings.
The emergency management program administrator should
maintain a root cause investigation and corrective action
program that establishes and documents an integrated site
program for corrective actions, including tracking
corrective actions, and validating the adequacy of
corrective actions resulting from the annual assessments.
The program should also include specific findings and
lessons learned from training, drills, exercises, and
particularly those from actual responses and self-
assessments, even though such findings may not have the same
visibility and urgency as those associated with external
oversight. Site emergency management program administrators
should maintain an open door policy for employee concerns
regarding emergency management.
1.5.3 Documentation
The emergency management program administrator ensures the
timely preparation of facility ERAP elements for inclusion
in the site ERAP. The contributions to the site ERAP are
made on an annual basis and reflect current and projected
facility emergency management program capabilities,
resources, and requirements (e.g., personnel, facilities,
equipment, emergency planning and preparedness activities,
etc.). Guidance on this topic may be found in
DOE G 151.1-3, Appendix C.
Lessons learned from evaluations of exercises should be
included with such records to enable facility emergency
management program administrators to identify areas
requiring additional training or that could require changes
to the facility emergency plan and implementing procedures.
1.6 Response Responsibilities
The responsibilities of the program administrator(s) related
to plans and procedures associated with emergency response
have been covered in Section 1.4. However, associated with
several of the response emergency management program
elements are functions/activities that must be maintained on
a regular or periodic basis in order to be ready in the
event of an OE. The program administrator(s) must ensure
that these functions are performed regularly. DOE G 151.1-4
contains guidance related to all of the response program
elements, and, in particular, descriptions of the associated
programmatic functions. These response-related programmatic
functions/activities are identified and associated
documentation requirements are indicated in the following
sections.
1.6.1 Programmatic Activities
ERO. An adequate number of fully trained personnel, with
periodic participation in an exercise, an evaluated drill,
or an actual response, are assigned to facility- and site-
level ERO positions to ensure adequate staffing for
emergency response. The standby staffing of ERO emergency
facility positions and response teams is effectively
accomplished. ERO rosters are periodically reviewed for
accuracy (e.g., current qualifications, correct phone
number, correct response time, etc.) Communication systems
used to activate both on shift and off shift emergency
response personnel are periodically tested.
Offsite Interfaces. The emergency management program
administrator(s) should meet with local emergency planning
officials at least annually and upon significant program
change to ensure their collective understanding of the site
emergency plan and emergency plan implementing procedures.
This should occur as the documents affecting their roles,
responsibilities, and activities change or require greater
emphasis or attention, particularly in the area of emergency
categories, classifications, notifications, and protective
action recommendations.
The program administrator(s) is also responsible for the
development, review, and update of facility/site-specific
mutual aid agreements/memoranda of agreement/memoranda of
understanding (MAAs/MOAs/MOUs) relevant to a comprehensive
and effective emergency management program. These
MAAs/MOAs/MOUs routinely involve support provided to and/or
from offsite organizations or, on a multiple-facility site,
support provided to and/or from other facilities,
contractors, and/or offsite organizations. On a multiple-
facility site, MAAs/MOAs/MOUs with offsite organizations
should be developed, maintained, and updated by the site
emergency management program administrator and are typically
maintained as a part of the site emergency plan. There are
DOE/NNSA locations where DOE/NNSA retains full
responsibility for development and maintenance of agreements
with offsite organizations.
Organizations which may be needed in a supporting role
and/or needed for long-term support have been identified and
pre-designated offsite points-of-contact, including
organization, names, and telephones numbers, are documented,
maintained, and available to the response organization.
Planned response functions to be provided by offsite
organizations are periodically tested and verified.
Emergency Facilities and Equipment. Designated response
facilities, especially multi-use facilities, are adequately
maintained. Inventories of all emergency equipment and
supplies are maintained in identified locations. Periodic
inspections, operational checks, calibration, preventive
maintenance and testing of equipment and supplies are
carried out as required.
Categorization and Classification. Emergency Action Level
(EAL) sets are reviewed and tested regularly against a range
of initiating conditions and emergency event/condition
scenarios.
Consequence Assessment. A formal Quality Assurance Program
is implemented and maintained for control of the tools used
in consequence assessment.
Emergency Medical Support. Arrangements with offsite
medical facilities to transport, accept, and treat
contaminated, injured personnel are established, documented,
and periodically reviewed. Onsite and offsite medical
personnel are periodically offered information and training
on facility/site-specific hazardous materials and offered
opportunities for participation in drills and exercises.
Emergency Public Information. Workers and site personnel
are informed of emergency response plans, response
capabilities, and planned protective actions. Information
is disseminated periodically to the public regarding
facility hazards, how they will be alerted and notified of
an emergency, what their actions should be in the event of
an emergency, and points of contact for additional
information. Continuing education is provided to the area
news media for the purpose of acquainting the media with the
facility, management personnel, facility hazards, emergency
plans, and points of contact. A list of 24-hour media
points-of-contact is available and maintained current.
1.6.2 Documentation
Written MAAs/MOAs/MOUs should be developed to ensure that
the provision of support during an exercise and an actual
emergency is not dependent on the presence of specific
individuals. MAAs/MOAs/MOUs may be mutual aid or support
agreements between onsite and offsite response organizations
or may require Departmental elements or contractor
organizations to provide specific capabilities, training,
and/or information in exchange for assistance from offsite
organizations.
Copies of supporting MAAs/MOAs/MOUs between Departmental
entities and Tribal, State, and local governments or
response organizations should be maintained as an appendix
to the emergency plan. If the potential release of phone
numbers and radio call information is of concern, a table
listing just the MOAs/MAAs/MOUs with renewal dates can be
included in the emergency plan.
1.7 Document Control
The volume of information and documents that support and
define an emergency management program, together with
supporting technical information and reports, represents a
significant challenge to emergency management program
administration. A reliable document control system for
document review, approval, distribution, and change control
should be established, where none exists, or emergency
management documents should be controlled under an existing
site-wide document control system. The following list
represents some of the documents that should be managed
under a document control system:
Technical Supporting Information (e.g., diagrams,
illustrations, maps, reference documents, and technical
documents, such as risk assessments and Material Safety
Data Sheets)
Emergency Management Documents (e.g., facility/site
Hazards Surveys and/or EPHAs, Plans and Procedures,
Emergency Planning Zone (EPZ) documentation, all MOU,
MOA, MAA, and all other documents required by Orders or
other applicable laws or regulations)
Auditable Program Records (e.g., complete training and
drill records; exercise records, including participation
and evaluation reports; program assessment and evaluation
reports; and records resulting from actual emergencies;
corrective actions and associated closure
verification/validation records)
The program administrator(s) must determine the appropriate
controls to be placed on each document, based on the need
for review, approval, distribution, and change control. No
specific document control system is required, but the system
should meet industry standards. It also includes ensuring
the availability of vital records essential for the
continued functioning, operation, or reconstitution of a
site organization/activity during or after an emergency
[e.g., continuity of operations (COOP)]. DOE/NNSA
encourages the program administrator(s) to make maximal use
of technological tools, such as the Internet, to increase
document and information availability. However, the
availability of sensitive, unclassified [e.g., Official Use
Only (OUO)] facility/site documents containing information
that could be exploited by malevolent interests
(e.g., EPHAs, facility/site diagrams and maps) must be
secured following DOE/NNSA guidelines.
The program administrator(s) ensures that adequate
documentation of all technical data which supports the
emergency management program is maintained, kept current
using both hard copy and electronic media where possible,
and shared with those who require access to it. The program
administrator(s) should ensure that up-to-date and
controlled, if appropriate, copies are maintained,
information is properly distributed and/or made available or
accessible, documents are updated when needed or required,
and required supporting information is maintained. This
enables the emergency management program administrator(s) to
ensure that changes and updates are distributed to all
organizations using and/or maintaining these documents.
Copies of such documents maintained on electronic media
should be read-only, access-controlled; the specific
procedures used to address access to these electronic media
files should be determined by the respective facility and/or
site. At the same time, the emergency management program
administrator(s) must ensure that Federal and Departmental
security regulations and guidance associated with sharing
such information and documents are being met and/or complied
with.
1.8 Classified/Sensitive Information
If classified and/or sensitive information or materials are
being used or generated at/by a facility or site, the
emergency management program administrator(s) is responsible
for ensuring that required security procedures and controls
are incorporated at the appropriate facility and/or site
levels. This also includes ensuring that required security
reviews are conducted, documented, and lessons learned
implemented. The administrator also ensures that a
Derivative Classifier (DC) or an Unclassified Controlled
Nuclear Information (UCNI) reviewing official reviews
emergency management documents [e.g., plans and procedures,
Hazards Surveys/EPHAs (especially location and quantity of
nuclear materials and malevolent event scenarios), and
supporting program documentation].
APPENDIX A. Standard Format and Content of Emergency
Plans for Hazardous Material Programs
A.1 Introduction
Emergency management programs for each DOE/NNSA
facility/site and activity are documented in an emergency
plan. The plan describes provisions for response to
Operational Emergencies and activities for maintaining the
emergency management program. The recommended emergency
plan format and content for Operational Emergency Hazardous
Material Programs is provided in this chapter. The
requirements for Base Programs and Hazardous Material
Programs should be seamlessly integrated into one Emergency
Plan for the facility or site with hazardous materials that
require a more substantial level of planning and response
capabilities. Section A.2 addresses the format and content
of the Hazardous Material Program emergency plan.
A.2 Emergency Plan
This section provides a candidate format and associated
content for the Hazardous Material Program Emergency Plan.
Figure A-1 contains a recommended format for the emergency
plan. Specific content for each section in the plan
follows.
EXECUTIVE SUMMARY
Summarize the Emergency Plan by briefly stating its
purpose and a description of what is included in each
chapter.
TABLE OF CONTENTS
See Figure A-1.
EXECUTIVE SUMMARY
TABLE OF CONTENTS
1.INTRODUCTION
1.1 Purpose of Emergency Plan
1.2 Scope
1.3 Concept of Operation
1.4 Site Description
2.EMERGENCY RESPONSE ORGANIZATION (INTERNAL)
2.1 Organization Structure
2.2 Emergency Direction and Control
2.3 Emergency Management Operations
3.OFFSITE RESPONSE INTERFACES
3.1 Overview
3.2 Other Federal Agencies
3.3 Tribal Organizations
3.4 State Government
3.5 Local Organizations
3.6 Private Organizations
3.7 Mutual Aid Agreements (MAAs), Memoranda of
Agreement (MOAs), and Memoranda of
Understanding (MOUs)
3.8 Offsite Medical Facilities
4.EMERGENCY FACILITIES AND EQUIPMENT
4.1 Emergency Facilities
4.2 Emergency Equipment
5.EMERGENCY CATEGORIZATION AND CLASSIFICATION
5.1 Definitions
5.2 Criteria for Operational Emergencies Not
Requiring Classification
5.3 Emergency Action Levels (EALs)
6.NOTIFICATIONS AND COMMUNICATIONS
6.1 Notifications
6.2 Communications
7.CONSEQUENCE ASSESSMENT
7.1 Consequence Determination
7.2 Coordination
8.PROTECTIVE ACTIONS AND REENTRY
8.1 Protective Action Criteria (PACs)
8.2 Records
8.3 Protective Actions
8.4 Reentry
8.5 Emergency Planning Zones (EPZs)
8.6 Communication
8.7 Termination of Protective Actions
8.8 Shutdown of Operations
Figure A-1. Recommended Format for Operational Emergency
Hazardous Material Program Emergency Plan
9.EMERGENCY MEDICAL SUPPORT
9.1 System
9.2 Staff
9.3 Equipment
9.4 Transportation and Evacuation
9.5 Communications
10. TERMINATION, AND RECOVERY
10.1 Emergency Termination
10.2 Recovery
11. EMERGENCY PUBLIC INFORMATION
11.1 Public Information Organization
11.2 Public Information Facilities
11.3 Public Education
11.4 Public Inquiries
11.5 Security
11.6 Field and Headquarters Coordination
12. PROGRAM ADMINISTRATION
12.1 Emergency Management Program Administrator
12.2 Document Control
13. TRAINING AND DRILLS
13.1 Courses
13.2 Training, Certification, and Proficiency
Requirements
13.3 Examinations
13.4 Record Keeping
13.5 Offsite Personnel
13.6 Offsite Training Support
13.7 Offsite Personnel Training
13.8 Instructor Training and Qualification
13.9 Drills
14. EXERCISES
14.1 Exercises
14.2 Offsite Coordination
15. READINESS ASSURANCE
15.1 Self-Assessment
15.2 Corrective Action Program
15.3 Lessons-Learned Program
Appendixes
List of Figures
List of Tables
List of Acronyms
List of Definitions
Agreements
Maps
Listing of Emergency Management Personnel
References
Figure A-1. Recommended Format for Operational Emergency
Hazardous Material Program Emergency Plan (contd)
1. INTRODUCTION
1.1 Purpose of Emergency Plan
State that the purpose of this Emergency Plan is to
provide the Departmental community with an effective
and efficient emergency management operation that will
provide acceptable levels of protection. For example,
the Emergency Plan provides an efficient and effective
response operation that, should an emergency occur,
will protect the health and safety of workers,
responders, the public, and the environment. Identify
the Departmental Orders and legislation that require
this plan and that this Emergency Plan satisfies.
Describe the operational use of the Emergency Plan and
Emergency Plan Implementing Procedures (EPIPs).
1.1.1 Update of Emergency Plan
State the process and time table for Emergency Plan
updates, including required periodic updates and
updates made necessary by changes in emergency planning
or facility operations and/or hazards.
1.1.2 Distribution of Copies
Identify which organizations, Departmental and non-
Departmental, are to receive copies of the Emergency
Plan.
1.2 Scope
Identify the types of emergencies, per DOE O 151.1C, to
which this Emergency Plan applies and does not apply.
(Emergency Plans generally are for Operational
Emergencies and not Energy or Emergency Assistance
emergencies.) Identify the boundaries and define the
site to which this Emergency Plan applies (i.e., site
emergency plan versus building or facility Emergency
Plan).
1.3 Concept of Operations
Describe the concept on which site-wide emergency
planning is based. Identify the documents, reports,
surveys, and assessments used to develop this Emergency
Plan, or refer to where this information can be found
in the Emergency Plan.
1.4 Site Description
1.4.1 Overview Site Description
Identify the overall function and mission of the site.
Broadly describe the site and the buildings and
facilities within the site. Use maps and other
graphics/diagrams as appropriate to describe the site.
Do not use classified information.
1.4.1.1 Detailed Facility Description
In detail, describe the specific facilities that, by
the nature of the hazards present on those facilities,
could cause an emergency to be declared. Do not use
classified information in the facility description.
Identify the maximum number of employees in each
facility described. Provide facility floor plans where
appropriate.
1.4.1.2 Hazard Survey and Hazards Assessment
A Hazards Survey shall be used by the site, facility,
or activity Emergency Manager to identify the planning
requirements addressed in the Operational Emergency
Base Program. A Departmental site, facility or
activity may then be required to establish and maintain
a quantitative Emergency Planning Hazards Assessment
(EPHA). The quantitative EPHA analyzes hazards
significant enough to warrant consideration in a
facility/site or activity Operational Emergency
Hazardous Material Program.
Include or summarize the results of the Hazards Survey.
Describe known hazards originating outside the
Departmental facility that could impact the health and
safety of onsite personnel or other Departmental
interests.
List and/or summarize the significant radiological and
non-radiological hazards present. Describe the system
for updating the EPHA. The EPHA may be included as a
separate appendix, if desired or necessary for
completeness. Identify technical supporting documents
that describe the methodology and information of EPHAs
used as the bases for emergency planning. Describe the
hazards associated with leased facilities (if
applicable).
1.4.1.3 Contractors
Identify the major contractors and their contractual
commitments and responsibilities.
1.4.1.4 Leased Facilities on Site (if applicable)
Identify facilities onsite that are leased to others,
including contractual arrangements and agreements.
Identify emergency management agreements and interfaces
with the site emergency management program.
1.4.2 Physical Attributes of the Site
1.4.2.1 Geography
Identify the state, county, and any other appropriate
local subdivision in which the site is located.
Discuss the site location with respect to prominent
natural and man-made features such as rivers, lakes, or
dams. Describe land use of surrounding area. Discuss
any groundwater features. Identify other vital
features, such as fault lines or flood plains.
1.4.2.2 Topography and Geology
Briefly describe the terrain of the site and the
surrounding area, including ground cover and
elevations. Describe the geology of the site and the
surrounding area, particularly as it relates to
possible seismic activity.
1.4.2.3 Population Distribution
Describe the surrounding area (offsite) population,
including population density. Provide maps identifying
potentially affected onsite population groupings and,
based on the most recent census information available,
offsite populations to distances of 10 and 50 miles
from the site boundary. Discuss projected population
growth or change trends and the basis for these
projections.
1.4.2.4 Meteorology
Briefly describe the general climate of the region,
including types of air masses, synoptic features (high-
and low-pressure systems and frontal systems), general
airflow patterns, temperature and humidity,
precipitation, and relationships between synoptic-scale
atmospheric processes and local meteorological
conditions.
1.4.2.5 Natural Phenomena
Describe seasonal and annual frequencies of severe
weather phenomena, including hurricanes, tornadoes, and
waterspouts, thunderstorms, lightning, hail, severe
drought, and high air pollution potential. Describe
the potential for earthquakes and floods.
1.4.2.6 Transportation System
Describe major public and private transportation
systems used by employees and the surrounding public.
This includes waterways, airports, rail systems; major
highways located on, through, and near the site and
major local access routes. Describe any transportation
systems operated within the site. Describe any
transportation interfaces required for site
evacuations.
1.4.2.7 Utility System
Describe the public and private utility systems used by
the site that would be affected by an actual emergency
or declaration of an emergency. Describe how the
utilities would be affected and the effect on the
surrounding population. Identify and describe any back-
up utility systems present and the plan for their use.
2. EMERGENCY RESPONSE ORGANIZATION (INTERNAL)
2.1 Organization Structure
Generally describe the overall organizational structure
of the site and describe in detail the emergency
response organization, including its relationship to
the overall structure. Figures, diagrams, and
organization charts may be used to show lines of
authority between the various government officials, the
emergency manager, and heads of various departments.
Specifically delineate the functions, authorities, and
responsibilities of all internal organizational
elements with emergency responsibilities. Outline the
relationship of all emergency organizations to each
other, with Departmental (field and Headquarters) and
other Federal, Tribal, state, and local organizations.
List all committees with emergency management or
emergency planning roles and responsibilities.
Describe the purpose and make-up of each committee.
Include both onsite and offsite committees in which
employees serve either as a working member, a
participant, or an observer. List the members, the
authority and responsibility of each committee, and the
authority and position of each member. Identify the
meeting frequency and any other pertinent details to
describe the committee.
2.2 Emergency Direction and Control
Delineate the site chain of command in the event of an
emergency. Discuss the organizational structure,
authorities and responsibilities, and roles played by
each position. Include an organization chart
specifying, at a minimum, the positions responsible for
emergency direction and control, both during routine
operations and emergency conditions. Identify the
succession of authority for emergency positions.
2.3 Emergency Management Operations
Describe the actions and activity for the following:
Declaration of an Operational Emergency
Activation of the command center or Emergency
Operations Center (EOC)
Emergency response
Reentry
Emergency termination
The actions describing the activation of the command
center or EOC include the time required for staffing
(during both normal duty hours and non-duty hours) and
the minimum positions required for activation. Note
that this section is not intended to include detailed
emergency procedures, which are contained in separate
implementing procedures. Describe each position's
emergency management responsibilities, its place in the
overall organization, and the authority and
responsibility of each position.
3. OFFSITE RESPONSE INTERFACES
3.1 Overview
Provide an overview of relationships, both formal and
informal, with offsite organizations, including other
Departmental elements and other Federal government,
Tribal, state, and local organizations with emergency
management or emergency planning responsibilities.
MAAs, MOAs, and MOUs should be described in this
section.
3.2 Other Federal Agencies
Describe agreements with other Federal agencies,
specifying the role of the agency, potential response,
regulatory control, and notification chain required.
Discuss the relationship of the organization in the
activation of the National Response Plan (NRP) and its
role in the National Incident Management System (NIMS).
Examples of Federal agencies that may be involved in a
Departmental response are as follows:
Department of Defense
Defense Threat Reduction Agency
Department of Homeland Security
U.S. Forest Service
Federal Bureau of Investigation
Federal Aviation Administration
U.S. Coast Guard
U.S. Environmental Protection Agency
Nuclear Regulatory Commission
National Oceanic and Atmospheric Administration
3.3 Tribal Organizations
Describe the roles of Tribal organizations with
emergency response or regulatory control
responsibilities relevant to Departmental facilities
and/or sites. Summarize primary and secondary support
roles. Describe Tribal emergency plans or procedures
that affect the Departmental facility or program.
Specify the nature of any MAAs, MOAs, or MOUs with the
local tribal organizations.
3.4 State Government
Describe the roles of state organizations with
emergency response or regulatory control
responsibilities relevant to Departmental facilities
and/or sites. Summarize primary and secondary support
roles. Describe emergency plans or procedures with
impact upon the Departmental facility or program.
Specify the nature of any MAAs, MOAs, or MOUs with the
State.
3.5 Local Organizations
Describe the roles of local organizations with
emergency response or regulatory control duties as they
pertain to Departmental facilities and/or sites.
Summarize primary and secondary support roles.
Describe local emergency plans or procedures that
affect the Departmental facility or program. Specify
the nature of any MAAs, MOAs, or MOUs with the local
authorities.
3.6 Private Organizations
Describe the roles of private organizations with
emergency response responsibilities relevant to
Departmental facilities and/or sites. Summarize
primary and secondary support roles. Describe private
emergency plans or procedures that affect the
Departmental facility or program. Specify the nature
of any MOAs or MOUs with the local private
organizations. Describe any contractual arrangements
and annual funding obligations in order to maintain the
desired level of emergency preparedness.
3.7 MAAs, MOAs, and MOUs
List all MAAs, MOAs, and MOUs with offsite
organizations. Include in the list the parties to the
agreement, points of contact, the date of the
agreement, and the expiration date of the agreement.
Identify all organization(s) responsible for
negotiating, executing, and maintaining agreements.
Specify where documents are on file, and include copies
of the unclassified MAAs, MOAs, and MOUs in an appendix
to this Emergency Plan. List all classified MAAs,
MOAs, and MOUs, identify unclassified points of
contact, and state where the agreement can be viewed.
3.8 Offsite Medical Facilities
Discuss capabilities of local medical centers to
support mass casualties and contamination events.
4. EMERGENCY FACILITIES AND EQUIPMENT
4.1 Emergency Facilities
List and provide a brief description of the following
facilities. Distinguish between dedicated and non-
dedicated facilities. Maps and floor plans of
facilities should be included when a complete
description of the facility will be useful in a
response.
4.1.1 EOC or Command Center
4.1.2 Alternate or Secondary EOC
4.1.3 Emergency Response Facilities
4.1.4 Technical Support Center
4.1.5 Primary and Alternate Onsite JIC
4.1.6 Offsite Communications Center
4.1.7 Decontamination Facilities
4.1.8 Medical Facilities
4.1.9 Security Control Centers
4.2 Emergency Equipment
List and describe the equipment likely to be used for
responding to emergencies. Include in the list:
equipment capability and limitations, quantity of
equipment, locations (both fixed and portable
equipment), consumables, maintenance requirements,
certification requirements, expiration dates, and
computer/communications compatibilities.
4.2.1 Communications Equipment
4.2.2 Heavy Construction Equipment
4.2.3 Decontamination Equipment
4.2.4 Alarm Equipment
4.2.5 Rescue Team Equipment
4.2.6 Sanitation and Survival Equipment
4.2.7 Transportation Equipment
4.2.8 Personnel Protection Equipment
4.2.9 Gas and Liquid Monitoring Equipment
4.2.10 Damage Control Equipment
4.2.11 Fire Fighting Equipment
4.2.12 Emergency Power Equipment
4.2.13 Logistics Support Equipment (maps, plans,
etc.)
5. EMERGENCY CATEGORIZATION AND CLASSIFICATIONS
5.1 Definitions
State the definitions of Operational Emergencies and
emergency classes per DOE O 151.1C. In the interest of
consistency, the definitions as provided in the
Departmental Orders can be repeated.
5.2 Criteria for Operational Emergencies Not Requiring
Classification
State the criteria used to define an emergency.
Briefly describe the methodologies used to develop
criteria and reference specific technical supporting
documents.
5.3 Emergency Action Levels (EALs)
Identify the EALs used to define an emergency. Briefly
describe the methodologies used to develop EALs and
reference technical supporting documents. The EALs
should be described for all potential emergencies at
the facility or site, including radiological, non-
radiological, terrorism, sabotage, fire, explosion,
security, and natural phenomena. Describe the criteria
for each classification of emergency at the facility or
site. Identify personnel (positions) responsible for
determining the classification and action level.
Discuss the level of emergency staffing required at
each level. Describe how the EALs are incorporated
into and integrated with the facility procedures that
govern response to alarms and/or abnormal events.
Identify where the complete EALs are kept on file.
6. NOTIFICATIONS AND COMMUNICATIONS
6.1 Notifications
Discuss the required and proceduralized notification
process for onsite and offsite notifications for all
operational emergencies. Specify time limits in which
notifications are required, and the authority for the
time limit. Identify personnel (positions) responsible
for both initiating and receiving notifications.
Discuss the method of notification (e.g., beepers,
telephone). Discuss notification procedure for
termination of an incident. Discuss the procedure
variance for classified notifications. Include copies
of all notification record forms, particularly those
forms used in response to DOE O 231.1A and its
successors.
6.1.1 Offsite Notifications
Identify the applicable requirements for notification
and communication with appropriate offsite agencies and
organizations, including, at a minimum, Tribal
government; state government; local government; local
fire, police, and medical organizations; private
organizations; contractor organizations; other Federal
agencies; and any organization for which an agreement
of notification has been signed.
6.1.2 Onsite Notifications
Identify personnel (positions) required to be notified
for any emergency, specifying any differences for day
shift or night shift. Discuss, if appropriate, the
Duty Officer Program and specific responsibilities.
6.1.3 Departmental Radiological Emergency Response
Assets
Identify the notification procedure for requesting
Departmental radiological emergency response assets,
and the specific circumstances under which notification
is permitted or required.
6.1.4 Field EOC and Headquarters Operations Center
Notifications
Identify the circumstances under which the
operations/field/site EOCs and/or the Headquarters
Operations Center are notified of an emergency and
describe the procedures for notification, including the
responsible personnel.
6.2 Communications
Describe the communications systems and equipment
employed by emergency personnel at the site or any
specific facility for any notifications, sirens, or
warnings to the public, including a description of
primary and alternate systems. Discuss communications
interface with offsite organizations; describe the
integration of the sites communications with offsite
response resources, such as the police, fire and
offsite EROs.
Identify what portions of the system are dedicated to
the Emergency Management System. Describe the
equipment, back-up equipment, readiness assurance, and
testing procedures. Describe the troubleshooting
system for ensuring that problems noted during tests
and drills are identified, tracked, and resolved.
Reference to any listing of communication equipment in
the Emergency Equipment chapter is acceptable.
Describe the procedures and plans for communicating
classified information.
7. CONSEQUENCE ASSESSMENT
7.1 Consequence Determination
Describe the procedure(s) used to determine the
potential consequences based on the results of hazard
assessments and input from all other pertinent areas,
such as intelligence and meteorological information.
Describe the methodologies used for consequence
assessment and reference technical supporting
documentation.
Describe the expected utilization of the National
Atmospheric Release Advisory Center (NARAC)
capabilities during a response. Describe the
procedures for continually (and in real time, where
appropriate) monitoring an emergency or a continuing
situation to update the consequence assessment.
Describe the processes for initiating and performing
field monitoring for both radiological and chemical
releases. When appropriate, include a discussion of
any special circumstances associated with coordination
and execution of offsite field monitoring.
7.2 Coordination
Describe the procedure to coordinate with other
Federal, Tribal, State, and local organizations
information necessary to make accurate and timely
consequence determinations.
8. PROTECTIVE ACTIONS AND REENTRY
Identify the purpose and intended use of protective
actions. Describe protective actions used at the
facility/site or activity and under what circumstances
they are implemented.
8.1 Protective Action Criteria (PACs)
8.1.1 Radiological PACs. List and summarize
existing radiological Protective Action Guides
(PAGs). Reference applicable supporting technical
documentation.
8.1.2 Chemical PACs. List the AEGL/ERPG/TEEL used
for chemicals involved in potential Operational
Emergencies. Reference applicable supporting
technical documentation.
8.2 Records
Describe the procedures and the responsible
organization tasked with maintaining an accurate log of
the events of the emergency, including all follow-up
health and hygiene surveys. Describe the coordination
procedures with medical personnel and facilities.
Identify the length of time and method of storing the
records.
8.3 Protective Actions
Present the assumptions for the development of
protective actions for both offsite and onsite
populations. Discuss what constitutes potential
protective actions at the site, such as sheltering-in-
place, monitoring activities, and accounting of
personnel. Discuss the process for implementing the
protective actions. Discuss the procedures for
ensuring that the protective actions are timely,
communicated, safe, and complete. Identify the
notification process and responsibilities.
Describe the considerations used to determine whether
shelter-in-place or evacuation is appropriate. Identify
the notification process and responsibilities. Discuss
conditions requiring shelter-in-place. Discuss the
method(s) for implementing shelter-in-place and for
accounting for personnel. Identify the location of
shelters.
Discuss conditions requiring evacuation (full or
partial). Identify onsite evacuation routes and
include maps. Discuss the method for collecting and
housing the evacuated individuals. Describe access
control procedures for evacuated areas. Discuss the
method and procedures for accountability of onsite
personnel and visitors.
8.4 Reentry
Describe the plan and criteria for reentry at each
facility, where applicable, for the entire site, and
identify all reentry plans. Identify and discuss,
where appropriate, the criteria for reentering areas
under emergency conditions or which had restricted
access during the emergency. Describe the procedure
used to assess damage and/or contamination. Identify
personnel who develop, approve, or implement reentry
and indicate their relationship to the emergency
organization. The reentry plan shall also include:
methods for protection of workers from hazardous
exposure, exposure guides for rescue personnel,
facility accessibility, security considerations, access
to protective clothing and equipment, availability of
medical assistance, and debriefing procedures. Provide
references to technical supporting documentation if
applicable. Note that some activities of reentry may
be relevant to recovery.
8.5 Emergency Planning Zones (EPZs)
Describe the procedures and/or the predetermined
emergency planning zones in determining potentially
affected areas. Use maps, as appropriate, for an
accurate and complete description. Identify the
persons (positions) responsible for determining and
recommending protective actions for the public within
the plume exposure EPZ and receiving protective action
recommendations from the site.
Specify the evacuation routes to be used in an
emergency. Discuss sheltering and evacuation plans for
the EPZ. Define the size of the plume EPZ limit,
specifically noting what portions of the EPZs fall
onsite and offsite. Describe the exposure pathways.
Describe conditions, procedures, and authorities for
evacuation of local populations.
Describe the ingestion pathway planning zone. Identify
the persons (positions) responsible for determining and
recommending protective actions for the public within
the ingestion pathway planning zone.
8.6 Communication
Describe the communications to notify other Federal,
Tribal, State, local, and private organizations of
necessary actions required for their protection or for
which they are responsible for informing the public or
otherwise need to take action. Define and list, if
necessary, sources of information used by Federal,
Tribal, State, and local organizations in further
determining their course of action.
8.7 Termination of Protective Actions
Describe how protective actions are lifted or modified,
the authorities for removal of protective actions, how
this information is communicated, both onsite and
offsite, and how the activity is accomplished.
Describe any post-emergency communications or follow-up
actions.
8.8 Shutdown of Operations
Describe the system to ensure safe shutdown of facility
and/or site operations following the declaration of an
emergency.
9. EMERGENCY MEDICAL SUPPORT
Describe the medical capabilities available onsite and
offsite to respond to an emergency.
9.1 System
Describe the onsite medical care organization
responsible for medical care for managing injured
and/or contaminated personnel. Describe the onsite
medical care capabilities and facilities. Discuss
roles, responsibilities, and procedures for treatment
of radiological and chemical exposures (e.g.,
radiological prophylaxis). Describe the provisions in
place to ensure coordination among onsite medical,
industrial hygiene, health physics, environmental
response, security, and management personnel during
emergencies.
9.2 Staff
Identify the lead medical emergency director. Describe
the staff available both permanently and on call,
outlining qualifications and training required.
Identify the minimum requirements for offsite medical
assistance including contractual arrangements and
offsite staff training requirements.
9.3 Equipment
Describe the health services available onsite and
offsite for response to emergencies. Describe the
equipment available for extrication, rescue, and
transport of injured personnel. Describe the onsite
facilities and equipment for decontamination of injured
personnel. Describe the equipment available for
bioassay and whole body counting. Identify the types
of medical supplies maintained onsite and any special
equipment maintained offsite for emergencies. Describe
how the quality and quantity of these supplies are
determined, maintained, and ensured.
9.4 Transportation and Evacuation
Describe the transportation and evacuation
capabilities, equipment, and process for moving
contaminated and uncontaminated casualties. Identify
person/positions with responsibility and authority for
evacuation of injured or ill personnel.
9.5 Communications
Describe the communications procedures in place for
emergencies. Identify the persons/positions
responsible for notifying emergency medical teams,
security, administration, offsite hospital and offsite
emergency services.
10. EMERGENCY PUBLIC INFORMATION
Describe the program to provide information concerning
the emergency to the media and the general public,
including information release approval. Identify the
recommended time requirements for information release.
10.1 Emergency Public Information (EPI) Organization
Describe the organization, including the relationship
to the overall emergency organization, which will be
used to disseminate information to the media and the
general public. Identify the personnel within the
Public Information Office who are authorized to release
information (e.g., to employees and their families,
media, and the public), including the designated
spokesperson.
10.2 Public Information Facilities
Describe the facilities and communications equipment
used to disseminate information to the public. Include
meeting rooms, press areas, and communications
facilities. Describe the function and staffing of the
Joint Information Center (JIC). Discuss the
coordination roles at the JIC (both onsite and
offsite).
10.3 Public Education
Describe the public education program and methodology
for informing workers and the public of potential
hazards at DOE/NNSA sites and providing information on
emergency plans and protective actions before and
during emergencies as well as how they will be notified
of the protective actions, including recommended
evacuation routes and sheltering.
10.4 Public Inquiries
Describe the plan to respond to public and worker
inquiries, including rumor control.
10.5 Security
Describe the plan to ensure that security is not being
compromised with the release of sensitive or classified
information to the public.
10.6 Field and Headquarters Coordination
Describe the plan to coordinate with the
operations/field/site office and Headquarters on the
release of information to the public.
11. TERMINATION AND RECOVERY
Describe the plan and criteria for declaring the
emergency condition terminated and transitioning to
recovery at each facility, where applicable, and for
the entire site, and identify all termination and
recovery plans. The plan includes termination
authority and responsibility and recovery criteria for
protection of workers and the general public from
hazardous exposure, exposure guides for recovery
personnel, facility accessibility, security
considerations, access to protective clothing and
equipment, availability of medical assistance, and
requirements for establishing the recovery
organization. Provide references to technical
supporting documentation if applicable.
11.1 Emergency Termination
Describe the procedures for terminating the state of
emergency, including the personnel responsible for
decision-making and their relationship to the overall
emergency organization described in Chapter 2 of the
emergency plan. Address the special circumstances of
an error in initial categorization that necessitate an
emergency downgrade. Describe the conditions or
identify the document under which the emergency may be
terminated and initiation of recovery activities may
occur.
11.2 Recovery
Describe the recovery (transition) process from an
emergency condition to the restoration of a safe, pre-
emergency environment. Discuss the plan to restore
vital systems, such as power, water, and
communications. Include a discussion of the areas that
must be verified for safety, such as fire hazards,
toxic gas, and radiation. Describe the measures taken
to ensure that security procedures are maintained.
Describe the continued recovery (transition) process
from a safe environment to the pre-emergency
conditions.
Describe the recovery organization and the authority
and responsibility of the chain of command that
restores pre-emergency conditions. Describe how this
organization may differ from the emergency organization
described in Chapter 2. Describe the plan, either here
or in Chapter 10, to notify the media and the public
about the conditions of emergency recovery.
12. PROGRAM ADMINISTRATION
12.1 Emergency Management Program Administrator
Provide the name, position, mailing address, and
telephone number of the Emergency Management Program
Administrator at the facility and/or site level. (This
information should also be provided in the appendix
that lists the emergency management personnel.)
Indicate, where appropriate, whether the Emergency
Management Program Administrator has been given
emergency management responsibility through delegation
of authority.
12.2 Document Control
Identify the procedure used to control dissemination of
or access to the facility or site Emergency Plan and to
assure its annual review and update.
13. TRAINING AND DRILLS
Describe the goals and objectives of the facility
and/or site training and drills program. Describe the
overall approach to the design of the training and
drill program, including training analysis methodology,
overall curriculum design, and qualifications.
13.1 Courses
List the available courses for emergency response
planning, preparedness, and analysis, including title,
length of course, target audience, a brief summary, and
the periodicity or schedule.
13.2 Training, Certification, and/or Proficiency
Requirements
Describe courses given to emergency management
personnel. Identify training, certification, and/or
proficiency requirements for key emergency management
positions and response teams. Identify periodicity of
courses and employee requirement for training and
retraining or refresher training.
13.3 Examinations
Describe the examinations, if any, required for
emergency response organization personnel qualification
and/or certification and for documenting individual and
team proficiency.
13.4 Record Keeping
Describe the system of record keeping for verifying
that training and proficiency requirements are met.
13.5 Training for Onsite Public
Describe the system of training available to and
required for visitors, vendors, and subcontractors.
13.6 Offsite Training Support
Describe the available offsite training resources
available to onsite emergency response organization
personnel, which can substitute for or complement
existing onsite training courses and/or meet training,
certification, and/or proficiency requirements.
13.7 Offsite Personnel Training
Describe the in-house training available to offsite
organizations in order to support their abilities to
participate in site emergency response actions.
Describe training available, if any, for the general
public. Describe procedures for documenting attendance
of offsite personnel at training.
13.8 Instructor Training and Qualification
Describe the plan to provide qualified instructors for
the onsite training available to emergency response
organization personnel and the required qualifications
of such instructors, including training courses for
instructors.
13.9 Drills
Describe the drill program, per DOE O 151.1C, including
the goals, frequency, complexity, and integration of
lessons learned into emergency planning. Describe how
the drills develop expertise and proficiency in
performing emergency activities such as notification,
communication, fire control, medical planning, and
Hazardous Materials (HAZMAT) response. Describe how
drills will be controlled and evaluated, and how
lessons learned from drills, improvements, and/or
corrective actions, are incorporated into emergency
planning.
14. EXERCISES
Discuss the intended purpose of the exercise program.
14.1 Exercises
Describe the emergency management exercise program and
how it conforms to the requirements of DOE O 151.1C and
any other applicable Federal, State, and local
legislative-based regulation. Describe how exercises
will be controlled and evaluated and how lessons
learned from exercises (improvements and/or corrective
actions) are incorporated into emergency planning.
14.2 Offsite Coordination
Describe the method of coordination with Headquarters
and participating Federal, Tribal, State, local, and
private organizations for drill or exercise planning,
and the level of participation.
15. READINESS ASSURANCE
Describe the procedures for developing a structured
readiness assessment program, including program and
exercise evaluations (e.g., self-assessments, external
evaluations, performance indicators).
15.1 Self-Assessment
Describe the site internal assessment program, which
requires an internal assessment to be conducted
annually.
15.2 Corrective Action Program
Describe site validation and verification procedures
for Corrective Actions and the tracking system used to
monitor Corrective Action Plan schedules and
milestones.
15.3 Lessons-Learned Program
Describe the program responsible for collecting
relevant site-wide and community-wide lessons learned,
evaluating them and identifying potential applications,
and implementing the lessons learned in site processes
and activities.
Appendixes
List of Figures
List of Tables
List of Acronyms
List of Definitions
Agreements
Maps
Listing of Emergency Management Personnel
References
2. TRAINING AND DRILLS
2.1 Introduction
The purpose of this chapter is to assist DOE and NNSA field
elements in complying with the DOE O 151.1C requirement to
ensure that a coordinated program of training and drills for
developing and/or maintaining specific emergency response
capabilities is an integral part of the emergency management
program. The program must apply to emergency response
personnel and organizations that the facility/site expects
to respond to onsite emergencies. The Order further
requires that emergency-related information must be
available to offsite response organizations.
This chapter is designed primarily for facilities/sites and
activities that are required to implement an Operational
Emergency Hazardous Material Program and is directed at
operations and emergency management staff at Field Elements
and operating contractor organizations that are responsible
for DOE and NNSA facilities/sites and activities.
2.2 General Approach
DOE/NNSA emergency management training programs must ensure
that personnel are prepared to respond to, manage, mitigate,
and recover from emergencies involving hazards associated
with facilities and onsite activities.
A comprehensive and systematic training program should be
established to accomplish emergency management training
goals. The training program should provide a current and
structured view of program-specific training requirements
and also address position-specific requirements for all
primary and alternate personnel assigned to the emergency
response organization. Minimum program standards should be
defined for emergency responder position training,
proficiency, performance, and refresher training. The
program should be integrated and coordinated with related
training programs provided by other site organizations.
Training courses should be performance-based and include
testing to validate learning. The program should ensure
that instructors are qualified in both instructional skills
and technical competency for the training subject.
Training documentation and records should be formally
managed and controlled to ensure that training programs
support current emergency plans and requirements and that
training records are maintained for instructors and for all
personnel assigned Emergency Response Organization (ERO)
positions. Drill and exercise participation and performance
should be documented for each member of the ERO.
Requirements for initial and periodic refresher training
should be identified for all emergency response organization
personnel. This should include special team training for
functional groups with technical and management assignments
and training for decision makers to ensure they can perform
duties promptly and accurately. Training needs should also
be addressed for offsite emergency response personnel and
organizations that are expected to support onsite response
to emergencies. This includes training on facility and site-
specific hazards and emergency plans and participation in
training and drills to ensure integration of onsite and
offsite response resources.
Emergency drills should be developed, scheduled, and
conducted to provide supervised hands-on training and
validation of classroom training for emergency responders
and to provide practical training on interface between site
groups that support emergency response. Drills should be
developed based upon feedback from actual events and
exercise experience, to validate corrective actions from
program evaluations, and to validate new or revised
procedures and equipment or facility changes.
The Department of Homeland Security (DHS) employs a broad
definition of exercise that focuses on many of the general
functions attributed to the training and drills programs
[cf. DHS Homeland Security Exercise and Evaluation Program
(HSEEP)], as described in this emergency management
guidance. For example, the drill is considered an
exercise that can be used for training or testing
performance in a single or limited functional area. The
broad range of purposes attributed to exercise activities
described in the HSEEP series emphasizes the design of
exercises to familiarize personnel with plans and
procedures, achieve teambuilding, build consensus, examine
contingencies, solve problems, evaluate functions, measure
resources, and examine interfaces. Although these training
aspects of drills are emphasized in DOE guidance, the term
is commonly used throughout the DOE/NNSA complex as a small-
scale exercise (e.g., facility-level exercise).
This chapter of the EMG provides a system-based approach to
emergency management training and is organized into the
following sections:
Training program management
Training needs assessment for onsite and offsite
personnel
Training requirements
Training development
Training delivery
Training drills/practical applications
Sites should reference DOE G 414.1-2A for information on
training plans, training effectiveness, qualification of
personnel, and management responsibility regarding training
programs and also reference DOE O 360.1B, Federal Employee
Training.
2.3 Training Program Management
Effective management of a training program requires a formal
training plan be developed that describes program goals and
objectives, organizational responsibilities, resources, and
planned activities. To accomplish the elements of the
program plan, a schedule of development, delivery, and
evaluation activities should be developed and updated as
needed. Annual internal assessments of training development
and implementation identify needed improvements in the
program. Trainer/instructor qualifications should be
established and updated to reflect changes in instructional
techniques as well as relevant technical disciplines.
Evaluation of the training staff ensures their appropriate
skill levels and the knowledge base. Documentation of
training requirements and lesson plan reviews ensure that
the course materials meet expectations for the subject
positions. Finally, a system for managing emergency
responder training records ensures that staff personnel on
ERO rosters are trained for the positions assigned.
2.3.1 Program Plan
A comprehensive and systematic training program plan
includes the following:
A full description of training program goals and
objectives, compliance with requirements, and
administrative policies and procedures
Identification of current training needs for all
emergency responder positions
Identification of training resources, staff, facilities,
and reference material to support training activities
Schedule for training activities, including development,
delivery and evaluation of training programs and courses
Description of the process for identifying and
documenting training needs for emergency responders
Requirements for ERO qualification and re-qualification,
including retraining and remedial training
The program plan should also identify administrative
processes that support the systematic approach to training.
Such processes should be identified for the following
elements of training management:
Identifying training program approval and signature
authority
Establishing a matrix of training requirements for ERO
positions
Identifying methods for selecting qualified instructors
and establishing a list of training staff qualified to
teach each course or program
Describing how training records are maintained in a
manner that can be audited
Describing how refresher training addresses the details
of program changes and lessons learned from actual
events, exercises, and program evaluations
Describing how the emergency management training program
is integrated and coordinated effectively with related
training provided by other organizations (e.g., confined
space entry training, radiological monitoring training,
and hazard communication training)
The training program plan should address training for all
primary and alternate personnel assigned to the facility-
and site-level ERO. A training program plan typically
distinguishes the following levels of training requirements:
Initial training to qualify for a position on the ERO
Refresher training to maintain competency and receive
information on changes and lessons learned related to
required knowledge and skills
Remedial training to correct deficiencies in performance
or testing related to ERO positions
Annual participation in performance-based training
methods such as drills, simulations and exercises in
order to maintain ongoing proficiency and skills
An effective way to illustrate the emergency management
organizations training plan is to use a matrix to list ERO
positions and specific training required for each position.
The matrix is both an internal tool for tracking positions
and training as well as a tool for satisfying external
evaluators of the completeness of the program.
2.3.2 Schedule
A schedule for developing, delivering, and evaluating
training activities should be developed and updated as
needed. The schedule should provide a current and
structured view of program-specific training requirements,
including a detailed list of courses and drills provided by
the emergency management department, as well as dates for
scheduled implementation. Internal program assessments
should be also indicated.
2.3.3 Program Assessments
Internal assessments of training development and
implementation should be performed once a year as part of
the required annual assessment of the overall emergency
management program. A process should be identified to
ensure that recommendations from training assessments and
lessons learned from previous training drills are
incorporated into future training development and
implementation efforts. Internal assessments should aim to
improve training programs, including administration,
development and delivery. The site internal corrective
actions tracking system provides a convenient tool for
ensuring that identified corrections are made to the
program.
2.3.4 Trainer/Instructor Qualifications and Evaluations
Qualifications. Each training program should develop a list
of requirements for qualifying instructors/trainers. These
requirements should be reviewed and updated periodically to
keep pace with changes in instructional techniques as well
as relevant technical disciplines. Two primary
qualification areas to be addressed in instructor
requirements are as follows:
Instructional skills - These are skills related to the
imparting of knowledge, regardless of the subject.
Examples include adult learning methodologies,
presentation skills, and training in the use of various
instructional media, such as video and on-line computer
instruction.
Technical knowledge and experience - Adequate
understanding of theory, practical knowledge, and
experience in the content area are needed. Technical
competency is based on instructor credentials, job
references, and demonstration of technical expertise.
Proficiency in instructing the subject areas should also
be evaluated as part of instructor qualification.
Examples of areas where technical or subject-specific
expertise are necessary include dose assessment,
emergency medical and emergency public information.
A schedule should be established to ensure continuing
education and professional development of emergency
management trainers/instructors in their areas of expertise.
Evaluations. Management should also conduct internal
reviews of the training staff as part of the annual
assessment of the training program. These evaluations
should demonstrate the following:
The instructor methods are consistent with the site
training program standards and are appropriate to the
course objectives.
Instruction adheres to the documented lesson plan and
evaluation.
Subject-matter knowledge and experience are appropriate
for course content.
Instructional presentation styles are appropriate and
support course methodology.
Instructor-related feedback/ratings from course
evaluation forms are analyzed and documented.
Post-training evaluations of instructors are analyzed and
documented.
Instructor deficiencies identified during the evaluation
should be corrected and documented within a specified
period. Input from the evaluation should also be used to
improve knowledge, skills, and abilities of the staff.
Documentation supporting the staff qualifications should be
maintained in a manner that may be audited. Documentation
should include the following:
A matrix of staff positions, including requisite
education and experience cross-referenced with each
training staff member;
Qualification records; and
Feedback and post-training evaluations
2.3.5 Course Documentation
All documentation for a particular training program should
be kept in either hard copy or electronic format. Files
should be organized by date, iteration, or topic. Course
history files should include rosters/attendance sheets,
evaluations of knowledge and performance, and lesson plans
and tests.
Training Requirements - Training requirements for each
emergency response position should be documented and
reviewed and updated on an annual basis. This ensures
training requirements are still relevant to that position
and provides an opportunity to add any new requirements
assigned to the position. For example, should hazards
change or regulatory requirements for select training
change, the organization may need to revise training
requirements for some emergency response positions.
Lesson Plan Reviews - Each site is responsible for
maintaining current, documented lesson plans for all site-
specific training developed by that site. Lesson plans
include course documentation of classroom training, on-
the-job training (OJT) programs, practical drill training
and computer-based training. Lesson plans should be
reviewed prior to their use. This process includes a
review by a subject matter expert (SME) to ensure that
information contained in lesson plans continues to be
consistent with current procedures and practices and
remains applicable to DOE emergency management. Lesson
plans should be updated prior to use if there have been
changes to the emergency plan and related procedures
since the last annual review. Updates should include the
dated signature of the SME.
2.3.6 Emergency Responder Training Records
A system for managing emergency responder training records
should include a means for tracking the following:
Course attendance and completion
Status of individual emergency responder qualifications
Scheduled training, including a system for reminding
employees and program administrators when training is
needed
In-house and external training
Training dates, location, length, and name of the
instructor
Participation in emergency drills and exercises
Training records should also include all documentation
supporting the implementation of training developed by a
DOE/NNSA facility/site or activity. Such documents include
the following:
Memos relating to scheduled and canceled training or
training exemptions
Certificates for training conducted outside of DOE
Course and program evaluations
There have been situations in which training received by
an individual emergency responder has become an issue
during litigation after an accident or emergency.
Facilities should seek advice from General Counsel to
determine whether to include additional information in
the training records. Examples of additional records to
maintain may include lesson plans by course iteration,
participant evaluations, and any memoranda or
documentation regarding remedial training received by an
individual.
2.4 Training Needs Assessment
2.4.1 Training for Onsite Emergency Responders
Training needs are based on tasks to be performed by an
emergency responder, hazards that may be encountered by
response personnel, and established requirements and
standards for emergency responder training. A systematic
process should be used to identify and document performance-
based training requirements for emergency responder
positions. Training needs are identified initially by
reviewing regulatory requirements and existing training
programs, and then conducting a needs analysis.
In the case of training requirements originating from a
regulatory source [i.e., the Occupational Safety and Health
Administration (OSHA), National Fire Protection Association
(NFPA), DOE Order], a needs analysis may have already been
accomplished and included in the regulation. Further in-
depth analysis need not be conducted, once applicable
requirements have been determined. These requirements
should be included and addressed in the design and
development phases of the training program. Examples of
Federal requirements that specify training for emergency
responders are included in Section 2.5.
A needs analysis should ensure that training for individual
emergency response positions provides knowledge and skills
associated with assigned tasks to be performed. The needs
analysis should document the training that the emergency
responder receives related to their normal position in the
work force that is applicable to their ERO responsibilities.
Training topics should reflect specific function, position,
and responsibilities consistent with activities associated
with the Program Elements of the emergency management
program. Training should incorporate lessons learned in
emergency planning and response based on site experience, as
well as experience from throughout the DOE/NNSA complex,
other government agencies, and private industry.
Appropriate topics to be considered for inclusion in
emergency management training programs are listed in Section
2.5.
Analysis of emergency response training needs should be
ongoing. Additional analysis is appropriate when a
discrepancy or problem is identified in the performance of
an emergency response task and whenever program changes
occur. Responsible managers should receive and use
information from a variety of sources to ensure that
training continues to reflect changes and to address lessons
learned. Training requirements may be modified based on
changes in hazards, response facilities or equipment,
communication systems, site or facility mission or layout,
reorganization of the ERO, or revision of procedures or
requirements. Training needs may also change based on drill
or exercise evaluations, results of independent evaluations,
occurrence reports, and industry lessons learned.
Tabletop analysis is a recommended method for identifying
site-specific training needs. This method utilizes a
facilitator to guide a group of subject matter experts
through a process of job analysis and selection of tasks to
be addressed in training.
For personnel who have transferred from another DOE site or
for contractor or personnel with experience in a closely
related industry, a streamlined and standardized
qualification process can be established. The feasibility
of streamlining qualifications must be evaluated on a case-
by-case basis and documented by proof of experience. Some
additional training will usually be required to become
familiar with facility-specific hazards and procedures.
2.4.2 Training for Offsite Emergency Responders
Training needs should also be identified for offsite
emergency responders who may be involved in response to site
emergencies. The applicable agreements [i.e., Memorandum of
Agreement (MOA) or Memorandum of Understanding (MOU)] should
identify the type of training needed.
A systematic process should be used to determine which
offsite organizations have emergency responders that may
require site-specific training. For example:
Facility-specific orientation training on hazards and
emergency response procedures, as well as emergency
notification and communications should be offered
annually to state, tribal and local emergency response
organizations.
Private hospitals, medical and ambulance services
expected to support onsite response efforts or receive
contaminated injured personnel from the site should also
receive training on site hazards and protection from
those hazards as well as interface and communications
with site responders.
Offers of annual training for all parties in mutual aid
agreements should be considered. These same parties
should be offered the opportunity to participate in
training drills and exercises. (Ref. DHS HSEEP program
for specific information on requirements pertaining to
emergency exercises for local and state agencies and
integration with offsite responders)
Training, drills and exercises should aim to achieve team
building, consensus building, contingencies examination,
problem solving, resources measurement, and interface
examination.
2.5 Specific Training Requirements
Each site must determine the specific emergency response
training requirements that apply to the ERO positions
based on specific site hazards, conditions, resources, and
emergency plans. Applicable requirements will differ from
site to site. For example, for sites that involve mining
operations, the Mine Safety and Health Administration
(MSHA) regulations for mine rescue training may apply.
Differences would result from the situation where a site
must maintain an onsite fire brigade vs. a site that
depends solely upon offsite agency fire response. The
following sections describe specific training requirements
derived from applicable regulations, position-specific
functions, site characteristics and operations, and
details of the emergency management program.
2.5.1 Regulatory Requirements
Training personnel should monitor applicable regulations
promulgated by OSHA, MSHA, Nuclear Regulator Commission
(NRC), Environmental Protection Agency (EPA), NFPA, and DHS.
The impact of regulatory changes on training needs can be
evaluated using the following questions:
What conditions do the changes address?
Do those conditions exist at this facility/site?
Will changes influence the way our personnel perform
their tasks?
What specific effects will this change have on training?
Does the condition require an immediate response?
The following are examples of non-DOE Federal and national
requirements that specify training for emergency responders:
29 CFR 1910.156 specifies fire fighter training
40 CFR 265.37 specifies training for EROs Granted
Facility Access and Training for Hospitals Receiving
Facility Patients
40 CFR 112.7 specifies training for Spill Prevention,
Control and Countermeasures
29 CFR 1910.120 specifies training for Hazardous
Materials Emergency Response
29 CFR 1910.38 specifies training for Emergency Action
Plan/Evacuation of Employees
29 CFR 1910.157 specifies Fire Extinguisher Education
29 CFR 1910.146 specifies Confined Space Rescue Training
NFPA 472 specifies training for Professional Competence
of Responders
NFPA 473 specifies training for Professional Competence
of EMS Personnel
NFPA 1500 specifies training for fire department
Occupational Safety and Health Programs
NRP and NIMS
DHS HSEEP, Volumes I through IV.
Emergency Operations Training Academy (EOTA)
Other Federal or State training requirements may apply for
some DOE/NNSA sites.
2.5.2 Site-Specific Training
General training for employee response, including
training on protective actions in an emergency is
required as part of the Operational Emergency Base
Program. This may be included as part of an employer's
General Employee Training (GET) Program. Emergency-
related information in this training should include
emergency awareness, overview of the organizations
emergency response plan, warning systems and alarms,
protective action (e.g., evacuation and sheltering),
accountability for site workers in the event of an
emergency, and first aid. Employees assigned to specific
responsibilities for onsite emergency response should
receive additional training to address those
responsibilities. At a minimum, this includes emergency
managers, building wardens who support personnel
accountability and protective action procedures
(e.g., personnel assigned to close doors and windows and
shutdown of ventilation systems), personnel assigned to
perform first aid/cardio pulmonary resuscitation (CPR) or
use fire extinguishers, emergency spokespersons, and
personnel responsible for interface with offsite response
organizations that may support onsite emergency response.
2.5.3 First Responder Training
Initial and annual refresher training should be provided to
workers who are likely to witness emergency conditions
involving hazardous materials and who are expected to notify
the proper authorities. These workers are expected to
attain the applicable training level according to the
requirements specified in 29 CFR 1910.120.
2.5.4 Offsite Responder Training
As discussed in Section 2.4, emergency-related information,
instruction concerning notification procedures, and training
on site-specific conditions and hazards should be made
available to offsite personnel who might be requested to
respond to an emergency at the DOE facility/site, as well as
hospitals that have agreed to receive patients from facility
emergencies. Offsite responders should also have the
opportunity to participate in drills conducted to validate
procedures and test integration of resources with the
facility response organizations.
2.5.5 Training on Change
When changes occur involving facilities or sites
(e.g., change in mission, facility decommissioning,
organization reengineering) or when hazards change,
employees and responders should receive training on how
these changes apply to them and their responsibilities for
emergency response. Annual re-training should address
changes; however, training on changes may need to occur
prior to scheduled re-training to ensure safe and effective
response.
2.5.6 Operational Emergency Hazardous Material Program
The Operational Emergency Hazardous Material Program
described in DOE O 151.1C requires a coordinated training
program consisting of formal training and drills. The
program develops and/or maintains specific emergency
response capabilities for all personnel and organizations
expected to respond to onsite emergencies. This training
program should consist of a combination of self-study,
homework, formal classroom training, field training focusing
on skills, and drills. All personnel (including both the
primary and alternates) who constitute the ERO should
receive both initial and annual refresher training.
Emergency-related training should also be made available to
offsite response organizations.
Training should emphasize the need for prompt, accurate, and
practical judgments involving event categorization and
classification, protective actions and the urgency of
notifications of Operational Emergencies. Training should
address decision-making when information is incomplete or
uncertain and when events and conditions are not covered
explicitly by Emergency Action Level (EAL) procedures.
The training program should be commensurate with the
hazards identified in the Emergency Planning Hazards
Assessments (EPHAs). Individual training programs should be
commensurate with assigned emergency-response
responsibilities. Training topics should reflect the
trainees functional position and responsibilities.
Appropriate topics to consider in the training program
include:
Hazards assessments
Emergency Management Program Administration
Management and decision-making
ERO
Incident command
Activation and coordination of response resources
Hazardous materials emergency response
Fire-fighting
Offsite interfaces
Coordination and liaison with offsite response and
support organizations
Emergency facilities and equipment operation
Emergency categorization and classification
Decision-making
Notification and communications
Consequence assessment
Dose projection
Field monitoring
Decision-making
Protective Actions and Reentry
Protective actions/protective action recommendations
decision-making
Reentry planning
Rescue
Decontamination
Emergency medical support
Emergency public information
Emergency spokesperson skills
Emergency Termination and recovery
Recovery planning
2.6 Training Development
In this section, development of individual training courses
is discussed. The initial step in training course
development is establishment of goals and objectives. The
method for delivery of the training will depend on the
target audience and the goals and objectives of the
training. Course materials will be designed to facilitate
the learning process and will be consistent with the
delivery method selected. A key element of the course
development is identification of the method for testing
whether students have successfully achieved the specified
level of competency in the material presented. Once all
aspects of the course have been developed, the training
should be evaluated to determine whether it meets
expectations. Finally, the special subject of remedial
training development is briefly discussed in this section.
2.6.1 Training Goals and Objectives
The training or instructional goals for a specific course
state the anticipated outcome of instruction. The
instructional goal provides a broad statement of what will
be accomplished at the conclusion of the lesson or course.
Training objectives are measurable statements of intent that
specify expected outcomes of each stage of training. They
should state clearly what participants would know or be able
to demonstrate after training. Course results will be
measured against the original goals and objectives set at
the beginning of course development.
2.6.2 Training Delivery
Methods of training delivery are an important consideration
in the design of each course. Appropriate methods for
delivering the specific training depend on the target
audiences composition, location, need, and job complexity.
The type of delivery for emergency management training
should also be based on learning objectives, learning tasks,
and group size. Class size may need to be controlled to
maximize instructor/ student interaction. Classroom-style
delivery of training may effectively use live classroom,
video presentations, computer-based instruction, or self-
paced instruction via computer disk and Internet. Most
training should combine instructor presentation with student
participation and hands-on demonstration and experience.
Training demonstrations, role-playing, and practical skills
training should be realistic but with due consideration for
student and facility safety. ISMS practices should be fully
implemented when bringing realism into practical training or
drills.
To maximize student involvement, classroom training may be
augmented with tabletop simulations, hands-on drills, role-
playing exercises, group tasks, facilitated group
discussions, and assigned reading/reporting. Practical
exercises that put knowledge into practice are very
effective to help ensure student ability to apply knowledge
in the context of realistic hypothetical situations.
2.6.3 Developing Course Materials
Course materials are the materials used by instructor and
participants to facilitate training. When preparing to
develop course materials, first review any existing training
materials, regulations, manuals, and industry guidelines for
usability. Existing materials should be reviewed against
instructional objectives to determine whether they partially
or fully meet training requirements. Course materials can
include lesson plans/course documentation, training support
materials, and participant materials.
Lesson Plans/Course Documentation. An instructor uses a
lesson plan as the primary training tool for guiding the
learning process. Standard lesson plans for classroom
instruction promote consistent, effective instructor
presentation and may include the following:
Administrative information (name of course, time
allotted, target population, instructional method, and
approval)
Training goal and objectives
Details about training methodology
Lesson content based on learning objectives
Lesson content sufficiently detailed to ensure consistent
and repeatable training
Safety information, as relevant
Training support requirements
Training Support Requirements. Training support materials
should be selected and/or developed to support and reinforce
the learning objectives. For each course documented,
identify, develop, and maintain a list of the resources
including trainers, technology, equipment, and facilities
(classrooms, laboratories, and response facilities) required
to support training activities. Training support materials
or aids to consider include computers, software, video,
models, demonstration equipment, scenarios and classroom
exercises.
Participant Materials. The purpose of participant materials
is to enhance learning and to provide reference materials
for participants. Materials might include any of the
following:
Student workbook
Job aids such as procedures or equipment operating
instructions
Glossary of terms
Checklists used to document action steps
Copies of viewgraphs used by trainer to illustrate key
points
References discussed during training session
Lesson plan outline
Participant materials should be marked For Training Use
Only to ensure that trainees do not confuse them with
procedures.
2.6.4 Test Methodology
Training and drills should include some form of measurement
or demonstration that indicates completion of training
objectives and achievement of qualification standards.
Tests document the knowledge and skills a participant has
gained from training. Written examinations and performance
evaluations measure achievement of each instructional
objective. Each evaluation item should reference a specific
training objective.
Emergency drills in particular provide an excellent
opportunity to incorporate performance tests of individual
emergency responder and response group proficiencies.
Instructors who evaluate practical performance tests during
drills must have the technical experience and expertise to
provide a valid assessment of performance.
When constructing a test, the following characteristics and
constraints should be considered:
Test length should reasonably reflect the length and
complexity of the lesson plan that is delivered.
Multiple test items for each learning objective may more
accurately verify that learning has taken place.
Vary test materials or use randomly selected test banks
to ensure long-term validity of tests and ensure that
test information is not shared and compromised between
students.
Performance tests should include checklists addressing
each learning objective.
Development of pre-tests and post-tests, and the
comparison of results, can help validate that learning
has taken place and help verify the effectiveness of the
specific training course.
Intermediate evaluation (i.e., measuring progress during
the course) can help instructors to verify learning
before the training course is complete and may prevent
the need for remedial training at the end.
Evaluation of course objectives measures what the
participants know or can perform at the end of training.
2.6.5 Course Evaluation
Training programs should be evaluated by the training
organization for the adequacy of the following: program and
lesson content; examinations; presentation; documentation;
and, post-training performance. Participants should be
encouraged to provide evaluation of course materials and
delivery as part of the overall program evaluation process.
This evaluation process determines strengths and weaknesses,
improves content and delivery, and ensures that revisions
are made as appropriate. Development of new courses should
include evaluation and validation of the effectiveness of
course materials using pilot presentations, peer reviews,
and/or review by subject matter experts.
2.6.6 Remedial Training
Remedial training is additional training provided to a
participant who did not correctly answer the required number
of test questions or who was unable to successfully complete
a formal training session in the previous iteration.
Because remedial training requirements are lesson-specific,
they should be prescribed in each lesson plan so they are
ready for use in case they are needed. Remedial training
focuses on the specific knowledge or skills challenging the
participant. Remedial training might consist of additional
instruction or training directly related to the training
objectives for the portion of the course in which the
participant had difficulty. The remedial training is
intended to raise the individuals competency to a level
that allows attainment of the knowledge and skills required
to successfully complete the lesson or demonstrate the skill
proficiency required to perform the job.
2.7 Training Delivery
The previous section briefly discussed methods for delivery
of training and how the delivery method should be part of
the training plan. This section provides specific guidance
to training organizations, including instructors responsible
for actually delivering training. In preparation for
delivery of training, training personnel should confirm that
the following are ready for the training session:
Equipment instructional equipment such as projectors,
VCRs, computer hardware, computer software and television
monitors
Facilities classrooms and setup of the classroom,
exercise sites, ranges, computer labs, etc.
Course administration materials, supplies,
documentation, such as attendance sheets, certificates,
and participant notebooks
Instructor Qualifications/Skills knowledge of lesson
plan content and knowledge of target populations needs;
preparation of lesson plan and participant learning
activities
The following is practical guidance for instructors to help
ensure successful delivery of training:
Adhere to the lesson plan content and presentation method
defined in the training design document.
Adhere to all safety measures listed in the lesson plan.
Create an instructional atmosphere that enhances the
learning process.
Use effective communication skills to keep students
engaged in discussions and activities.
Dress appropriately for the environment and training
activities. For example, if the normal dress is business
casual or if training will be conducted outside, dress
accordingly.
Provide and monitor feedback from participants to ensure
active learning.
Ensure that participants accomplish all the objectives
during training.
Ensure involvement of all students in training
activities, group tasks, discussions, and hands-on
learning experiences.
Evaluate participant performance during and at completion
of the training session.
Ensure that all required materials are made available to
the participant for self- paced training.
Use standardized materials for on-the-job and technology-
based training programs.
2.8 Drills
Drills are training methods that allow an individual to put
knowledge into practice in the context of a scenario-based
simulation. The drill provides practical training to
enhance preparedness for emergency response personnel and
organizations that are expected to respond to onsite
emergencies. Qualification requirements for each emergency
response position should include annual participation in at
least one training drill (or, alternatively, an exercise)
during which practical knowledge and skills are
demonstrated.
Drills are supervised hands-on instruction and application
sessions for individuals or teams. These sessions provide
an opportunity to demonstrate and maintain individual and
organizational proficiency. During drills, the desired
skills or actions may or may not be first demonstrated by
the instructor(s). These training activities are documented
by a plan, which includes a performance checklist used by
the instructor or evaluator. The checklist has two
purposes: to provide feedback during the training and to
summarize overall performance. Because the focus is
training, it is often appropriate for the instructor to stop
and correct participant actions during the activity rather
than waiting until the end.
Drills should be of sufficient scope, duration, and
frequency to ensure adequate training for all emergency
response functions applicable to the facility. The size and
complexity of any drill will depend on the objectives. Many
drills will be functional, focusing on training responders
involved in a specific response function. Drills can range
from hands-on instruction involving one procedure to a multi-
organizational, scenario-driven event. Drills should be as
realistic as possible, using realistic scenarios based on
hazards surveys and EPHAs as well as actual facility
conditions. Conduct of drills requires a skilled and
experienced instructor(s) who can present the scenario,
control activities of responders, and provide feedback that
enhances learning.
Within the DOE/NNSA emergency management system, the
distinction between a drill and an exercise is reflected in
their primary purpose, namely, a drill is oriented toward
training and is not a graded evaluation of the response
activity. Because the focus of a drill is training, some
aspects of drill conduct can be made more flexible than in
an exercise. Some emergency response roles may be combined
and the instructor/controller may be free to stop and
correct the responder actions during the drill. In a small
drill, one instructor may plan, conduct and evaluate the
performance.
Consideration should always be given to the need for safety
and security plans when drills are conducted. Any drill
that has the potential to affect or might be observed by an
offsite population (e.g., activities of a field monitoring
team, smoke from a fire drill, etc.) should be planned to
avoid public concern or inconvenience.
The following represent typical functions/activities for the
focus of drills:
Emergency medical team response
Hazardous Material (HAZMAT) response
JIC activation
Dose assessment drill
Field monitoring drill
Emergency notifications/communications with offsite
agencies
Protective Force interface with Fire Department
Successful implementation of drills involves systematic
planning, conduct, and assessment.
Planning for drills should involve the following
components:
Performance objectives. Identify the performance,
including conditions and standards of performance. The
objectives should reference a specific policy,
procedure, or training requirement.
Performance checklist. Develop a checklist based on
the performance objectives, conditions, and standards
of performance. Also, identify how the drill will be
conducted in the context of the assessment checklist.
Scenario description. Describe the elements or system
being trained through use of the specific scenario.
Scenarios may be restricted to specific, limited
aspects of the emergency management system. Scenarios
should be based on site hazards identified in the
Hazards Survey or EPHA. The scenario description
should include a detailed description of events and
conditions that emergency responders need to deal with
and a timeline of events and actions that are expected
to take place.
Controls. Describe the controls imposed to ensure the
integrity of the drill (e.g., safety plans, security
notifications, use of trusted agents, equipment
controls, time limitations and coordination with other
operational elements).
Resource requirements. Description of resources needed
to conduct the drill (e.g., facilities, personnel, and
equipment).
Compensatory measures. Describe the measures to be
taken to compensate for any degradation of security or
response capabilities during the training.
Documentation and approval process. Identify the
approval process for drill plans, the dates on which
drills were conducted, and the results/corrective
actions identified.
References. Included are lesson plans, DOE
Headquarters and site orders/manuals, and site policy
documents containing requirements for objectives being
tested.
Conduct of a drill requires that the instructor adhere to
the Emergency Plan. This ensures that the training
provides an accurate and valid representation of the
emergency management program. Feedback from the
instructor during the training is essential.
Guidelines for conduct of a drill include:
Explain the purpose and objectives of the training
drill/practical application.
Maintain a calm and professional attitude.
Question to verify the knowledge gained by
participants.
When there is a deficiency, stop the drill/application
and provide immediate corrective action.
Provide fact-based feedback.
Respect participants experience and expertise.
Assessment requires the instructor to document results of
training drills in a formal report. Consider including
the following elements:
Description of the drill. Describe the conditions
under which the test was performed and specifics of who
participated.
Summary of performance. Describe what was observed
during the training drill. Include not only positives,
but also opportunities for performance improvement.
Results. Analyze data to describe observed performance
against the performance standards from the objectives.
Remediation/corrective actions. List and discuss
recommendations for emergency response measures that
did not meet requirements. Identify the individuals or
organizations for corrective actions. If remedial
training is required, identify the schedule for
conducting the training. Both immediate and long-term
solutions should be addressed.
Tabletop Activities. Tabletop activities represent a cost-
effective type of training drill experience during which
emergency responders have an opportunity to interact with
other response positions and learn their individual
responsibilities, decision-making functions, and
communication requirements in the context of these
interactions. Tabletops may range from lecture and guided
discussion to a detailed verbal simulation of response to a
particular scenario. The instructor and/or students
verbally walk-down response to a facility-specific scenario
to illustrate the overall direction the response is taking
and to clarify participant perceptions of their roles.
Learning objectives for a tabletop will determine the focus
of the activity (e.g., overall coordination versus detailed
problem-solving). Because of the inherent flexibility of
this approach, trainers are free to structure the training
experience creatively, controlling scenario time and trainee
activity. However, a tabletop requires significant
preparation to ensure that objectives are satisfied. The
instructor must be skilled to facilitate and record the
training session. A co-instructor or recorder can also be
used to note questions and problems to be addressed later
through procedure revisions, additional training, or
agreements between response groups. If a tabletop involves
multiple response groups or a detailed or highly technical
scenario, then representatives of the involved groups/
agencies or technical specialists should be involved in the
planning of the activity to ensure that scenario details,
procedures, and expected response actions are correct.
Typical topics for the focus of tabletop training activities
may include:
Coordination and interfaces between the site and offsite
agencies
Emergency Operations Center (EOC) management decision-
making
Interfaces within the JIC
Interfaces between site response groups (e.g., Health
Physics, Security, Fire Department)
Hostage negotiations
Emergency categorization/classification
3. EXERCISES
3.1 Introduction
The purpose of this chapter is to assist DOE and NNSA field
elements in complying with the DOE O 151.1C requirement to
establish a formal exercise program that validates all
elements of a facility/site or activity emergency management
program over a 5-year period. The exercise program should
validate both facility- and site-level emergency management
program elements by initiating a response to simulated,
realistic emergency events or conditions in a manner that,
as nearly as possible, replicates an integrated emergency
response to an actual event.
Planning and preparation for exercise conduct should use an
effective, structured approach that includes documentation
of specific objectives, scope, timelines, injects,
controller instructions, and evaluation criteria. Each
exercise should be based on a realistic scenario derived
from the facility/site or activity Emergency Planning
Hazards Assessment (EPHA) and must be conducted, controlled,
evaluated, and critiqued effectively and reliably. Lessons
learned from the exercise evaluation should be developed to
ensure that corrective actions are implemented and
improvements are made to the program.
Requirements, development, and implementation of an exercise
program, including program planning and management, are
addressed in this chapter. Guidance is provided on the
design and development of an exercise including: scheduling,
work planning, determining objectives, and production of an
exercise plan. Exercise conduct, control, and evaluation
processes are also described.
The Department of Homeland Security (DHS) Homeland Security
Exercise and Evaluation Program (HSEEP) is a Federal-level
exercise program developed by the DHS for State, county and
local emergency management programs. The DHS approach
addresses not only Homeland Security sponsored exercises,
but also those exercises where Federal level agencies may
interact with State, county and local emergency management
programs. Therefore, to ensure consistency with the DHS
approach to exercise development, conduct, and evaluation,
common exercise concepts and processes of the HSEEP are
incorporated in the guidance presented in this chapter using
DHS terminology where applicable.
Two generic types of exercises will be defined in this
chapter based on HSEEP exercise methodology: discussion-
based exercises and operations-based exercises. Although
both types can play a significant role in facility/site and
activity preparedness activities, the guidance in this
chapter will focus primarily on operations-based exercises,
which are the subject of emergency management program
requirements specified in DOE O 151.1C.
An easy-to-use computer-based tool for developing DOE
emergency exercises, the Exercise Builder, has been
developed by participants in the Emergency Management Issues
Special Interest Group (EMI SIG) Exercises and Drills
Subcommittee, under the sponsorship of the DOE/HQ Office of
Emergency Management. It makes available generic exercise
components such as scenarios, objectives and criteria and
provides a PC-based application that can be used to develop
exercise packages. End products include exercise scopes,
objectives, scenario materials, and evaluator modules. Once
exercise packages have been developed, they can be modified
for future use.
Training is also available for using Exercise Builder. Web-
based tutorials are available to prepare emergency
management staff to plan and develop exercise/drill
objectives and scenario materials. Information for
obtaining this training and the Exercise Builder tool can be
found on the following website:
http://www.orise.orau.gov/emi.
This chapter is designed primarily for facilities/sites and
activities that are required to implement an Operational
Emergency Hazardous Material Program and is directed at
operations and emergency management staff at Field Elements
and operating contractor organizations that are responsible
for DOE and NNSA facilities/sites and activities.
3.2 General Approach
Emergency management exercises are formal, evaluated tests
and demonstrations of the integrated capabilities of
facility/site and activity emergency response resources
(i.e., personnel, facilities, and equipment), conducted for
the purpose of testing/validating multiple elements of an
emergency management program. Exercises include realistic
simulations of emergencies and tests of response
capabilities, such as command, control, and communication
functions and event-scene activities. Exercises can vary
significantly in scope, size, and complexity to achieve
their respective objectives.
A valid test of response capabilities requires a formal and
structured approach for planning, developing, and conducting
each exercise. Exercise-specific objectives are used to
specify the emergency response functions to be tested.
Exercise objectives must be well defined and achievable.
The set of objectives should effectively define the pre-
determined extent of organization/personnel participation
and scope (i.e., breadth and depth) of exercise activities
to be accomplished or simulated. The scenario must be based
on the specific hazards associated with the facility/site or
activity that is the focus of the emergency. The scenario
must provide the opportunity for participating
organizations/personnel to demonstrate each objective in
order to evaluate the function or activity. The flow of the
scenario timeline and events must be effectively controlled
and the response of the participants must be realistic and
professional. An effective evaluation and critique process,
based on specific evaluation criteria, ensures clear and
useful findings are accurately developed and ultimately will
lead to lessons learned and corrective actions resulting in
an improved emergency management program.
General guidance in this chapter is primarily applicable to
operations-based exercises throughout the DOE/NNSA complex
at all levels of the emergency response organization:
facility, site, Cognizant Field Elements, and Headquarters
levels for DOE/NNSA Federal and contractor organizations,
including the response activities of the Radiological
Emergency Response Assets and Transportation Emergency
Preparedness Program. This guidance follows the DOE/NNSA
commensurate with hazards approach to emergency management.
Guidance is provided for facilities/sites and activities
with varying types and levels of hazards and with differing
organizational structures and complexity.
Functional aspects of planning, development, and conduct of
exercises are addressed, but not roles and responsibilities
of specific organizations or individuals. Requirements,
development, and implementation of an emergency management
exercise program, including program planning and management,
are presented. Guidance is provided on design and
development of an exercise including scheduling, work
planning, determining objectives, and production of an
exercise package. Conduct, control, and the evaluation
process for exercises are also described.
3.3 Exercise Program
A formal exercise program ensures testing and validation of
the elements of a facility/site or activity emergency
management program over a 5-year period. The program should
provide a continuing series of periodically conducted
exercises to evaluate emergency response capabilities and to
provide assurances that members of the ERO are prepared to
respond promptly, efficiently, and effectively to an actual
emergency. The program includes a plan for validating all
elements of each program by incorporating specific
objectives in exercises over the planning period. The
exercise program includes provisions for incorporating
specific exercise objectives in each exercise designed to:
Periodically test specific aspects of emergency response
Validate plans/procedures
Validate implemented corrective actions
Test program improvements
Evaluate notifications and communications
A formal exercise program should be established and
maintained for each DOE/NNSA facility/site or activity to
address the following:
Long-range planning and scheduling for future exercises,
and short-range planning for the current years exercises
Overall planning, preparation, conduct, control, and
evaluation of exercises
Development of comprehensive exercise objectives based
upon Base Program and Hazardous Material Program
requirements from DOE O 151.1C and program-specific EPHAs
Development of exercises commensurate with, and based
upon, the facility/site hazards and types of scenarios
identified in the EPHAs
Application of sufficient resources to the exercise
program
Roles and responsibilities of all aspects of the exercise
development, conduct, and evaluation process are assigned
Long- and short-range program planning should include:
A long-range plan to be prepared and maintained as part
of the Emergency Readiness Assurance Plan (ERAP). The
long-range plan should be developed in concert with the
various organizations affected by its provisions. The
plan should include the general schedule, scope, and
objectives of the exercise over a 5-year period and
provide for demonstrating all aspects of the emergency
program in a systematic and comprehensive manner.
A short-range plan to address fiscal year planning. It
should include the scope, exercise objectives,
participants, and schedule for the major tasks and
activities associated with the current years
exercise(s). Planning and scheduling for a specific
exercise includes confirming or modifying the planned
scope, developing detailed objectives, committing the
participants and resources, and identifying and
scheduling the various activities.
Each organization should identify a single individual who
is responsible for the exercise program. Depending on
the organizations size and the scope/complexity of the
exercise program, these responsibilities may be the
primary or collateral duties of the individual.
Responsibilities include the authority or capability to
commit and coordinate the resources necessary for an
effective exercise program. Exercise program functions
to be performed by the designated individual should
include the following:
Resolving conflicts identified during the exercise
scheduling process.
Concurring on the scope and objectives of each exercise.
Coordinating organizational resources for development,
conduct, response, and critique of an exercise.
Monitoring potential programmatic impacts from the
exercise development process, as well as resolving any
specific exercise development difficulties or conflicts.
Coordinating with the training and drill program to
ensure that all participants have completed their
required fundamental emergency management training (not
specific to an exercise) prior to a scheduled exercise.
3.4 Types of Exercises
Various types of exercises can be used to test and validate
DOE/NNSA facility/site and activity emergency response
capabilities. The type used will be based on DOE/NNSA
requirements and the identified goals of the exercise, and
can include discussion-based and operations-based exercises.
3.4.1 Discussion-Based Exercises
Discussion-based exercises are used as a starting point in
the building block approach to the cycle, mix, and range of
exercises. Discussion-based exercises include seminars,
workshops, tabletop exercises (TTXs), and games. These
types of exercises highlight existing plans, policies,
mutual aid agreements, and procedures. Discussion-based
exercises typically focus on strategic, policy-oriented
issues and are ideal tools for familiarizing agencies and
personnel with current or expected jurisdictional
capabilities. Facilitators and/or presenters usually lead
the discussion, keeping participants on track while meeting
the objectives of the exercise.
NOTE: Although referred to as exercises in DHS
terminology, discussion-based exercises can accomplish
evaluation functions similar to several frequently used
techniques for conducting DOE emergency management
program evaluations. These include emergency plan and
procedure reviews, interviews, Limited Scope Performance
Tests (LSPT) and tabletop drills (both of which are
scenario-based discussions between evaluators and
interviewees). Chapter 4, Section 4.5 of DOE G 151.1-3
describes some of these readiness assurance evaluation
techniques.
A.Seminars
Seminars are used to orient participants to, or provide
an overview of, authorities, strategies, plans, policies,
procedures, protocols, response resources, and new
concepts/ideas. Seminars can be a starting point when
developing or making major changes to plans and
procedures. Seminars offer the following attributes:
Low-stress environment employing a number of
instruction techniques such as lectures, multimedia
presentations, panel discussions, case study
discussions, expert testimony, and decision support
tools
Informal discussions led by a seminar leader
Lack of time constraints caused by real-time portrayal
of events
Effective with both small and large groups
B.Workshops
Workshops, while similar to seminars, differ in two
important aspects: player interaction is increased and
the focus is on achieving or building a product (such as
a plan or a policy). Workshops provide an ideal forum
for:
Collecting or sharing information
Obtaining new or different perspectives
Testing new ideas, processes, or procedures
Training groups in coordinated activities
Problem solving of complex issues
Obtaining consensus
Team building
Workshops, used in conjunction with operations-based
exercise development, are useful in achieving specific
aspects of exercise design (e.g., determining program or
exercise objectives, developing exercise scenario and key
events listings, and determining evaluation elements and
standards of performance).
A workshop may be used to produce new emergency operating
procedures, mutual aid agreements, Multiyear Exercise
Plans, and Improvement Plans (IPs). Workshops share the
following common attributes:
Low-stress environment
No-fault forum
Employ different instructional techniques to convey
information
Facilitated, working breakout sessions
Plenum discussions led by a workshop leader
Goals oriented toward an identifiable product
Lack of time constraint from real-time portrayal of
events
Effective with both small and large groups
C.Tabletop Exercise (TTX)
TTXs generally involve senior staff, elected or appointed
officials, or other key personnel in an informal setting
in which situations are discussed that arise during
simulated scenarios. The TTX can be used to assess
response plans, policies, and procedures, or types of
systems needed to mitigate and respond to the specific
emergency event. The TTX is typically aimed at
facilitating understanding of concepts, identifying
strengths and weaknesses, and/or achieving a change in
attitude. The TTX format focuses on slow-paced problem
solving rather that the rapid, spontaneous decision-
making that occurs during actual emergencies or
operations-based exercises.
TTX methods can be divided into two categories: basic and
advanced. In a basic TTX, the scene set by the scenario
remains constant. The emergency event is described to
the participants up to a certain point in time. The
leader is then presented with a set of problems to be
discussed by participants, resolved by the group, and
summarized by the leader. In an advanced TTX, play is
initiated through injects to the participants that alter
the original scenario. The exercise controller usually
introduces problems one at a time in the form of written
injects; participants then discuss the problems, using
appropriate plans and procedures.
The TTX can have the following attributes:
Practicing group problem solving
Familiarizing senior officials with a situation
Conducting a specific case study
Examining personnel contingencies
Testing group message interpretation
Participating in information sharing
Assessing coordination among participants
Achieving limited or specific objectives
D.Games
A game is a simulation of operations that often involves
two or more teams, usually in a competitive environment,
using rules, data, and procedures designed to depict an
actual or assumed real-life situation. It does not
involve the use of actual resources, and the sequence of
events affects, and is in turn affected by, the decisions
made by the players. Games stress the importance of
planner and player understanding and comprehension of
interrelated processes.
Players are presented with scenarios and asked to perform
a task associated with a scenario episode. As each
episode moves to the next level of detail or complexity,
it takes into account players earlier decisions; thus,
the decisions made by players determine the flow of the
game. The goal is to explore decision-making processes
and the consequences of those decisions. In a game, the
same situation can be examined from various perspectives
by changing the variables and parameters that guide
player actions. Large-scale games can be multi-
jurisdictional and include active participation from
Federal, Tribal, State, and local governments.
Games are excellent vehicles for the following:
Gaining policy or process consensus
Conducting what-if analyses of existing plans
Developing new plans
3.4.2 Operations-Based Exercises
Operations-based exercises represent the next iteration of
the exercise cycle and can be used to validate the plans,
policies, agreements, and procedures, possibly solidified in
previous discussion-based exercises. Operations-based
exercises include drills, functional exercises (FEs), and
full-scale exercises (FSEs). They can clarify roles and
responsibilities, identify gaps in resources needed to
implement plans and procedures, and improve individual and
team performance. Operations-based exercises are
characterized by actual response, mobilization of apparatus
and resources, and commitment of personnel, usually over an
extended period of time. Operations-based exercises play
the primary role in the readiness assurance program for a
facility/site and will be the focus of much of the guidance
in this chapter.
A.Drills
According to the DHS exercise program HSEEP, a drill is a
coordinated, supervised activity usually employed to test
a single specific operation or function. Drills are also
commonly used to provide training on new equipment,
develop or test new policies or procedures, or practice
and maintain current skills. Typical attributes include:
A narrow focus, measured against established standards
Realistic environment
Performance in isolation
Instant feedback
Opportunity to stop, correct/educate, and restart
In Chapter 2 of DOE G 151.1-3, the drill program is
focused on drills as part of the overall training
program, whereas in this chapter the focus is on the role
of drills as operations-based exercises, employed to test
operations or functions.
B.Functional Exercise (FE)
The FE is designed to test and evaluate individual
capabilities, multiple functions or activities within a
function, or interdependent groups of functions. The FE
is generally focused on exercising the plans, policies,
procedures, and staffs of the direction and control nodes
of Incident Command (IC) and Unified Command (UC).
Generally, events are projected through an exercise
scenario with event updates that drive activity at the
management level. Movement of personnel and equipment is
simulated.
The objective of the FE is to execute specific plans and
procedures and apply established policies, plans, and
procedures under crisis conditions, within or by
particular functional teams. An FE simulates the reality
of operations in a functional area by presenting complex
and realistic problems that require rapid and effective
responses by trained personnel in a highly stressful
environment. Attributes of an FE include:
Evaluating functions
Evaluating Emergency Operations Centers (EOCs),
Headquarters, and staff
Reinforcing established policies and procedures
Measuring resource adequacy
Examining facility or site internal relationships
C.Full-Scale Exercise (FSE)
The FSE is the most complex of the operations-based
exercises. FSEs are exercises that test many aspects of
an integrated emergency response. An FSE focuses on
implementing, analyzing, and evaluating plans, policies,
and procedures. Events are projected through a scripted
exercise scenario with built-in flexibility to allow
updates to drive activity. A FSE is conducted in a real-
time, stressful environment that closely mirrors a real
event. First responders and resources are mobilized and
deployed to the scene where they conduct their actions,
as nearly as possible, as if a real incident had
occurred.
The FSE simulates the reality of operations in multiple
functional areas by presenting complex and realistic
problems requiring critical thinking, rapid problem
solving, and effective responses by trained personnel in
a highly stressful environment. Other entities that are
not involved in the exercise, but would be involved in an
actual event, should be instructed not to respond.
Typical FSE attributes include:
Assessing organizational and individual performance
Demonstrating interagency cooperation
Allocating resources and personnel
Assessing equipment capabilities
Activating personnel and equipment
Assessing inter-agency cooperation
Exercising public information systems
Testing communications systems and procedures
Analyzing memorandums of understanding (MOUs), standard
operating procedures (SOPs), plans, policies, and
procedures
The level of support needed to conduct a FSE is greater
than needed during other types of exercises. The
exercise site is usually extensive with complex site
logistics. Food and water must sometimes be supplied to
participants and volunteers. Safety issues, including
those surrounding the use of props and special effects,
must be very closely monitored.
3.5 DOE/NNSA Exercise Requirements
DOE O 151.1C contains specific requirements associated with
the conduct of operations-based exercises for DOE/NNSA
facilities/sites or activities. The following specific
exercise requirements apply:
A.Facility Requirements. Each DOE/NNSA facility [or group
of facilities with common facility-level Emergency
Response Organization (ERO) positions] must test and
demonstrate the proficiency of personnel in facility-
level ERO positions in accomplishing facility-specific
emergency response duties and responsibilities in a
facility operations-based exercise, including facility-
level evaluation and critique, at least annually.
Evaluations of annual facility exercises by Departmental
entities (e.g., Cognizant Field Element, Program
Secretarial Officer, or Headquarters Office of Health,
Safety, and Security) must be performed periodically so
that each facility has an external Departmental
evaluation at least every 3 years. [These facility-level
operations-based exercises can exhibit a number of the
defining characteristics of functional exercises (FEs) in
HSEEP terminology].
B.Site Requirements. A site operations-based exercise is
designed to test and demonstrate integrated emergency
response capabilities of personnel in facility- and site-
level ERO positions. Site-level ERO elements and
resources must participate in a minimum of one site
operations-based exercise annually. For multi-facility
sites, the basis for the exercise should be rotated among
facilities or groups of facilities on a site. [In
general, site-level operations-based exercises may be
considered full-scale exercises (FSEs) in HSEEP
terminology].
A site operations-based exercise that involves
participation of offsite response organizations is
referred to as a full participation operations-based
exercise. According to DOE O 151.1C, offsite response
organizations must be invited to participate in a site-
level exercise at least once every 3 years. [In general,
a full participation operations-based exercise may be
considered a full-participation FSE.]
C.Specific Activity Requirements. Operations-based
exercises of each of the Departments radiological
emergency response assets must be conducted at least once
every 3 years.
D.No-Notice Exercises (NNXs). Contractor facilities/sites
and activities participate in a program of No-Notice
Exercises, conducted in concert with and at the
discretion of the Associate Administrator, Office of
Emergency Operations, to determine if the facility/site
or activity ERO accomplishes selected objectives, based
on applicable plans, procedures, and/or other established
requirements. Although generally operations-based
exercises, NNXs can use the discussion-based exercise
format if the specific objectives can be accomplished.
The NNX is designed to require minimum resource
expenditure and cause only limited disruption of
facility/site or activity operations. In addition to
participation of initial responders, the site/facility
organization assigns a trusted agent to assist in the
identification of a credible emergency scenario and to
provide facility- /site-specific information. The
Headquarters (HQ) DOE/NNSA Office of Emergency Management
schedules (with facility/site or activity concurrence),
conducts, and documents the NNX and its evaluation,
including the development and coordination of the
exercise design package, providing an exercise director
and controllers/evaluators, conducting participant and
formal controller/evaluator critiques, and producing an
After Action Report (AAR).
The primary purpose of the NNX is to provide an objective
test of the ability of key elements of emergency response
capabilities to respond without prior notice to a
simulated Operational Emergency. NNXs are currently
focused on initial activation, mobilization, and response
activities in six program elements:
ERO Activation and mobilization of key elements of
the ERO
Emergency Categorization and Classification
Categorization/classification of the simulated
emergency event
Notification and Communication - Initial and follow-up
notifications and communications
Consequence Assessment Initial assessments of the
emergency [e.g., Timely Initial Assessments (TIA)]
Protective Actions Determination of initial
protective actions and offsite protective action
recommendations
Emergency Public Information Initial activities
(e.g., initial press release)
The NNX provides a low-impact and low-cost opportunity to
test the initial aspects of an emergency response and to
enhance key initial function. The NNX is deliberately
designed to minimize resources necessary for exercise
planning, conduct, control, and evaluation. The NNX is
characterized by a limited number of exercise objectives and
either no effort or a minimal development effort expended on
characterizing the consequences of the event for use by
controllers during the exercise. An essential component of
the NNX exercise is a limited disruption of normal
operations that usually ensures the exercise will be of
short duration (e.g., 2 to 3 hours).
The frequency for including NNXs in an annual readiness
assurance program for a facility/site or activity should be
based on the need for validating lessons learned;
reevaluating areas based on observations during prior
training, drills, or exercises; or validating recently
implemented or revised plans and procedures without
incurring the cost and impact of the facility- or site-level
exercise.
3.6 Exercise Planning
This section addresses generic aspects of exercise planning.
Planning for an exercise is fully documented by an exercise
plan that includes: specific exercise objectives, scope,
scenario, participants, simulations, timelines, injects
(i.e., messages), technical data, safety and security
provisions, controller instructions, and evaluation
criteria. Planning should be coordinated among onsite ERO
components and offsite organizations or groups regarding
their respective participation and exercise objectives.
3.6.1 Exercise Planning Team
The exercise planning team is responsible for designing,
developing, conducting, and evaluating all aspects of an
exercise. The planning team determines exercise design
objectives, tailors the scenario to the needs of the
participating organizations, and develops documentation used
in exercise evaluation, control, and simulation. Planning
team members also help with developing and distributing pre-
exercise materials and conducting exercise briefings and
training sessions. Due to this high level of involvement,
planning team members are ideal selections for controller
and evaluator positions during the exercise itself.
An Exercise Director manages the exercise planning team
(also referred to as the Lead Exercise Planner or Exercise
Planning Team Leader). The team should be a manageable size
and include a representative from each major participating
onsite and offsite emergency response organization/agency,
with team membership modified to fit the type or scope of an
exercise, (e.g., an operations-based exercise may require
more logistical coordination than a discussion-based
exercise). Depending on the individual exercise, planning
team members can be drawn from a variety of response
disciplines within a DOE/NNSA facility/site
(e.g., fire/hazardous materials, emergency medical services,
security, emergency management, occupational health, and
emergency public information). The planning team may expand
or contract in size according to the scope of a given
exercise, causing a member or group leader to assume
additional roles.
Exercise planning teams should follow a combination of
common considerations and principles, including:
Exercise planning teams can be efficient and effective
when they adhere to an Incident Command System (ICS)-
based structure.
Planning teams should be formed in advance of the
exercise to ensure adequate time for effective planning,
preparation, and review of the exercise package.
Members assigned to these teams should be familiar with
emergency plans and procedures in their areas of
technical expertise and be experienced in exercise
development.
Team members may work independently or meet in subgroups
to develop their respective parts of the scenario;
members can participate in more than one team, if
necessary.
Effective project management ensures identification,
development, and management of critical and supportive
tasks; frequent communication about project status; and
use of management plans and timelines (e.g., task
schedules, Gant charts).
Exercise planning team members should be aware of both
their individual responsibilities and team
responsibilities. Tasks should be identified and
assigned to an appropriate planning team member, and
clear deadlines should be established.
Subject matter experts (SMEs) should be used in the
planning process to ensure that a realistic and
challenging scenario is chosen. For example, in a
biological terrorism scenario, public health departments
and hospitals will have larger roles than special weapons
and tactics teams or the bomb squad.
Certain exercise objectives may require detailed
technical or specialty areas of expertise for the
development of scenarios, injects, and data. In these
situations, a special team can be formed. Typically,
this expertise is in specific areas or disciplines such
as process operations, health physics, medical,
chemistry, safety engineering, or plume modeling.
Team members should demonstrate appropriate leadership
principles, including mentoring, motivation, discipline,
personnel management, and time management. Team leaders
and members should delegate tasks as necessary. Planning
team members should strive toward group and common goals,
using all available expertise while fostering creativity.
Team members should implement standardized processes
(such as incorporating task, time, and project
management) into exercise design and development.
Exercise planning meetings/conferences should be
scheduled to develop and review tasks and outputs.
Both DOE/NNSA and contractor senior management
representatives should be briefed to gain their support.
Coordination with the emergency management training
program manager (if designated as a trusted agent) should
occur in the exercise planning stage. This allows
sufficient time before an exercise is conducted to
satisfy any new management training or qualification
requirements (not specific to an exercise).
Exercise planners are to consider themselves as trusted
agents and understand that, in most cases, they will
participate as facilitators or SMEs, rather than as
participants. Planning team members, as a general rule, are
not exercise players except at smaller, less populated
DOE/NNSA facilities with limited emergency response/
management capabilities. In those cases, exercise planning
team members who act as both planners and players should be
especially careful not to divulge exercise information in
advance.
3.6.2 Exercise Planning Functions
Development and conduct of a DOE/NNSA exercise requires a
structured and coordinated planning process. For each
exercise, the following list includes several key functions
or activities that should be accomplished at some level,
depending on the type and scale of the exercise:
Development, documentation, and scheduling.
Scope - Who, what, where, how, and why of the exercise.
Objectives - Specific objectives provide the basis for
evaluating/validating the performance of response
capabilities by each participating organization. Each
exercise objective should clearly state what is to be
demonstrated and be specific, attainable, and
measurable.
Participants - Who will plan the exercise and who will
respond, control, and evaluate. Any limitations or
simulations regarding their participation are
identified and documented.
Safety - Safety is an integral part of each exercise.
The exercise should be conducted in a manner that
protects workers and other personnel and does not cause
harm to the environment.
Security - Instructions on facility access, use of
firearms, and classification issues.
Scenario Technically accurate mechanism developed to
provide responders with the opportunity to meet
objectives. The scenario is consistent with the set of
exercise objectives and explicitly supports an
evaluation/validation of each objective.
Budget - What the exercise will cost to plan, conduct,
and evaluate, and the financial obligations of
participating organizations.
Logistical Support - Specific responsibilities for
support activities.
Administrative Activities - Procurement, documentation,
and reproduction responsibilities.
Public Affairs Plan - A public information/education
plan should be developed, especially for full-
participation site-level operations-based exercise, to
coordinate activities with appropriate offsite State,
Tribal, and local authorities, the media, and the
public. This plan should be developed early in the
planning process to ensure coordination with interested
offsite authorities/officials.
Oversight of the exercise development process.
Exercise control
Exercise evaluation and critiques
Exercise AAR
Implementation of corrective actions
These key functions or activities should be reflected in the
Exercise Planning Team structure presented next.
3.6.3 Exercise Planning Team Position Descriptions
Providing exercise planning team members with clearly stated
roles and responsibilities, along with assigned specific
tasks and completion timelines, will facilitate the exercise
planning process by ensuring that tasks are not overlooked,
forgotten, or identified only at the last minute.
Regardless of the size of an exercise planning team certain
core groups must be formed as described below:
Exercise Director/Lead Exercise Planner
Overall responsibility for exercise planning,
development, conduct, and evaluation
Planning Group
Schedules exercise activities
Determines exercise scope, objectives, participants, and
planning schedule
Develops scenario guidelines
Coordinates administrative, logistics, safety, and
security activities
Maintains fiscal responsibility
Scenario Development Group
Includes members from all participating organizations
Coordinates development, assembly, and production of
exercise package
Develops scenario component, including, for example
scenario narrative, Master Scenario Events List (MSEL),
time line, injects and messages, and the exercise
technical data
Control Group
Responsible for the safe and effective conduct of the
exercise
Exercise control
Safety
Evaluation Group
Responsible for observing, evaluating, and critiquing the
exercise
Other planning groups (tasks may be separated from above
main teams)
Administration/Logistics
Communications
Technical data preparation
Security/safety
Public information
Visitors
If an ICS management model is used for organizing the
Exercise Planning Team, then the groups described above will
be formed under the headings of core ICS groups: Command
Group, Operations Group, Planning Group, Logistics Group,
and Administration Group (Cf. HSEEP reference).
During the planning process, as tasks increase in frequency
and complexity, the planning team grows. It may be
necessary to expand positions to include several functional
experts or SMEs and additional logistical support or service
staff. Many large, complex exercises may start with a
planning team that fills most, if not all, of the
organizational structure represented by the groups indicated
above.
3.6.4 Exercise Planning Meetings/Conferences
The Exercise Director/Lead Exercise Planner and the exercise
planning team should decide on the number of meetings needed
to successfully conduct a given exercise. To effectively
host planning meetings, the Exercise Director needs access
to information on the program, its objectives, and its
flexibilities and limitations. Listed below are basic
descriptions of primary objectives for each type of planning
meeting along with information on tools (e.g., agendas,
draft documents, checklists, and presentations) used to
assist the exercise planning team in designing, developing,
and conducting an exercise.
Providing advance information to the planning team members
significantly enhances the efficiency of a planning meeting.
These materials may be provided to team members in a read-
ahead packet, which may include proposed agenda items, any
relevant background information, and expected meeting
outcomes. In addition to making the attendees better
informed, a read-ahead packet also allows them to understand
the relevancy and importance of the meeting.
The scope, type (operations- or discussion-based), and
complexity of an exercise should determine the number of
meetings necessary to successfully conduct an exercise.
Planning meetings are listed in typical chronological order:
A.Concept and Objectives (C&O) Meeting. The C&O meeting is
used to identify the type, scope, objectives, and purpose
of the exercise. Typically attended by the Exercise
Director, management of the sponsoring facility, and DOE
and Management & Operating (M&O) contractor management
responsible for other facilities and the response assets
expected to be involved in the exercise, and
representatives from participating offsite response
organizations. The formal beginning of the planning
process (when held directly before the Initial Planning
Conference/Meeting [IPC]) helps planners identify an
overall exercise goal, develop rough drafts of exercise
objectives, and identify exercise planning team members.
For less complex exercises and for sites or facilities
with limited resources, the C&O meeting can be conducted
in conjunction with the IPC.
Possible topics or issues for a C&O meeting include:
Exercise purpose
Review of the facility/site or activity
Review of the applicable EPHA
Proposed scenario, goals, and objectives
Exercise location, date, and duration
Assumptions and artificialities (the scenario is
plausible and events occur as they are presented; there
are no hidden agendas or trick questions)
Control and evaluation
Security organization and structure
Logistics
B.Initial Planning Conference/Meeting (IPC): The IPC lays
the foundation for exercise development. It is used to
gather input from the exercise planning team on the
scope, design, requirements and conditions (such as
assumptions and artificialities), objectives, level of
participation, and scenario variables (e.g., hazard
selection, venue). The IPC obtains the planning teams
input on exercise location, schedule, duration, and other
details required to develop exercise documentation.
Planning team members should be assigned responsibility
for the tasks outlined in the meeting. Unless a C&O
meeting is held, the IPC is typically the first step in
the planning process.
During the IPC, planning team members are assigned
responsibility for tasks associated with designing and
developing exercise documents and logistics
(e.g., incident scene management, personnel). In
addition to conducting the meeting, the team should
gather appropriate photos and audio recordings for use in
preparing the final documents and/or multimedia
presentations used in support of the exercise. An
important discussion point for an IPC includes
understanding the rationale for developing the exercise.
Outcomes expected as a result of an IPC include:
Clearly defined, obtainable, and measurable objectives
Identified scenario variables (e.g., threat scenario,
number of casualties, venue)
Appropriate participants are invited to participate
SMEs and presenters identified and recruited, if
necessary
Information delivery method identified
Responsibility assigned for exercise documents and
presentations/briefings
All source documents (e.g., policies, plans,
procedures) acquired to draft exercise documents and
presentations
Responsibility for logistical issues
(e.g., registration, badges, invitations) identified
and assigned
Dates for completion of action items and tasks
established
Planning schedule developed
Critical tasks for the next planning meeting identified
Date, time, and location of the next meeting and the
actual exercise
In the period between the IPC and the next meeting,
exercise planning team members should prepare their
assigned draft exercise documents and presentations. If
possible, these materials should be provided to planning
team members in advance of the next meeting.
C.Mid-Term Planning Conference/Meeting (MPC): Typically
employed for operations-based exercises (e.g., drills,
FEs, and FSEs), the MPC presents an additional
opportunity in the planning timeline to settle logistical
and organizational issues that arise during planning
(e.g., staffing concepts, scenario and timeline
development, scheduling, logistics, and administrative
requirements). It is also a session to review draft
documentation. A MSEL meeting can be held in conjunction
with or separate from the MPC to review the scenario
timeline for the exercise.
The second half of the MPC can be devoted to development
of the scenario timeline. If necessary, the exercise
planning team may allow sufficient time to conduct a
walkthrough of the exercise site and gather pictures,
maps, and other visual aids. The MPC should be held at,
or near, the exercise site to facilitate the walkthrough.
The following outcomes are expected as a result of an
MPC:
Final exercise plan details (if applicable) agreed upon
Scenario timeline revised
Documentation revised
Scenario injects developed
Date, time, and location of the Final Planning
Conference/Meeting (FPC) finalized
The time between the MPC and the FPC should be used to
finalize the exercise plan, scenario timeline, and
remaining exercise documentation (as determined at the
IPC).
D.Master Scenario Events List (MSEL) Meeting: For more
complex operations-based exercises, one or two additional
planning meetings, or MSEL meetings, may be held
specifically to review the scenario timeline. If not
held separately, MSEL meetings can be incorporated into
the MPC and FPC. The MSEL meeting should focus on MSEL
development.
The MSEL is a chronological list that supplements the
exercise scenario with event synopses,
expected/anticipated responses, anticipated timing and
location of the opportunity for meeting exercise
objectives, and responsible personnel. The MSEL should
include scenario events that will prompt players to
implement the plans, policies, and procedures that
planners want the exercise to test. The MSEL identifies
the timing and summary content of all key events,
messages, or injects and contingency messages. It should
also establish the methods that will be used to inject
each particular event (e.g., phone call, fax, radio call,
e-mail).
In developing the MSEL, the exercise planning team must
first consider tasks, conditions, and standards set forth
by each exercise objective (as determined during the
IPC). A task consists of performing a function or
activity that demonstrates the ability to accomplish an
objective. A condition is the environment in which a
task is performed and can be established by the scenario
or through the MSEL. Standards are the criteria by which
each task is evaluated. The planning team determines if
tasks were completed; this allows evaluation to take
place.
If scenario conditions do not stimulate the appropriate
behavior, the planning team must develop a MSEL entry to
explain the situation. A well-written entry considers
the following:
If the entry is a key event (i.e., is it directly
related to meeting an exercise objective)
The target behavior, who will demonstrate the target
behavior, and what will stimulate the behavior
(e.g., course of play, phone call, actor, video)
What/Who originates the stimulant, who receives it, and
how is it received
What the expected action is
A contingency inject in case the behavior fails to be
demonstrated
Once the MSEL is drafted, the exercise planning team
should coordinate and sequence entries and resolve any
conflicts between events, thus forming a credible and
challenging MSEL that will enhance the exercise
experience for the players. It is essential that the
final MSEL be reviewed with quality assurance procedures
in mind.
E.Final Planning Conference/Meeting (FPC): The FPC provides
a forum to review the process and procedures for
conducting the exercise, final drafts of all exercise
materials, and all logistical requirements. There should
be no major changes made to either the design or the
scope of the exercise or to any supporting documentation.
The FPC ensures that all logistical requirements have
been arranged, all outstanding issues have been
identified and resolved, and all exercise products are
ready for printing. The FPC should be located in close
proximity to the planned exercise site to allow a final
site walkthrough. The facility should be conducive to
discussion and accomplishment of work-related tasks.
The following items should be addressed during the FPC:
Resolve any open issues related to exercise planning
and identify last-minute concerns that may arise
Review all exercise logistical tasks (e.g., schedule,
registration, attire, special needs)
Conduct a comprehensive final review of and approve all
exercise documents and presentation materials
The FPC should not generate any significant changes or
surprises. Outcomes should include finalization of
exercise documents and multimedia presentation materials
for production, identification and resolution of last-
minute issues, and coordination of other support
requirements (e.g., A/V equipment, room configuration and
setup, refreshments, and schedule). Final approval of
exercise processes and procedures should occur.
Contact should be maintained between all exercise
planning team members regarding any outstanding issues,
especially issues related to the logistics for conducting
the exercise. The planning team should finalize all
publications, prepare all supporting materials, rehearse
presentations and briefings, and prepare to conduct the
exercise. Prior to the exercise, information and
documentation should be disseminated to appropriate
personnel (e.g., presenters, facilitators, controllers,
evaluators, simulators).
3.6.5 Exercise Planning Schedule
Planning and scheduling an exercise requires the involvement
and cooperation of all participating organizations. A well
planned, executed, and documented exercise requires the
coordination and cooperation of senior management, facility-
and site-level EROs, and, when applicable, offsite response
organizations. Participating offsite response organizations
must be included in the initial planning stages of the
exercise. Their participation may range from the limited
staffing of a control cell for the purpose of receiving
notifications to the complete staffing and activation of all
applicable response facilities and assets. In planning the
exercise, adequate time should be allowed for effective
preparation and review of the exercise plan.
Table 3-1 contains a sample schedule applicable to a
DOE/NNSA complex site-level exercise; planning times should
be adjusted down for smaller scale exercises.
Table 3-1. Sample Planning Schedule for a Site-level
Annual Exercise
Calendar Planning Activity
Days Prior
To the
Exercise
365 Establish or confirm exercise date.
Establish exercise scope.
270 Establish planning organization.
Confirm scope and level of participation by all
organizations.
Develop initial exercise objectives.
180 Verify plans and procedures to be used.
Begin scenario development.
150 Finalize exercise objectives.
90 Submit scenario narrative, scope, MSEL (draft),
objectives and participant list to Cognizant Field
Element, Program Secretarial Office and Associate
Administrator, Office of Emergency Operations (NA-
40).
60 Complete planning group review/revision of draft
Exercise Plan (EXPLAN).
Submit EXPLAN to DOE/NNSA Cognizant Field Element
for approval. Plan must be approved at least 30
days prior to exercise.
30 Submit approved EXPLAN to Program Secretarial
Office and Associate Administrator, Office of
Emergency Operations (NA-40).
15 Complete generic controller/evaluator training.
1-5 Conduct exercise-specific controller/evaluator
training.
Conduct responder and observer briefings.
1-2 Finalize exercise preparations.
Post Conduct critiques.
Exercise
Post Complete draft AAR.
Exercise 15
Post Finalize AAR and submit a copy to Cognizant Field
Exercise 45 Element, Program Secretarial Office and Associate
Administrator, Office of Emergency Operations (NA-
[~30 40).
working
days]
Post Develop/prepare corrective and improvement actions
Exercise 85 and submit a copy to Cognizant Field Element,
Program Secretarial Office and Associate
[~60 Administrator, Office of Emergency Operations (NA-
working 40).
days]
3.7 Exercise Documentation
Typical exercise documents resulting from the efforts of the
Exercise Planning Team are addressed in the following
sections. They provide essential components for preparing,
conducting, and evaluating exercises.
3.7.1 Situation Manual (SITMAN)
The SITMAN is a participant handbook for discussion-based
exercises, particularly TTXs. It provides background
information on the exercise scope, schedule, and objectives.
Also included is the scenario narrative that will drive the
participant discussions during the exercise. The SITMAN
should mirror the multimedia briefing and supporting
narrative allowing participants to read along while watching
events unfold.
3.7.2 Exercise Plan (EXPLAN)
An EXPLAN or exercise package, typically used for operations-
based exercises, is published and distributed prior to the
start of an exercise. The EXPLAN provides a synopsis of the
exercise and addresses the exercise objectives and scope.
The EXPLAN assigns tasks and responsibilities for successful
exercise execution and provides documented components
essential for preparation, conduct, and evaluation of the
exercise. The EXPLAN contains all the documentation
necessary to control and evaluate the exercise, however, the
extent and detail of the information will vary with the
scope and complexity of the particular exercise.
The EXPLAN contains an explanation of the exercise and
provides the documented components essential for
preparation, conduct, and evaluation of the exercise.
Development of an EXPLAN by an exercise planning team
involves an iterative process consisting of several steps:
Address issues of exercise scope and duration,
participants, objectives, administrative and logistical
considerations, and operational or technical constraints
Develop a scenario timeline, a listing of the sequence
and timing of key operational, technical, and logistical
events comprising the scenario
Refine the timeline, develop detailed scenario
information, prepare message injects (instructions to
controllers) and data, and prepare control, evaluation,
and other supporting documentation
This iterative development and refinement process is
followed by a final review. Final review of the EXPLAN is
conducted to ensure overall completeness and technical
accuracy and that players/responders are provided the
opportunity to meet the exercise objectives. The EXPLAN
should be completed in sufficient time to allow DOE or NNSA
line management and the DOE Associate Administrator of
Emergency Operations to review and comment before the
conduct of the exercise. However, some elements of the
EXPLAN, such as, telephone directories or lists containing
names of controller/ evaluators will be subject to change up
to the conduct of the exercise. Prior to final review and
distribution, a Derivative Classifier (DC) should review the
EXPLAN.
The scenario reflects current facility/site- or activity-
specific hazards, correlates technically with the EPHA, and
is technically accurate in terms of operations and
radiological, chemical, and meteorological data. A unique
scenario should be developed for each exercise to prevent
responder anticipation of events and to ensure a valid test
of integrated response capabilities. The final approved
EXPLAN or exercise package provides documentation to conduct
and evaluate the exercise.
The EXPLAN or exercise package contains all documentation
necessary to control and evaluate the exercise, however, the
extent and detail of information will vary with the scope
and complexity of the particular exercise. The format can
be tailored by individual organizations but should include
the information outlined below. The exercise web site at
http://www.orise.orau.gov/emi contains specific examples of
EXPLANs. A typical DOE/NNSA EXPLAN includes the following
components:
A.Scope and Purpose. All participating organizations, the
extent of their participation, and the organizations
being simulated are identified. States the purpose of
the exercise. Contains the type of exercise, the
location of the event scene (e.g., specific facility),
facility/site or activity background information, and the
date and expected duration.
B.Exercise Objectives. The objectives are the key to the
exercise. Each exercise objective should clearly state
what is to be demonstrated and should be specific,
attainable, and measurable. They should contain specific
conditions, performance/action, and standard of
performance to define how the objective is to be
evaluated. Exercise objectives are discussed further in
Section 3.8.
Use of an exercise objective matrix is recommended as a
tool to facilitate administration of the exercise
program. The matrix should identify all programmatic
exercise objectives and correlate with facility/site- or
activity-specific hazards and the specific objectives to
be demonstrated in individual exercises. It should
support/document validation of emergency management
Program Elements over the five-year period. In order to
test the interfaces between site security and
facility/site emergency response capability, the exercise
program at a DOE/NNSA facility/site should include
security scenario events.
C.Exercise Organization. The exercise organization is
comprised of all participants in the actual conduct of
the exercise and includes the following:
Exercise Director/Lead Exercise Planner ensures
exercise conducted according to the EXPLAN
Players include actual responders and onsite and
offsite organizations
Evaluators are subject mater experts who observe,
monitor, and evaluate player performance; they are
responsible for critiques and final AAR
Controllers ensure that the exercise proceeds on
schedule; monitor the sequence of events and input
contingency injects to keep exercise on time; monitor
safety; input technical data at appropriate times in
the scenario; and, assist with critiques and the final
AAR
Observers/visitors should have no interaction with
players/responders
D.Scenario Narrative. The scenario narrative is a
storybook summary of the background, initial conditions,
initiating events, and expected player/responder actions.
It contains descriptions of the simulated emergency
situation, including the overall sequence of events,
details, supporting data, and timing of activities.
The scenario reflects current facility/site- or activity
specific hazards, correlates technically with the EPHA,
and is technically accurate in terms of operations,
radiological, chemical, and meteorological data. A
unique scenario should be developed for each exercise to
prevent player/responder anticipation of events and to
ensure a valid test of integrated response capabilities.
E.Rules of Conduct. Design and development guidelines are
established for each exercise and include:
Limitations are management policies and guidelines of
concern to the exercise developers and scenario
designers; include issues such as conducting exercises
on weekends, overtime restrictions or authorizations,
and financial constraints.
Protocols (ground rules or rules of conduct) remind
players/responders of drillsmanship and safety issues.
Pre-approved simulations list the major simulations
applicable to the exercise; may include pre-determined
meteorological data, response vehicle red lights,
simulating road blocks without interfering or
disrupting public traffic patterns, use of water to
simulate a chemical liquid hazardous materials spill,
use of a smoke generator to simulate fire/smoke, use of
protective equipment, simulated operation of
systems/equipment, and photographs to simulate
equipment damage.
F.Safety Issues. Safety of personnel and the facility is
paramount during exercises. A major concept of DOE
Integrated Safety Management (ISM) is the integration of
safety awareness and good practices into all aspects of
work conducted at DOE. Simply stated, exercises should
be conducted in such a manner that protects workers and
other people, and does not cause harm to the environment.
Safety is an integral part of each exercise; it is not a
stand-alone program.
The planning process and management of exercises must
ensure that sufficient precautions and limitations are
established and followed for safe conduct of the
exercise. A person with the sole responsibility for
ensuring safety during the exercise, such as an exercise
safety director, should be appointed to the exercise
planning team.
During an exercise, all participants must comply with
established safety rules and practices. Participants
must understand that safety of exercise participants, non-
participants, the public, and the environment is the
highest priority. An exercise safety plan is an
effective method of documenting safety concerns and
solutions. The plan should address generic and specific
safety concerns, solutions for mitigating the problem,
and required actions or notifications if a safety concern
or emergency occurs during an exercise.
G.Security and Access Planning. Adherence to security
requirements by all participants in all phases of an
exercise is a necessity. Planning and management of
exercises should include provisions for the participation
of appropriate security personnel and should establish
parameters for exercise design, development, and conduct
in view of identified security issues. Controllers are
responsible for conducting the exercise within security
limitations.
Persons involved in exercise planning must be sensitive
to information or activities that may have security
implications. An exercise security plan is an effective
method of documenting security concerns and solutions.
This plan should address generic and specific security
concerns, mitigative solutions, and required
actions/notifications if a security problem or emergency
occurs during the exercise. Special provisions should be
made for visitors and observers since they may not be
familiar with DOE or site security requirements.
H.Public Information Planning. Scheduled exercises,
especially large scale operations-based exercises, should
be coordinated with the media and announced to the
public. Interfaces with the public and offsite Tribal,
State, and local authorities require management awareness
and sensitivity.
The public typically has no involvement or participation
in an exercise. However, all exercises conducted at a
facility/site that have the potential to affect the
offsite population, either directly or indirectly, should
include adequate provisions to prevent public concern,
rumor, or inconvenience. The planning process and the
management of exercises should provide for the
development of a public information/education plan to
coordinate activities with appropriate offsite Tribal,
State, and local authorities, the media, and the public.
This plan should be developed early in the planning
process to ensure coordination with interested offsite
authorities/ officials.
I.Timeline of Key Scenario Events. The exercise timeline
should include key scenario events and expected responder
actions and, where possible, the events and
player/responder actions should be tied to exercise
objectives.
J.Message Injects. Message injects include instructions to
controllers to begin simulations, insert information,
provide earned information, acting instructions, and
contingency messages. Message injects should contain
accurate, unambiguous, and non-prompting information and
technical data for the players/responders and provide
proper direction for the exercise. They should be
formatted/presented in a manner as to reflect the actual
data that would be observed by players/responders in a
real event (e.g., strip charts, alarm printer output, use
of accident mock-ups).
K.Exercise Data. Exercise data varies greatly depending on
the type and scope of the exercise. For example, the
data requirements for a NNX are limited by design in
order to minimize the resources needed to conduct this
type of exercise. The technical data that supports the
scenario, including both general and facility-specific
(e.g., operational, radiological, chemical, medical,
meteorological), should be technically accurate and
clearly and unambiguously presented:
General facility information is especially important
when non-facility personnel participate in the
exercise; includes a facility description, (area, site,
and facility maps), mission description, emergency
management program information, and a description of
offsite interfaces.
Specific facility information provides operational data
at the time of the event; may include diagrams,
schematics, and data tables that will augment the
scenario.
Meteorological data provides weather conditions and
forecasts, both real and simulated, as required.
Hazardous material data may include radiation or
chemical plume plots and tables, decontamination
levels, and exposure levels; technical basis and
assumptions used to develop this data should be
provided.
Medical information includes a description of medical
conditions and moulage procedures, actor behavior
instructions, and vital signs.
L.Exercise Control. The control organization is
responsible for controlling the exercise and is usually
depicted on an organizational chart showing the
categories of controllers and lines of communication.
The categories of controllers include the lead
controller, timeline coordinator, area controllers, on-
scene controllers (i.e., at the location of the activity
to be controlled), the control or simulation cells and
their associated actors. Controllers are assigned by
name to each position listed in the control organization.
The controller assignments should include alternates.
Detailed controller instructions include a schedule of
events for all controllers, basic controller
instructions, and requirements for each controller
assignment. These should include the message injects
that the controller is responsible for inserting in the
exercise, contingency message injects and the
authorization process for their use, and special
equipment required for the position. A special type of
controller instructions, called profiles, can be used for
actors to define roles. Profiles are normally used for
media actors in either a control cell or for interviewing
in person or for control cell actors representing
political figures (profiles are generally used only with
experienced controllers).
The Master Scenario Events List (MSEL) identifies the
timing and summary content of all key events, messages,
or injects; contingency messages; and expected responder
actions for the duration of the exercise.
Suspension or termination of an exercise is managed
through the control organization. Responders are
instructed to contact a controller when an unsafe
condition exists or when a real emergency is identified.
This section details the notification of the control
organization, instructions for exercise suspension and re-
start, and for exercise termination.
M.Exercise Evaluation. Exercise evaluation is conducted by
the evaluator organization. The evaluator organization
is usually depicted by a chart and description of the
categories of evaluators and lines of communication. The
categories of evaluators include the lead evaluator, lead
area evaluators, and evaluators. Evaluators and
alternates are assigned by name and listed in the
evaluator organization.
Evaluation criteria provide the standards and activity-
or function-specific criteria used to evaluate an
exercise. Evaluator modules and/or checklists display
the expected response in a time-sequenced format used to
monitor player/responder progress. They are based on the
exercise objectives, the evaluation criteria, and the
participating organization's plans and procedures.
N.Logistics. A logistics plan is prepared to specify tasks
to be accomplished in support of exercise preparation,
conduct, and evaluation. This includes notification of
controllers, obtaining meeting rooms and classrooms,
identifying and setting up the control cell,
communications requirements, meals, transportation,
facility security badging/access, and acquiring/staging
props (e.g., moulage dummies, smoke generators, damaged
equipment, simulated material).
A method to identify exercise participants and, if
necessary, various non-participants should be documented
in this section. Vests, hats, or armbands of various
colors can identify participants. Ensure that the type
of participant is printed on the identification method to
assist those with color-impaired vision.
O.Schedule of Events. A master schedule should be
developed that addresses all preparation activities,
conduct of the exercise, the critique process, and the
evaluation AAR.
P.Communications Plan. This plan documents radio and
telephone requirements. It provides radio frequencies,
protocol, telephone numbers, and directories.
Additionally, it contains information concerning
controller communications, training, and systems testing.
Normally the following exercise telephone
(communications) directories are prepared:
Control Cell Directory provided to responders, which
lists the control cell telephone numbers of controllers
simulating individuals or organizations
Controller/Evaluator Directory which includes telephone
and radio channels/frequencies used for communication
within the control and evaluation organizations
Responder Directory is provided to control cell
controllers and lists the telephone numbers of
responders who may need to be contacted by the control
cell
Q.Glossary of Acronyms. This section contains
facility/site- and activity-specific acronyms and
definitions for the benefit of personnel who are not
familiar with the ERO, operations, and facility/site or
activity organization.
3.7.3 Controller and Evaluator Documentation
The Controller and Evaluator (C/E) Handbook supplements the
EXPLAN by presenting more detailed information about the
exercise scenario and describing exercise controller and
evaluator roles and responsibilities. The C/E should only
be distributed to individuals specifically designated as
controllers or evaluators. Larger, more complex exercises
may use the Control Staff Instructions (COSIN) and an
Evaluation Plan (EVALPLAN) in place of, or to supplement,
the C/E handbook.
Controllers ensure that player/responder behavior remains
within predefined boundaries. Simulation Cell (SIMCELL) or
Control Cell controllers continuously inject scenario
elements to simulate real events. Evaluators observe
behaviors and compare them against established plans,
policies, procedures, and standard practices (if
applicable). Safety controllers ensure all activity is
executed within a safe environment.
In addition to containing the same information as the
EXPLAN, the C/E Handbook usually contains the following
sections:
Detailed scenario information (including agent fact
sheets)
Assignment of personnel to specific controller/evaluator
positions
Roles and responsibilities of functional area or
individual controllers and evaluators
Controller communications plan
Exercise Evaluation Guides (EEGs)
The Control Staff Instructions (COSIN) document contains
guidance that controllers, simulators, and evaluators need
concerning procedures and responsibilities for exercise
control, simulation, and support. COSINs are typically
developed for large-scale, complex exercises that require
more coordination among control staff. The purpose of a
COSIN is to:
Provide scenario details
Develop guidelines for control and simulation support
Explain the exercise concept as it relates to controllers
and simulators
Establish management structure for these activities
Establish and define the control structures
communication, logistics, and administration
The Master Scenario Events List (MSEL) is a chronological
timeline of expected actions and scripted events to be
injected into exercise play by controllers to generate or
prompt player activity. The MSEL ensures that necessary
events happen so that all objectives are met. The MSEL
links simulation to action, enhances exercise experience for
players, and reflects an incident or activity that will
prompt players to implement the policy or procedure being
tested. Larger, more complex exercises may employ a
Procedural Flow (PROFLOW), which differs from the MSEL in
that it only contains expected player actions or events.
Each MSEL record contains:
Designated scenario time
Event synopsis
Controller responsible for delivering inject, with
controller/evaluator special instructions (if applicable)
Expected action (player response expected after an MSEL
inject is delivered)
Intended player (agency or individual player for whom the
MSEL inject is intended)
Objective to be demonstrated (if applicable)
Notes section (for controllers and evaluators to track
actual events against those listed in the MSEL, with
special instructions for individual controllers and
evaluators)
Times listed in an MSEL should reflect the times that
injects should occur. These times should be as realistic as
possible and should be based on input from functional area
representatives. For example, to determine when triage and
treatment should be established during the exercise, solicit
input from emergency medical services (EMS) or a hospital
representative. If the activity occurs sooner than
anticipated, the time should be noted but play should not be
interrupted.
There are three types of injects:
Contextual injects are introduced to a player by a
controller to help build the contemporary operating
environment. For example, if the exercise objectives
include information sharing, an MSEL inject can be
developed to direct a controller to select an actor to
portray a suspect. The inject message would then
instruct the controller to prompt another actor to
approach a security officer and inform him/her that this
person was behaving suspiciously.
Earned information injects contain data to be provided to
a responder when the function being performed would yield
specific information. For example, when Emergency
Medical Service (EMS) technician(s) begins to treat a
patient, vital signs are provided through these injects
when the proper actions to earn them have been achieved.
If the EMS does not perform the appropriate actions, no
inject information is provided.
Contingency injects are events that should be verbally
provided to a player by a controller if they do not take
place. Use of contingency injects during the exercise
should be documented. For example, if a simulated
secondary explosive device is placed at an incident scene
but is not discovered, a controller may want to prompt an
actor to approach a player/ first responder and say that
he/she witnessed suspicious activity close to the device
location. This should prompt the discovery of the device
by the responder and result in subsequent notification of
protective force (perhaps, specifically, the bomb squad).
MSELs are typically produced in two formats, short and long.
Short MSELs list: injects, the time, a short description,
the responsible controller, and a player. These can be used
as a quick reference guide during exercise play. Long MSELs
are used when greater detail is necessary; they include more
detailed descriptions, exact quotes for SIMCELL injects, and
descriptions of expected actions.
Message injects are typically used in exercises that involve
multiple simulated activities. These messages are typically
delivered via a SIMCELL and are used to simulate the
actions, activities, and conversations of an individual,
agency, or organization that is not participating in the
exercise but that would likely be actively involved during a
real event. For example, in an exercise with limited scope,
the State Governors office may not be playing. To simulate
the activities of the Governors office during an emergency
event, a message can be scripted to simulate notification of
the mayor. That message can be delivered by phone through
the SIMCELL. This script or message inject should be read
by a simulator acting on behalf of the Governors office.
Evaluation Plans (EVALPLANs) provide evaluation staff with
guidance and instructions on the evaluation or observation
methodology to be used as well as essential materials
required to execute their specific functions. During
larger, more complex exercises, planners may develop an
EVALPLAN in lieu of, or in addition to, a C/E Handbook. The
EVALPLAN is a limited distribution document that evaluators
use in conjunction with the EXPLAN and the MSEL. Level of
detail varies and can include the following:
Exercise overview
Evaluation control organization
Evaluation methodology and observation techniques
Evaluator roles and responsibilities
Evaluation communications plan
Exercise Evaluation Guides (EEGs) are developed to assist in
exercise evaluation. They incorporate the critical tasks
that should be completed in an exercise. EEGs are intended
for use by both experienced evaluators and SMEs who may have
little or no exercise evaluation experience. EEGs provide
evaluators with information on what they should expect to
see at the specific location or in the specific situation.
(See Section 3.11)
Controller and Evaluator Packets are provided to controllers
and evaluators prior to an exercise. The packets contain
materials that they need to carry out their
responsibilities. These materials can be extracted from the
more detailed information found in the C/E Handbook or the
COSIN.
A controller packet should contain:
Essential C/E Handbook or COSIN information
(e.g., scenario and threat/hazard information,
communications, safety, exercise staff organization)
MSEL (including injects for each responsible controller)
Maps/directions
These materials should be placed in a packet (e.g., folder,
notebook) for ease of use during the exercise.
3.8 Exercise Objectives
The success of emergency exercises is largely dependent on
the quality of the stated objectives. Exercise objectives
provide the basis for developing a meaningful and
challenging scenario, as well as a gauge to measure
performance of the response elements.
3.8.1 Developing Exercise Objectives
Considerations in developing exercise objectives include the
following:
Primary sources of exercise objectives are the
participating organizations emergency plans and
procedures. Other sources may include lessons learned
from past exercises, the specific plans/procedures being
exercised, and job-task analyses used to develop the
organizational response structure, requirements, or
training.
A recommended source for the specific exercise objectives
to be used in validating and testing components of the
response is the set of performance-based evaluation
criteria contained in Chapter 4, Appendix D of
DOE G 151.1-3. Each of the 15 DOE/NNSA emergency
management Program Elements has a performance goal and a
set of evaluation criteria that represent the expected
performance to achieve that goal. An individual
criterion or groups of criteria can be used as a single
exercise objective.
Exercise objectives need to be fully developed for all
organizations prior to the preparation of the scenario
script. If offsite organizations are participating in
the exercise, they should develop their own objectives
for evaluation during the exercises and these objectives
should be included in the exercise plan. These
objectives are not evaluated by the DOE/NNSA evaluators
but should be reviewed to ensure that the objectives that
rely on site input are attainable and measurable.
DOE/NNSA should evaluate the interfaces between DOE/NNSA
and the offsite agencies, if this is part of an exercise
objective.
Each exercise objective should clearly state what is to
be demonstrated by the responders
Is the objective clearly stated? It should be
specific, focus on the performance to be demonstrated,
and be interpreted in the same manner by all
participants.
Is the objective attainable? The performance required
in the objective must be attainable (achievable). The
function or activity specified must be within the
capabilities of the responders to accomplish.
Is the objective measurable? The performance addressed
by the objective should have observable and measurable
indicators. Specific evaluation criteria should be
developed for measuring performance using a procedure
or checklist. The evaluation criteria given in
Appendix D and modified to be facility/site-specific
can be used for the evaluation of exercise objectives.
Exercise objectives should contain a condition, an
action, and a measurable standard. For example, given an
Operational Emergency (condition), activate the EOC
(action), in accordance with the Site EOC Emergency Plan
Implementing Procedure (measurable standard). The
condition provides the evaluator an understanding of the
conditions that have to occur prior to the responders
taking the action. This allows the evaluator to position
him/herself to observe the action. The action should be
clearly stated and attainable. This is the function that
the evaluator will observe and analyze to report
performance. The standard is the plan, procedure, and/or
regulatory requirement listing the steps to be taken by
responders to meet the exercise objective.
For purposes of identifying responsibilities, it is
useful to categorize or group objectives. Typically
objectives may be grouped by geographical area
(e.g., event scene, command center, collocated facility),
function (e.g., notification, consequence assessment,
protective actions, etc.), by organization
(e.g., Operating Contractor, operations/field
office/service center/Headquarters, state, local
organizations, etc.), or by relationship to the DOE
facility/site. For example, a grouping by relationship
to the DOE/NNSA facility/site could result in a
categorization of objectives, as follows:
Facility/site objectives - involve only site and
facility organizational units
Offsite objectives - involve only offsite units
Shared objectives - involve coordinated site and
offsite units
Special purpose objectives - designed to accomplish a
specific purpose and may involve site and/or offsite
units
Using the categories in the example listed above and
grouping by organization will assist in identifying the
entity responsible for determining the extent of exercise
play (the How much?), and, therefore, the organizational
unit responsible for preparation of its objectives. Joint
objectives will require the collaboration of more than one
entity. Special purpose objectives may be prepared to test,
for example, a new capability, such as a mutual aid
agreement, that would dispatch a medic response unit within
a specified time.
Once established, the objectives should clearly define the
extent of play (including offsite organizations in the
exercise), identify types of events to be included in the
scenario, and provide the entire framework on which the
exercise will be designed.
3.8.2 Basis for Exercise Objectives
A recommended methodology for developing exercise objectives
uses evaluation criteria contained in Appendix D of
DOE G 151.1-3 as a basis for specific objectives used to
validate and test components of the emergency response. The
appendix contains standardized generic evaluation criteria
for judging the performance of functions and activities
associated with each Program Element.
Appendix D contains standard evaluation criteria for each
Program Element. Each criterion is labeled to identify the
type(s) of evaluations to which it applies:
P - Program Evaluation
E - Exercise Evaluation
P/E - Program and Exercise Evaluation
CE - Conduct of Exercise Evaluation
The P criteria can be used for evaluating planning and
preparedness activities, and the expected performance of
responders during an emergency based on the evaluation of
plans, procedures, facilities and equipment, and through
interviews with personnel on the ERO. The E criteria are
used for evaluating the actual, observed performance of
responders during an exercise or actual emergency. P/E
criteria are appropriate for either Program or Exercise
evaluations. CE criteria should be used in evaluating the
conduct of an exercise.
To use this set of criteria, the specific issue to be
validated or tested, the corrective action, improvement
item, plan activity, or procedure is matched with one or
more of the Appendix D evaluation criteria. This general
criterion or criteria becomes the exercise objective.
Accomplishment and evaluation of this general objective will
be reflected in the references to specific plans,
procedures, or standards that are incorporated in
facility/site- or activity-specific evaluation criteria.
This approach ensures standardization of exercise objectives
and evaluation criteria, while providing a facility- /site-
or activity-specific evaluation of the objective.
For example:
Emergency Response Organization (ERO). Validate that the
ERO responds within the required timeframe and with the
required level of staffing. Applicable evaluation
criteria under the program element ERO is:
P/E6.9 The ERO is functionally staffed and activated in a
timely manner; key emergency response facilities
are operational within an hour after declaration
of an OE.
This represents the exercise objective. To produce the
criteria for evaluating this objective, specific time and
staffing standards located within the facility/site or
activity plans and procedures must be accessed. For
example, if the procedures state that the ERO is operational
within an hour after declaration of an emergency and is
functionally staffed when certain specified ERO members
arrive, then these become the evaluation standards to
incorporate in P/E6.9.
3.9 Exercise Preparation
Pre-exercise activities include configuring props or staging
equipment, establishing controller and evaluator
communications, specifying safety and security precautions,
making arrangements to feed participants, and making
arrangements for minimizing the impact on non-participants
and ongoing operations.
Coordination among participants prior to the exercise should
include provisions for exercise initiation, interruption,
and termination. All participants (players, controllers,
and evaluators) should be reminded of their responsibility
to prevent unsafe acts and to stop the exercise, if
necessary, to ensure that they do not occur.
3.9.1 Controller and Evaluator Training
Generic Training. Generic training should be developed and
conducted for individuals participating as controllers and
evaluators in an exercise. This training should include
both initial training and a periodic refresher prior to each
exercise.
Individuals with experience in the control and evaluation
of exercises should provide the initial training. It
should include a classroom-type presentation and
discussions of correct controller/evaluator performance
in various exercise circumstances.
Classroom-type presentation should address all aspects of
an exercise and include such topics as objectives,
safety, participants, realism, simulation, free play,
contingency messages, earned information, prompting, and
the evaluator and controller-responder interface.
Discussions should provide examples of circumstances that
may occur during an exercise with proper controller
actions. Emphasis should be placed on the criteria for
controllers to intercede in responder actions and the
criteria for suspending or terminating the exercise.
Exercise-Specific Training. Just prior to any exercise, all
controllers and evaluators must receive a briefing on the
scenario package and the specific duties they are to
perform. This may include a presentation on the various
plans and procedures that the responders are expected to
use. Controller briefings should cover the entire scenario
and anticipated responder actions, the location and
assignments of each controller (including actors),
communication plans, administrative and logistical details,
an in-depth presentation of safety and security issues, and
an in-depth discussion of each controller's specific
assignments. Details for controlling complex or sensitive
parts of the exercise should be presented in the briefing.
A tour of locations and associated equipment involved in the
exercise may be performed as part of the briefing; this may
NOT occur if a tour will result in compromise of the
exercise.
NOTE. In preparing for an NNX, a tour of locations and
associated equipment involved in the exercise may NOT
occur if a tour will result in compromise of the NNX. In
this case, a limited tour may be conducted during an
earlier visit, thereby not compromising the NNX program.
In some cases, the NNX team will not arrive on site or be
seen until the implementation of the NNX. It is at the
discretion of the NNX Lead Evaluator to conduct any
controller/evaluator in-briefs offsite to avoid the
possibility of compromising the NNX.
3.9.2 Responder and Observer Briefings
Responder Briefing. Should not include information related
to the scenario. Responders shall be briefed regarding
rules of conduct; scope of the exercise; safety and security
precautions; approved simulations; methods for identifying
various exercise participants; and any special
administrative, logistical, or communications arrangements
in effect during the exercise. Briefing pre-approved
simulations must be carefully considered, since some may be
very scenario-specific and may divulge too much advance
information.
NOTE. Because an NNX is an unannounced exercise, there
is no pre-exercise responder briefing. During conduct of
the NNX (at the first slow down during responder
actions), the NNX Lead Evaluator will conduct a short
brief to all participants (players, controllers, and
evaluators) of the sponsorship and purpose of this NNX,
as well as remind them of their responsibility to prevent
unsafe acts and to stop the NNX, if necessary, to ensure
an unsafe act does not occur.
Observer Briefing. Should occur prior to the exercise to
ensure compliance with safety and security precautions and
other rules of conduct. Observers may attend the controller
briefing or may be provided separate briefings.
3.9.3 Exercise Setup
Exercise setup should be documented in the logistics plan
and includes setting up simulations, preparation of scenes
and visual areas (e.g., smoke generators, simulated spills,
actor moulage, etc.), performing controller communications
checks, positioning controllers/evaluators, conducting
responder initial conditions briefings, synchronizing
clocks, initializing computer simulation data, and other
scenario-specific activities. Exercise setup should be
carefully planned to ensure that all logistics necessary to
conduct the exercise are checked before the exercise begins.
Security of the exercise scenario must be properly managed;
pre-staging of players and/or prior knowledge of scenario
material by players must be effectively prevented.
3.10 Conduct of the Exercise
Control of the exercise ensures that the scenario unfolds
according to the exercise plan. Controllers are responsible
for staffing and positioning themselves for effective
control. They must ensure there is no interference or
prompting by non-responders. Players/ responders must
perform their respective functions, initially and throughout
the exercise, in a professional manner as if the situation
were an actual emergency. Simulation of activities during
the exercise must be sufficiently realistic to provide
confidence that the activity could have been performed
during a real emergency.
3.10.1 Roles of Participants
Exercise Director. During the exercise, the exercise
director is responsible for the following:
Safe conduct
Coordination and continuity
Providing the opportunity to meet exercise objectives
Commencing, suspending and terminating the exercise
Controllers. Controllers provide overall direction and
control of the exercise. They are primarily responsible for
ensuring continuity of the scenario and maintaining safety
and security precautions. Controllers should do the
following:
Review appropriate emergency response plans, procedures,
and checklists prior to the exercise.
Review safety, security, communications, and logistical
plans included in the exercise plan.
Attend required training and briefing sessions.
Allow freedom of responder decisions and actions
(i.e., free play) to demonstrate exercise objectives and
response capabilities.
Preclude responder decisions or control actions that
would result in loss of opportunity for a participating
organization to meet its objectives.
Inject approved contingency messages or provide
instructions, as needed, to keep the exercise on track
with the scenario.
Preclude responder decisions and control actions that may
compromise safety or security of the facility, personnel,
or equipment.
Refrain from prompting, in any fashion, decisions or
actions of responders.
Keep the lead controller informed of significant
unplanned activities.
Be prepared to suspend exercise activities in the
immediate area and to use pre-arranged protocols to
terminate an exercise.
Evaluators. In general, the only function performed by an
evaluator during the exercise is to observe and document the
responder actions; however, in some circumstances, because
of limitations on available personnel or financial
resources, evaluators may perform a dual role of
evaluator/controller. Formal evaluation is performed after
the exercise is terminated. Evaluators should be assigned
specific locations or specific exercise functions.
Evaluators should do the following:
Review appropriate emergency response plans, procedures,
and checklists prior to the exercise.
Review appropriate plans (e.g., safety, security,
communications, and logistical plans) developed for
conduct of the exercise.
Attend required training and briefing sessions.
Observe performance of responders during the exercise and
document their actions using their evaluator modules or
checklists.
Refrain from interfacing with responders to prevent
interrupting or prompting.
Evaluate responder performance (not the person) and
adequacy of procedures, facilities, and equipment based
on exercise-specific evaluation criteria and evaluator
checklists.
Document errors and problem areas in the scenario or
conduct of the exercise.
Maintain a time line of the events as they enfold.
Present their evaluations and recommendations in a formal
critique.
Observers. Observers should not interfere with or become
involved in any exercise activity, nor should they
contribute information or opinions to responders in any
fashion.
Responders. Responders represent the majority of
participants in an exercise. In addition to site DOE/NNSA
and contractor emergency response personnel, responders may
include personnel from DOE/NNSA Headquarters, DOE/NNSA
Operations/Field Elements and service centers, and various
other DOE/NNSA elements; Federal agencies; state, tribal,
local, and private organizations; and the media.
Non-Participants. Non-participants are individuals outside
the scope of play who will continue to perform their normal,
routine duties as though the exercise is not in progress.
Such routine duties include activities necessary for
continued safe and secure operation of the facility.
Efforts should be made to minimize the impact of the
exercise on non-participants and to avoid interface between
responders and those individuals.
3.10.2 Conduct
This section discusses various aspects of exercise conduct
that ensure that the exercise represents a valid test of
performance of the response capabilities in achieving the
exercise objectives.
Confidentiality. Scenario information should be closely
guarded and not discussed with potential responders.
Guidelines for maintaining exercise confidentiality include
the following:
Controllers/evaluators should be careful of what they say
and to whom they speak about the exercise because they
may be overheard; this includes conversations over radio
net communications.
Controllers/evaluators should be careful when positioning
themselves to observe an activity to ensure they do not
give away information by their actions.
Controllers/evaluators should take care that no one can
see their scenario notebooks or comments. They should
never lay their scenarios, notes, or messages in a
location where responders can read them.
Simulation and Realism. Realism should be emphasized
throughout any exercise.
Exercises should be managed to be as realistic as
possible. Exercises should attempt to duplicate the
sense of stress inherent in a real emergency situation
while, at the same time, ensuring safety of personnel and
security of the facility.
Exercise responders should receive scenario information
only when it is earned via demonstration of the
particular role and its response to the event.
Simulation should be kept to a minimum. During responder
briefings, responders should be briefed on which
functions/activities are simulated.
A control cell should be used whenever it is necessary
for responders to interact with entities not
participating in the exercise. The control cell is
located away from responders and is staffed by
experienced professionals who simulate or role-play
nonparticipating organizations. This method of
simulation enables realistic interactions to occur
between the exercise responders and those they would
expect to interact with during the course of an actual
response.
Actors/role-players (controllers) should be used to
simulate personnel who would actually be encountered by
responders if the scenario were real. Actors may come in
face-to-face contact with the responders or may be
members of a control cell.
Responders should implement their appropriate plans,
procedures, and training to respond as if the scenario
information is real. Responders should rely upon the
controllers or exercise simulation tools to supply
scenario information.
Presentation of Scenario Information. Data and evidence
should be presented to the responders as it would be found,
measured, or indicated, with a maximum degree of realism.
Information should be provided to responders only when it
is earned through their observations, correct use of
procedures, and correct reading and use of
instrumentation. For authenticity, data sheets, recorder
charts, and instrument output information should be
provided wherever possible in the scenario.
Time-related parameters should be provided to responders
at the time identified on messages to ensure progress of
the scenario timeline.
If responders require clarification (i.e., a reasonable
request) about a particular message or visual cue, the
controller should provide such data/information as
accurately as possible considering simulated time and
scenario conditions, then advise the Lead
Controller/Evaluator of their inject.
If controllers need to create additional information
(e.g., the message was incomplete) or do not know the
information required, they should use pre-arranged
protocols (e.g., obtain area controller or lead
controller permission) to formulate a response.
Free Play. Free play allows responders to make decisions
and take actions they consider appropriate to the response.
Realism is enhanced and responder motivation is improved
when responders are provided the latitude to make decisions
and take actions that may differ from those anticipated
during the scenario development.
The key management aspect of free play is to allow such
actions to occur, but to preclude actions by responders
that would do the following:
Jeopardize personnel safety
Jeopardize facility/site safety
Compromise security
Interfere with the scenario
Exceed established exercise scope or limitations
Preclude exercise objectives from being demonstrated
During exercises, responders may interject innovative,
unexpected response solutions or actions that can be
accommodated by the scenario. In such situations, the
controllers should allow the responders to proceed with
their actions and notify the exercise lead controller
that a deviation is occurring. If the responder actions
compromise safety or security, or limit demonstration of
stated exercise objectives, the controller should note
the intended action but preclude that intended action
from actually occurring. This information should be
reported to the evaluator.
Actual equipment and procedural problems that are
identified during the exercise interject a form of free
play. Solutions to actual equipment or procedural
problems on a real-time basis afford a valuable
evaluation of the conduct of operation, training, and
safety culture of the responders. Controllers should
allow responders to solve such actual problems unless
safety, security, or demonstration of exercise objectives
may be compromised.
Prompting. Explicit instructions should be given to all
participants to avoid prompting during an exercise.
Prompting occurs when responders are provided advance
scenario-related information or guidance regarding
appropriate response actions. Prompting may result from
either unintentional or intentional action by controllers,
evaluators, or observers.
Communications. All written and verbal communications among
participants should be clearly identified as exercise
information and all message transmissions should begin and
end with the statement:
THIS IS AN EXERCISE.
Because offsite parties can monitor radio and cellular
telephone transmissions, personal information such as the
names or phone numbers of individuals should never be
transmitted. All communications should be in compliance
with security practices.
3.11 Exercise Evaluation
Evaluation and critique of the exercise provide feedback to
resolve deficiencies and incorporate improvements in the
emergency management program. A well-planned, structured
evaluation is essential for performing a valid test of the
emergency response capabilities of the program. In this
section, the planning and organization of an evaluation of
an operations-based exercise will be described. This will
be followed by a discussion of the evaluation process.
3.11.1 Planning and Organization of the Evaluation
The following steps should be implemented to effectively
plan for an operations-based exercise evaluation:
Define evaluation requirements. Determine what will be
evaluated and where the observations will occur.
Prepare the EVALPLAN. Prepare the complete package of
information on the evaluation process.
Develop evaluation tools. Develop the forms evaluators
will use to capture information for evaluation during the
exercise observation.
Recruit, assign, and train evaluators. Determine the
necessary qualifications of evaluators, identify
appropriate individuals, obtain commitments from those
individuals, and train them.
Finalize the evaluation plan. Undertake the activities
necessary to organize the evaluation just before the
exercise.
Define Evaluation Requirements. While the exercise is being
designed, the evaluation planning team will be provided, via
the EXPLAN, the MSEL, and other exercise documents, with
information on:
Exercise goals and objectives
Exercise flow
Critical actions
Exercise participants
Functions and activities to be evaluated
The evaluation planning team will use this information to
plan the evaluation, as follows:
Step 1: The evaluation planning team will first use the
exercise goals and objectives to determine what
performance outcomes should be evaluated.
Step 2: Once the outcomes to be evaluated are determined,
the team identifies what activities should be evaluated.
Step 3: Based on these activities, the team identifies
which functions (e.g., individuals, teams, disciplines,
and organizations) should be evaluated.
Step 4: From the functions, the evaluation planning team
can identify where the observations should take place
(i.e., what locations) and which specific tasks should be
evaluated.
Step 5: From the tasks to be evaluated, the planning team
should develop the guidelines within the objectives and
criteria for meeting an objective when criteria are only
partially met.
Step 6: Once these steps have been completed, the
evaluation planning team can identify or develop the
appropriate evaluation tools for the evaluators to use.
Prepare the EVALPLAN. The planning for an evaluation is
incorporated in an EVALPLAN, which consists of the
following:
Exercise-specific information: The EVALPLAN should
include the scenario, the map of the play site (including
evaluation locations), and the exercise schedule
(including the evaluation schedule).
Evaluator team organization, assignments, and location:
The EVALPLAN should identify how many evaluators are
needed, where they will be located, and how they are
organized. Evaluators cannot see everything that occurs
at any one location during a response. Yet, during the
exercise, evaluators must be able to capture information
that provides insight into how effective each group is as
well as how well they operate with each other. Thus,
location and number of evaluators are crucial to the data
collection process.
Evaluator instructions: Evaluators should be given
instructions on what to do before they arrive
(e.g., review exercise materials, jurisdictional plans
and procedures, and the EVALPLAN) as well as how to
proceed upon arrival.
Evaluation tools: The EVALPLAN should include the data
collection instruments and guides as discussed below.
Develop Evaluation Tools. Once the evaluation planning team
has determined what will be evaluated and where the
observations will occur, specific evaluation tools are
developed for use in the data collection and analysis. The
Exercise Evaluation Guides (EEGs) are developed to assist in
exercise evaluation. The facility/site- or activity-
specific plans and procedures are used to describe the
expected response to be evaluated for each exercise
objective. EEGs provide evaluators with information on what
they should expect to see at the specific location or in the
specific situation. The EEG should provide the evaluator
with the important parameters and actions to look for in
observing the activities. Guidance is provided for
determining whether the objective is met. Space in the EEG
should be provided to record observations; a checklist
format might be useful to link observations with the
parameters and actions required in plans/procedures.
Questions to address after the exercise can also be recorded
in the EEG.
Recruit, Assign, and Train Evaluators. Selection,
recruitment, and assignment of evaluators are crucial
components of exercise design. The individual primarily
responsible for these tasks is the Lead Evaluator. Other
members of the evaluation planning team may assist the Lead
Evaluator in this task.
The EXPLAN, which is developed by the exercise planning
team, serves as the basis for determining the number and
expertise of evaluators needed for the exercise. This
document will define the scope and concept of play for the
exercise. It describes the response tasks that will be
demonstrated by exercise players and indicates whether
simulations will be used for nonparticipating organizations.
It also identifies exercise locations such as emergency
operations centers, medical facilities, decontamination
sites, and field locations.
The Lead Evaluator plays a critical role in operations-based
exercises and should be identified early in the process to
fully participate as a member of the exercise planning team.
The Lead Evaluator should be a senior-level person who
understands command and decision-making processes and
interagency coordination, as well as specific response
functions. Exercises with play in multiple sites will need
an Evaluation Team Leader for each site.
A number of evaluators will also be needed to observe and
record player performance during the exercise. Evaluators
should be chosen for their knowledge and understanding of
the specific functional area they will be assigned to
observe. Evaluators should be assigned to monitor all
participating organizations and player locations. The
following guidelines will help participants determine the
number of evaluators that are needed:
Field response. A minimum of one evaluator for each
function evaluated (e.g., incident command,
decontamination, and emergency medical services);
additional evaluators are needed for functions that
involve multiple activities that take place
simultaneously or activities that take place in multiple
locations.
Hospitals/Medical Facilities. A minimum of three to five
evaluators at each participating medical facility,
depending on size and expected patient/victim flow;
additional evaluators are needed for functions that
involve multiple activities taking place simultaneously
or activities taking place in multiple locations.
EOC. A minimum of three to five evaluators at each
participating facility, depending on the size and
organizational structure of the EOC.
Joint Information Center (JIC). Depending on the
expected number of participants at the JIC, one or two
evaluators may be sufficient.
Additional evaluators would be needed for a large exercise
with many players performing a function in a single location
or for each location when the function is performed at
multiple sites.
Generally, exercise evaluators will be peer reviewers
identified by reaching out to other facilities on a site, to
other DOE/NNSA sites, or to DOE/NNSA Headquarters offices.
Independent evaluators who can assist in monitoring
compliance may also supplement this peer review approach.
Potential evaluators may be identified from multiple
sources, including following:
Members of the Exercise Design Team, who are fully versed
in the scenario, players, and expected action, are a good
source for evaluators (if they are not already committed
to other duties during the exercises).
Experienced members of participating organizations and
the ERO who are not involved in the play are a good
choice for evaluators because they are familiar with the
organizations, plans, and procedures.
Professionals in similar agencies in adjacent or nearby
jurisdictions can be a source for evaluators, especially
when all of the participating jurisdictions members of a
specialized function, such as a Hazardous Materials
(HAZMAT) team, are involved in the exercise.
DOE/NNSA and contractor employees from other DOE/NNSA
facilities and/or sites might be available with
sufficient notice.
Although service as an evaluator requires a considerable
commitment of time, evaluators and their agencies can expect
to gain significant benefit from the peer evaluation
process. For example, observing other locations exercising
their emergency response plans may help evaluators gain
insight into best practices or other ways to provide
emergency response, which could benefit their own
communities.
Evaluators are expected to be available for the pre-exercise
training and briefing/site visit, the exercise itself, the
post-exercise hot wash, and the data analysis and
contribution to the AAR. This time commitment is usually
equivalent to one day before the exercise, the exercise
day(s), and one day after the exercise. One or more of the
evaluators may devote additional time to drafting the AAR
and briefing participant organizations and their management
on findings and recommendations.
3.11.2 Evaluation Process
Information is gathered and documented by the evaluators.
Evaluators assess the performance of the ERO and adequacy of
equipment, facilities, and resource documents used by the
responders. The assessment consists of a comparison of
performance against predetermined and documented
facility/site- or activity-specific evaluation criteria
based on program-specific plans/procedures. Information
from the evaluation and critique processes provides feedback
for use in identifying corrective actions and improvements
to the emergency management program.
The evaluator organization must be sufficiently staffed to
evaluate the performance and key decision-making of the
responders in satisfying the exercise objectives.
Evaluators should be familiar with responder organizations,
functions, procedures, and anticipated responder decisions
and response activities in order to accurately monitor
activities and functions performed by the players.
Responders/players are evaluated with respect to their
demonstrated proficiency in their respective
responsibilities and functions, communication and
coordination with other responders, familiarity and use of
applicable procedures and equipment, and overall
professional response. Facilities and equipment are
evaluated with respect to adequacy of functions/operability.
Procedures are evaluated with respect to their use by
responders, specifically, their adequacy of content for the
tasks performed. Notifications and communications are
evaluated during every exercise. When offsite agencies
participate, interfaces with offsite agencies are evaluated.
The following overview describes the steps in the exercise
evaluation process for operations-based exercises, not
including auxiliary activities such as development of the
evaluation tools or training for evaluators.
Step 1: Plan and organize the evaluation. As part of the
exercise planning and development process, the exercise
planning team will determine what information should be
collected, who will collect it, and how it will be
collected.
Step 2: Observe the exercise and collect data. Expert
(peer) evaluators collect data by recording their
observations during exercise play and collecting additional
data from records and logs. Evaluators of tabletop
exercises record discussion and review documents such as
plans, procedures, and interagency agreements.
Step 3: Analyze data. The analysis phase should answer the
following questions about the exercise play:
What happened?
What was supposed to happen?
If there is a difference, why is there a difference?
What is the effect of that difference?
What should be learned from this?
What improvement should be made or exemplary practices
adopted?
The first step in the analysis process is a player hot wash,
i.e., a short discussion session immediately following the
exercise to get player feedback.
Analysis of exercises is conducted using data collected to
reconstruct the timeline of events as they occurred, an
approach similar to reconstruction of events that most
agencies do following an accident or other type of incident.
This information is then used to identify and explore the
differences between what happened and what was supposed to
happen to ascertain the root causes for the differences.
Step 4: Develop the DRAFT AAR. As part of the analysis
phase, the evaluation team drafts the AAR, which provides a
description of what happened, exemplary practices, issues
that need to be addressed, and recommendations for
improvements.
The evaluators share the assessment information with
management and, if appropriate, facilitate identification of
improvements that can be made. This phase of the Exercise
Evaluation and Improvement Process generally consists of the
following steps.
Step 5: Conduct Exercise Debrief meeting. The exercise
planners and/or evaluation team will present their analysis
findings and recommendations in an Exercise Debrief meeting
with management from the sites, facilities, departments,
agencies and jurisdictions that participated in the
exercise. They will also solicit feedback and validation
from the attendees on their observations and
recommendations.
Step 6: Identify improvements to be implemented. Much of
the Exercise Debrief meeting will be devoted to discussing
specific actions that the exercise participants will take to
address the opportunities for improvement contained in the
recommendations in the draft AAR. This list of actions,
referred to as the Improvement Plan (IP), identifies what
will be done, who (person, department or agency) is
responsible, and the timeframe for implementation. Although
the IP is a written document, it should be viewed not as a
static document but as a dynamic program that is updated and
modified regularly in a constant cycle of improvement.
Step 7: Finalize the AAR. Following the Exercise Debrief
meeting, the evaluation team should finalize the AAR by
incorporating any corrections or clarifications related to
the observations or recommendations as well as the
improvement steps that will be taken. Some of the actions
may include only the preliminary step of a multi-step
activity (e.g., create a committee to review the issue and
make recommendations for further action).
3.11.3 Critiques
Formal critiques are conducted after the exercise to provide
a forum in which the exercise results can be addressed and
discussed among the participants. This can result in the
identification of lessons learned for improving the
response to an emergency. For large-scope exercises, it may
be necessary to conduct several critiques to ensure that all
participants are given the opportunity to take part.
Responder hotwash critiques are conducted immediately
following the exercise to provide an opportunity for
players/responders to discuss their own perspectives on the
activities and events. These critiques are typically
conducted in place (e.g., incident command post, field
teams, EOC) by the area lead responder or controller.
A formal verbal critique is conducted following each
exercise and should include participation by all controllers
and evaluators. This critique should provide the forum for
discussion and correlation of individual observations,
formulation of exercise findings, determination of
objectives demonstrated, and determination of overall
exercise performance. Recommendations for corrective and
improvement actions should be addressed. The product of
this critique provides the framework for the senior
management critique (plus any exit meeting) and the exercise
AAR.
Key participants, including manager-level responders, the
Exercise Director, the lead controller(s), and the lead
evaluator(s), should attend a senior management-level
critique. Overall exercise performance, significant
observations, findings, and preliminary corrective actions
and improvement items may be addressed. For exercises
evaluated by an external organization, an exit/closeout
meeting may be conducted for DOE/NNSA and facility/site
management.
Critiques should accomplish the following:
Be conducted in a questioning, objective manner to
maximize the benefit and learning experience from each
exercise.
Include a review of scenario events, identification of
shortcomings in the scenario or exercise conduct, and
analysis of expected and actual responder actions.
Discuss responder performance, the adequacy of procedures
and other documentation, and the adequacy of facilities
and equipment.
Provide the basis for documentation of findings to
facilitate identification of corrective actions and
improvement items for upgrading the emergency management
program.
3.12 Follow-up Activities
3.12.1 Corrective Actions and Improvement Items
Findings resulting from the exercise should be subject to an
in-depth review. For recurring problems, a root cause
analysis should be performed. A plan should be developed to
implement corrective actions and improvement items.
Management should budget, schedule, and implement the
actions to upgrade the emergency management program.
Activities should be coordinated with affected
organizations. Corrective actions, such as procedural
modifications, necessitate timely feedback to the
participants. Such timely feedback demonstrates management
attention and concern for upgrading the emergency response
capability and demonstrates management support for
involvement of participants in exercises.
3.12.2 Maintaining Records
Auditable records should be prepared and maintained for each
exercise. Long-range planning information such as exercise
objectives, schedules, and the exercise AAR are considered
auditable records. Records that may be maintained include
the following:
Training records
Participant rosters
Exercise participant packages
Critique minutes or summaries
Completed evaluator modules or checklists
Final report
Accounting summary
APPENDIX B. Controller and Evaluator Manual
B.1 Introduction
The purpose of the Controller and Evaluator Manual is to
prepare DOE/NNSA controllers and evaluators to effectively
perform assigned duties and functions during emergency
management exercises. It provides pertinent information
concerning the exercise development, control, and evaluation
processes, and details specific controller and evaluator
responsibilities before, during, and after an exercise.
This Manual:
Augments DOE G 151.1-3, Chapter 3
Details the roles of controllers and evaluators in
exercises
Explains the materials used by controllers and evaluators
Provides techniques on how to effectively perform the
controller and evaluator functions
Section B.2 provides an overview of the exercise
organization and participant selection and responsibilities.
Section B.3 details controller activities, while Section B.4
is dedicated to evaluator activities. Controllers and
evaluators may only need to review the section that applies
to their assigned functions during the exercise. However,
individuals assigned to both controller and evaluator roles
should read the section in its entirety. Section B.5
provides information relative to the exercise critique and
evaluation process that is applicable to both controllers
and evaluators. The After Action Report (AAR) is addressed
in Section B.6.
B.2 Exercise Organization and Participants
The exercise organization consists of the following
participants: the Exercise Director, responders,
controllers, evaluators, and observers. Each participant
performs specific assignments and roles as summarized below:
Exercise Director. The senior exercise official who has
primary authority and overall responsibility for the
design, development, control, and evaluation of the
exercise.
Controllers. Provide direction and control of the
exercise. They monitor the sequence of events as they
unfold, and are responsible for exercise safety within
their span of control. Individual controllers may
initiate certain actions in order to ensure the
continuity of events described in the exercise scenario.
It is their responsibility to ensure that responders do
not respond in a manner that might jeopardize safety and
that responders remain focused on exercise play that
demonstrates the exercise objectives. The control
organization will vary in number depending on the
exercise scope and may include the following controller
positions.
Senior controller. Responsible for coordination and
oversight of all other controllers.
Lead controllers. May be used to coordinate the
activities of several controllers for larger or more
complicated exercises that involve a number of response
locations and emergency functions. Controller teams may
be organized by location, function, or a combination of
both depending on the needs of the exercise. However,
controller team leaders should have previous experience
as an exercise controller before they are selected to
lead a team.
Control cell. A simulation center located away from the
responders. Staffed by experienced controllers (and/or
actors) who simulate or role-play non-participating
organizations by providing input to responders, via
telephone, on behalf of any non-participating
individuals, companies, offsite agencies, or Emergency
Response Organization (ERO) members who would normally be
involved in responding to an emergency. Role-players in
a control cell are subject to evaluation of their
performance just like any other exercise controller.
Timeline coordinator. For complex exercises, is
responsible for ensuring the exercise timeline remains on
schedulea key factor for proper attainment of exercise
objectives. Should exercise play cause deviation from or
a delay in the timeline, it becomes necessary to use
previously prepared contingency materials. The timeline
coordinator, typically co-located with the Exercise
Director, is responsible for specific tasks or actions
from the control cell. The timeline coordinator receives
timeline status reports from lead controllers and
provides this information to the senior controller and
Exercise Director.
Actors/role-players. Controllers who simulate members of
non-participating organizations and role-play key
individuals, such as injured personnel. They may have
face-to-face contact with responders, functioning as
media reporters, next-of-kin, or injured personnel. They
may be members of a control cell with telephone
communication being the only interaction with responders.
Evaluators. Document and evaluate responder performance
and adequacy of facilities and equipment against
established emergency plan/procedures and exercise
evaluation criteria. Evaluators are unbiased, objective,
technical or functional experts. The evaluator
organization will vary in number depending on the
exercise scope and may include the following evaluator
positions.
Senior evaluator. Responsible for the coordination of
all evaluation functions including preparation of the AAR
that identifies findings and recommends corrective
actions.
Lead evaluators. May be used to coordinate activities of
several evaluators for larger or more complicated
exercises that involve a number of response locations and
emergency functions. Evaluator teams may be organized by
location, function, or a combination of both, depending
on the needs of the exercise. Evaluation team leaders
are responsible for the coordination of a team of
evaluators assigned to particular locations and/or
similar response functions. As with the control
organizations, evaluation team leaders should be selected
on the basis of previous experience and demonstrated
ability to successfully perform as an evaluator.
Responders. Often referred to as players, usually
comprise the majority of participants in the exercise.
It is their responsibility to take the necessary actions
to mitigate the simulated emergency and thus demonstrate
the ability to ensure the safety of workers, the public,
and the environment, in accordance with established
emergency plans and procedures.
Observers. May be present to observe the exercise for
either official or educational purposes. Attendance of
observers at an exercise, their locations, and rules of
conduct should be determined by the Exercise Director.
Observers should not interact with responders, contribute
information or opinions, or interfere with the exercise
in any other way. Observers should direct all questions
or comments related to the exercise to the area
controller or their escort, if appropriate. Although
they may have prior knowledge of the scenario, observers
should be reminded of the responsibility for withholding
that information from the responders.
Video teams and still photographers. Considered
observers for exercise purposes, may be used to document
the exercise. These teams may film, record, and
photograph response activities, as long as they do not
interfere with exercise play.
B.2.1 Selection of Controllers and Evaluators
Controller and evaluator functions generally should not be
combined. Each role has specific responsibilities that
require total concentration to be performed effectively. If
circumstances require that an individual be assigned to both
roles, that individual must have a thorough understanding of
controller/evaluator requirements and responsibilities.
Individuals who fill controller positions should have
extensive emergency management experience and have
participated in a variety of tabletops, drills, and
exercises, so they know what behaviors and actions to expect
from responders. Controllers should participate in several
exercises to provide continuity and consistency. Personnel
who serve as controllers should be knowledgeable concerning
the updates and upgrades to the emergency plans and
implementation procedures that will be demonstrated at their
locations during the exercise.
Evaluators are selected based on their knowledge of the
functions they are to evaluate and their experience with
them. It is important that they are technically competent
to judge the actions of the responders. Wherever possible,
their experience should be equal to or greater than that of
the responders in their assigned area.
The conduct of effective emergency exercises depends on the
selection and assignment of top-quality controllers and
evaluators. Although these individuals may be drawn from
non-participating areas of a response organization, care
should be taken to ensure that use of these personnel to
support the exercise does not compromise the effectiveness
of the response organization.
B.2.2 Responsibilities of Controllers and Evaluators
Controllers.
Controllers are primarily responsible for ensuring the
continuity of the scenario and maintaining safety and
security.
Controllers play a crucial role throughout the exercise
process. Their first and most important function is to
maintain exercise safety. They maintain the sequence of
events and control the flow of message injects. Controllers
are responsible for the overall conduct of the exercise.
They are in a unique position to view exercise play,
understand the dynamics of an action or activity as it
unfolds, and comment on what they observe. Controllers do
interact with the responders. Controllers provide scenario
information to responders as it is earned and may be tasked
to inject approved contingency messages to keep the exercise
on track with the scenario and exercise timeline.
Controllers should do the following:
Prior to the exercise, review appropriate Emergency
Plans, procedures, and documents.
Prior to the exercise, review appropriate Exercise Plan
(EXPLAN) materials, including objectives, scenario,
messages, Safety and Security Plans, and controller
instructions.
Attend required training and briefing sessions.
Conduct the exercise by providing applicable scenario
information to responders.
Allow freedom of responder decisions and actions
(i.e., free play) to demonstrate exercise objectives and
response capabilities.
Inject approved contingency messages or provide
instructions, as needed, to keep the exercise on track
with the scenario.
Preclude responder decisions and control actions that may
compromise the safety or security of personnel or the
facility.
Refrain from prompting, in any fashion, the decisions or
actions of responders.
Prevent observers and evaluators from interacting with
responders.
Be prepared to suspend exercise activities in the
immediate area and to use pre-arranged protocols to
terminate an exercise.
Evaluators.
An evaluator's function during the exercise is to observe
and document exercise activities and conditions. The
evaluation assessment is performed after the exercise is
terminated.
Evaluators document and evaluate the performance of the
responders and the adequacy of facilities, equipment, and
resource documents (e.g., drawings, reference materials,
maps) used by the responders. Evaluators are assigned
specific locations or responder functions to evaluate.
Responder performance must be evaluated against plans and
procedures using criteria established prior to the exercise.
Evaluators document the responders' performance and attend
the critique facilitated by the controllers immediately
following the exercise during which the responders discuss
their performance. The evaluation, documentation, and
critique discussion(s) provide important data, which
substantiate the exercise findings.
The AAR summarizes the overall results of the exercise and
provides a comprehensive assessment of the emergency
response performance. Evaluators should do the following:
Review appropriate emergency response plans, procedures,
and documents prior to the exercise.
Prior to the exercise, review appropriate EXPLAN
materials including the objectives, scenario, messages,
Safety, Media, and Security Plans, and evaluator
instructions.
Attend required training and briefing sessions.
Observe the performance of the responders during the
exercise and document their actions using evaluation
modules or checklists.
Observe the performance of the control organization in
controlling and directing the exercise.
Refrain from interfacing with responders to preclude
interrupting or prompting their decisions or actions.
Evaluate responder performance and the adequacy of
procedures, facilities, and equipment based on specific
evaluation criteria.
Document errors and problem areas in the scenario or
conduct of the exercise.
Present their evaluations and recommendations in a formal
critique.
Exercise information should be closely guarded and not
discussed with potential responders. Scenario materials
should be secured at all times, when not in use. All copies
of the EXPLAN should be numbered and assigned while under
review to ensure accountability during the
review/development period. To ensure that exercise
confidentiality is maintained, controllers and evaluators
should do the following:
Be careful of what they say and to whom because it may be
overheard.
Take care in positioning themselves while observing an
exercise activity to ensure they do not give away
specific information by their actions.
Ensure responders cannot read their scenario, timeline,
notes, inject messages or other sensitive materials
before or during an exercise.
Exercises are subject to evaluation by external DOE/NNSA
organizations (i.e., an organizational entity beyond that of
the immediate facility/site conducting the exercise),
including any organization [e.g., Defense Nuclear Facility
Safety Board (DNFSB)] with oversight responsibility. An
external exercise evaluation may also include an evaluation
of the manner in which the exercise is conducted,
controlled, and evaluated. This evaluation would likely
include an evaluation of the performance of the controller
and evaluator organizations.
B.2.3 Controller and Evaluator Training and Briefings
A formal training program for controllers and evaluators
enhances the capability of the emergency management program
to maintain a level of consistency in how exercises are
managed and response capabilities are evaluated. The
training program should include initial and refresher
training and a pre-exercise specific briefing to provide
controllers and evaluators with the information and
direction necessary to perform their duties with confidence.
Both generic training and exercise-specific briefings are
discussed in Chapter 3, DOE G 151.1-3.
B.3 Controller Activities
Controller responsibilities include pre-exercise setup,
exercise conduct, and post-exercise activities. Selected
controller responsibilities during the exercise process are
given below:
Pre-Exercise Activities Initial Training
Refresher Training
Security Access
Simulation Setup
Communications Check
Activities During the Exercise Exercise specific
Briefing
Responder Briefings
Earned Information
Contingency Messages
Unplanned Suspensions Restart
Exercise Termination
Post-Exercise Activities Participant Rosters
Self critique Sheets
Responders Critiques
Evaluators Critiques
AAR
The following section reviews methods of controlling an
exercise, as well as specific responsibilities of
controllers before, during, and after an exercise.
B.3.1 Pre-Exercise Activities
Controllers must review the exercise and scenario materials,
attend exercise-specific training, perform communication
checks, set up simulations, and conduct pre-exercise safety
and security checks before an exercise.
Controller Package.
A cover page should remind the controllers of the
confidentiality of the scenario materials.
The Controller and Evaluator (C/E) Handbook supplements the
EXPLAN by presenting more detailed information about the
exercise scenario and describing exercise controllers and
evaluators roles and responsibilities. Larger, more
complex exercises may use the Control Staff Instructions
(COSIN) and an Evaluation Plan (EVALPLAN) in place of, or to
supplement, the C/E handbook.
Controllers should be issued their materials for review
prior to the exercise-specific briefing. The controllers,
being experienced personnel, may be tasked to help the
scenario developers finalize scenario details. The
controller package may consist of part or all of the
contents of the EXPLAN. Because some EXPLANs are very large
documents, controllers may need to reorganize the material
so that the information critical to their specific
assignment is readily accessible. Controllers should bring
their packages to the exercise-specific briefing sessions
and be prepared to discuss any concerns or questions they
have about this information. The controller package should
include, as a minimum, the following information:
Schedule of control activities
Control organization and assignments
Procedures for reporting within the control organization
Suspension and termination procedures
Scenario material
Objectives
Scenario narrative
Timeline and Master Scenario Event List (MSEL)
Position-specific messages or injects, including
associated data
Position-specific safety and security instructions
Exercise-Specific Briefing. The overall objective of the
pre-exercise briefing is to prepare the controllers to
safely and effectively control the exercise without
compromising the scenario or prompting responder actions.
Controllers who do not attend the briefing should not serve
as controllers during the exercise.
Day-of-the-Exercise Preparations. On the day of the
exercise, controllers report to the staging area with their
notes, scenario messages, data sheets, controller log forms
for recording activities, and any other materials assigned.
Controllers should have reviewed their instructions (example
in Figure B-1), and highlighted specific responsibilities
and messages that they are responsible for delivering.
Simulation Setup. The controllers may be required to assist
in setting up the simulations that will be used in their
control areas. This may include the preparation of smoke
generators, positioning of special equipment or vehicles,
simulated spills, and injured role players. It may also
include the pre-staging of simulations for use later in the
exercise or for contingency purposes. All simulations
should be checked before reporting ready. It should be
remembered that realism is second only to safety.
GENERIC CONTROLLER INSTRUCTIONS (SAMPLE)
1. Review the exercise objectives and controller package for
your area of responsibility.
2. Using the Master Scenario Events List (MSEL), highlight the
specific messages for which you are responsible.
3. Be located in the appropriate emergency response facility at
least 30 minutes prior to the start of the exercise. If you
are not assigned to a specific facility, be in place to meet
the responders at least 15 minutes prior to their activation.
4. Obtain or locate necessary communications equipment and test
it to ensure satisfactory communication between controllers
and the senior controller and/or the timeline coordinator.
5. Wear controller identification, such as the required badge,
armband, or vest.
6. Synchronize your watch with the lead controller to ensure
that the exercise timeline and the controller logs are
consistent. Verify weather conditions if actual meteorology
is to be used during the exercise.
7. As instructed, distribute an exercise participant package to
specific responders. This may include exercise limitations,
meteorology, instructions, and the exercise telephone
directory.
8. Do not enter into personal conversations with any exercise
responder.
9. Deliver the messages you have been assigned at the time
indicated. Caution: If the information depends on some
action to be taken by the responder, do not deliver the
message until the responder has earned the information by
successfully accomplishing the required action.
10.When you deliver a message, provide the senior controller
with the message number and delivery time.
11.Begin and end all exercise communication over the radio or
telephone with the phrase THIS IS AN EXERCISE. This
precaution is taken so that anyone overhearing the
conversation will not inadvertently mistake the exercise play
for an actual emergency event.
12.If you are to deliver specific data, deliver it as directed
on the message instructions. . (Examples: Do not deliver
vital signs of an accident victim until the first responder
attempts the appropriate actions for obtaining these; do not
volunteer radiation readings until the technician has turned
on and read the detection instrument.)
13.Record all activities and the time in your controller log.
Do not write opinions; rather, write about specific actions.
Page 1 of 2
Figure B-1. Sample Generic Controller Instructions
GENERIC CONTROLLER INSTRUCTIONS (SAMPLE) (contd)
14. If responders do not perform as expected and a contingency
message is not provided, notify your lead controller
immediately and ask for direction. No unplanned simulations
should be allowed without the seniorcontroller's approval.
This differs from free play, which is action taken by a
responder that is appropriate in solving the problem in a
unique way.
15. Do not prompt a responder as to what a specific response
should be unless a contingency message directs you to do so.
Clarify information as long as it does not provide coaching.
16. Ensure that all observers stay out of the exercise activity.
If you need assistance, notify your lead controller or
security.
17. Do not provide information to the responders regarding
scenario event progress or resolution of problems
encountered by others. Responders are expected to obtain
information through their own resources.
18 The senior controller will notify controllers when the
exercise has been terminated. The exercise will be
terminated when the Exercise Director, in conjunction with
the senior controller, determines that all exercise
objectives have been met, or enough time has elapsed for the
objectives to be demonstrated.
19. Pick up copies of responder logs and pertinent documentation
prior to the post-exercise debriefing and critique. This
information should be given to the senior controller.
Coordinate this task with the evaluator in your area.
20. At exercise termination, summarize your notes and prepare
for the local area critique. Have the summary ready to turn
over to your lead controller. The facility lead controller
shall provide this documentation to the senior controller.
Page 2 of 2
Figure B-1. Sample Generic Controller Instructions (contd)
Pre-Start Safety Checks. Prior to the beginning of the
exercise, pre-start safety checks are conducted. These
include the checking of simulations, posting EXERCISE IN
PROGRESS signs, weapons safety checks, and a final
communication check with the lead controller.
The Exercise Director will not start the exercise until
notified that all controllers have performed their
communications check, verifying they are at their assigned
location and their safety checks have been completed. The
senior controller will give the control organization a time
check (synchronizing watches). Large exercises may be
started at a predetermined time while smaller exercises may
be started by voice command over the communications network.
B.3.2 Activities during the Exercise
The exercise control organization plays the crucial role in
monitoring the sequence of events, injecting messages, and
ensuring the overall safe conduct of the exercise.
Monitoring For Safety.
A controller's primary function is to ensure the safe
execution of an exercise. The safety of everyone
involved in the exercise, as well as the facility,
public, and the environment, is the highest priority.
Controllers are responsible for knowing the limitations and
precautions for exercise safety and security and for
understanding and using this information to ensure that all
participants comply accordingly. Precautions and
limitations are provided to the controllers in the exercise
Safety Plan and Security Plan and also in exercise-specific
training. Such information may include details on physical
security (such as facility access control), safety (such as
the location of fire doors), information security (such as
the location of classifiers to ensure classified information
is not divulged), and other privileged instructions.
All participants in an exercise are responsible for acting
in accordance with the exercise safety plan and are bound by
DOE/NNSA requirements, as well as local laws and
restrictions. It is the responsibility of the controller to
stop any action that would violate any law or safety
protocol. General safety rules applicable to all
participants may include the following:
Comply with all Federal, state, and local legal
restrictions.
Obey all traffic laws when responding to the exercise.
Wear all personal protective equipment required for the
job.
Do not place yourself or anyone else in an unsafe
position.
Obey the directions of the controller at the scene.
Preface and end all radio, telephone, FAX, and other
communications with THIS IS AN EXERCISE.
Ensure any weapons being used are empty and on SAFE.
Contact a controller in the event of an actual emergency.
Methods of Control. Exercise control, in terms of the
autonomy and authority of individual controllers, varies
depending on the complexity of the simulated events, the
number of participating organizations, and the number and
experience level of the controllers. Individual controllers
may be delegated extensive, limited, or even no authority to
issue message injects and contingency message injects
without the Exercise Director or senior controller
direction. The method of control for each exercise should
be identified and documented, and the controllers should be
trained on their individual levels of authority for message
inject release and subsequent reporting.
Presenting Scenario Information. Controllers either
initiate simulations or provide the description of the
initial conditions to the responders. Controllers should
introduce themselves and the evaluators in the area and
identify the exercise safety rules. Simulations should be
identified to the responders. For example, responders might
be told that the real meteorological conditions will NOT be
used and provided with the pre-determined (canned) wind
speed and direction. Generally, it is the controllers'
responsibility to set the stage for the event in their
assigned area. The controllers depend on directions from
the lead controller.
Scenario information (including physical evidence and visual
cues) should be presented in a realistic manner to the
responders as it would be found, measured, or otherwise
indicated. After a valid field measurement is taken, the
controller should inject the scenario value. For
consistency and documentation, controllers should use data
and instrument readings provided in their controller
packages when giving technical information to responders.
Visual cues such as victim moulage, liquids, solids, smoke,
and other stage props should be used to make the event
appear to the responders as if it were actually occurring.
If emergency procedures require the use of protective
equipment and clothing, the participants should use the
actual protective equipment and clothing during the event.
Controllers should not intercede in exercise play unless it
is warranted by safety considerations. Controllers should
not prompt by providing information early, providing more
information than the responder has earned, or phrasing
sentences in a way that would cause the responders to
perform an action.
Realism versus Simulation.
Making the exercise as real as safety will permit is one
of a controller's prime considerations. The more
realistic the exercise is, the less information needs to
be provided to responders.
The purpose of an exercise is to demonstrate and evaluate
response capabilities under simulated emergency conditions.
Although it is impossible to predict or measure precisely
what response would be under actual emergency conditions, a
realistic exercise can provide a valuable picture and
assessment of the response capabilities of each
participating organization. The realistic presentation of
information during exercises can create the sense of
pressure and stress inherent in actual emergency situations.
Controllers should provide information to responders in a
form and manner consistent with what would occur in an
actual emergency, and present scenario information earned by
responders as a result of their actions. Responders should
carry out every activity and response action exactly as they
would in an actual emergency, such as using emergency
equipment and checking instrument and meter readings.
However, responders should only walk-down or discuss the
response actions that would be necessary to restore or
realign equipment using panel switches, to avoid changing
critical process or plant equipment alignments and
parameters.
Free Play.
It is the controllers' responsibility to monitor free
play, note activities, and intercede when free play
exceeds the limits established for the exercise or when
safety is jeopardized.
Free play is a welcome part of an exercise because it allows
responders to provide unique solutions to problems and to
respond in ways not foreseen by the exercise planners. In
order for free play to be successful, it must fall within
prescribed parameters. If the responder actions compromise
safety or exceed established limitations, the controller
should note the intended action but prevent that action from
actually occurring.
Tracking the Scenario and Responder Actions. One of the
controllers' most important and difficult tasks is
maintaining the timeline for scenario events and tracking
responder actions. Controllers should record the time of
all significant events associated with their part of the
exercise, to include the following.
Time of message delivery
Contingency message delivery
Responder key decisions and mitigation actions
Free play
Conversations with responders (or other controllers)
Controllers should note the affects of the messages on play
and any unexpected activities, and report any
discrepancy/deviation in scenario progress immediately to
the lead controller. Later, controllers from different
areas can meet to develop a consolidated timeline of
exercise play and discuss how the exercise progressed
overall.
Maintaining Exercise Pace and Focus. The senior controller,
with the assistance of the timeline coordinator, should
manage the exercise and ensure that the sequence of events
identified in the exercise timeline occurs as close to
schedule as possible. Individual controllers should follow
the overall exercise timeline in order to keep their
respective parts progressing in accordance with the
scenario.
Use of the Master Scenario Event Line (MSEL).
The MSEL is one of the primary tools that controllers use
to track the progress of the scenario. Evaluators will
find that the MSEL is a useful tool for determining
whether and when expected responses occur.
The MSEL lists all exercise messages and key events in a
table that specifies the time the message delivery is
expected, who delivers it to whom, a message number, and a
short description of the message. Some MSELs also contain
the responder-expected actions and associated exercise
objectives to assist the controllers and evaluators in
performing their functions.
Use of Scenario Messages. Controllers use prepared scenario
messages (also known as controller injects, cue cards, and
data input) included in their EXPLAN in conjunction with the
MSEL. These messages include information on the placement
of props, initial conditions, set-up of the area, and
placement of observers. Messages that contain information
on activities that are dependent on the completion of other
activities should also include information on what to do if
that initiating activity is not completed. If this
information is not readily available, the controller should
contact the senior controller. Exercise messages are
developed to do the following:
Create situations.
Provide specific instructions and data.
Correspond with activities required by emergency plans,
procedures, checklists, etc.
Notify participants of safety or compliance violations.
Keep the scenario on track.
During the exercise, controllers inject messages through one
of three methods: voice, hard copy, or face-to-face
contact.
Voice. Controllers inject oral messages to control
progress of the exercise scenario. These messages
include descriptive information that simulates an event
or condition, or they may initiate a specific activity
that will keep the scenario on track. These messages are
given by the controller on location, or over the phone or
radio by a controller at a different location such as a
control cell. At specified times, controllers will
contact the appropriate responder(s) and read the
prepared condition or event text verbatim.
Hard copy. Hard copy messages are designed to simulate
electronic messages, memoranda, Material Safety Data
Sheets, strip charts, news bulletins, etc. At designated
times, controllers will deliver these messages to the
appropriate responder. Messages of this type are
provided on data cards or sheets of paper with
appropriate time-related facility/site and hazards
parameters, real-time data generated by a simulator or
computer, or audiovisual presentations of data such as
moulage, charts, or pictures. Parameters can be posted
on appropriate control room panels, on cameras for
viewing in several locations, or posted at, or generated
by, computer terminals and printers. When hard copy
messages are provided, the controller makes no additional
comments.
Face-to-face contact. Occasionally, selected controllers
may play the role of a senior official, a decision maker,
or an outside agency representative. While role-playing,
the controller interacts face-to-face with participants
and provides information or responses to questions in a
fashion appropriate for the role he/she is playing.
Messages help direct the progress of the exercise and
clarify situations that cannot easily be described.
Messages should normally include only information the
participants could gain with their own senses (i.e., sight,
hearing, smell, touch, and taste) in their location.
Emergency situations like fires, tornados, injury of
personnel, alarms sounding on control boards, suspected
intruders, and radiation monitor alarms can be described in
exercise messages.
Messages may appear in several different formats based on
how they will be used during the exercise. Message formats
may include data sheets, charts, pictures, and hard copies
of voice messages. Information contained in the message
should include recipient, time of delivery, expected
responder action, and any additional information or
directions for the controller. The controller should
generally not give the entire hard copy of the message to
the responder. Only the information portion of the message
should be delivered to the responder. Occasionally, the
controller may give a message directly to a responder, such
as a note from simulated hostage takers.
Messages should not be used to describe situations that
participants can and should recognize from facility data. A
message such as temperature is increasing is usually
inappropriate because it prompts responders with information
that would not be provided in that form in a real emergency.
The controller should understand and be aware of the action
that a responder should take in response to a message.
Thus, a message with an actions expected section may be
provided to controllers (not passed to participants) so that
they will be aware of what the responders should do.
Generally, controllers should provide the information to the
responders as they progress through the scenario and earn
information. To earn information, responders must act in a
manner that would provide them the information in a real
event. For example, if a message states that the oil bull's-
eye on a pump is empty, the controller should provide that
information when the participant looks at the bull's-eye.
If participants follow appropriate steps for obtaining
information, the controller may then provide it.
Controllers should only provide earned information and
nothing else that would instruct responders how to proceed.
For simulated events such as fires, controllers should
provide the scene setting information as responders earn
it. For example, in approaching a closed fire door, the
controller should tell a team member the door is hot only
after the team member has actually touched the door.
Likewise, the team member would be told the room is filled
with smoke only after the door has been opened.
Information is provided in a sequence replicating the
approach to an actual fire.
Contingency Messages. Controllers use contingency messages
to force an action by a responder or response organization
to keep the exercise on track. Controllers should issue
contingency messages in accordance with the pre-established
exercise protocol for their use. If responder actions
require a contingency message, the controller should contact
the responsible lead controller before injecting the
message.
Contingency messages provide supplemental symptoms or
necessary information specifying existing conditions that
will elicit the appropriate decision or response.
Contingency messages should begin with an explicit directive
such as the following.
Declare a General Emergency for the following
reasons....
Contact the state at this time to recommend the
following action....
To keep the exercise on track, order a site assembly at
this time.
In some cases, a contingency message may be issued solely to
keep the exercise scenario coordinated. An example
contingency message would be one preventing operations
personnel from beginning a radiological release until a pre-
established time so that time-dependent, hard copy
radiological data remain credible for the facility/site
status and conditions being simulated.
A negative exercise finding may result if exercise play does
not occur as expected and responders must be provided a
contingency message to induce activities that should have
occurred without controller intervention.
Suspension or Termination of an Exercise.
The controller's role in suspending, restarting, or
terminating an exercise is to ensure that responders
clearly understand when these actions have been
implemented. Controllers also provide specific
instructions to responders covering any requirements or
activities they must undertake. In the case of restart,
responders may have to redo activities because they are
critical for responder performance at other locations.
Suspension or termination of an exercise is managed through
the control organization. Responders are instructed to
contact a controller when an unsafe condition exists or when
an actual emergency is identified. The EXPLAN identifies
how the control organization will be notified and the
procedures for exercise suspension, restart, and/or
termination. Figure B-2 is an example of suspension and
termination instructions.
Suspension. Provisions for suspension or premature
termination of the exercise for safety reasons are
provided in the controller package. If an unidentified
or questionable situation arises that may affect the
participants, a controller may suspend play and
immediately notify the senior controller. Play may be
restarted if the situation is resolved.
In the event of a real emergency, it is the controller's
responsibility to suspend the exercise in the immediate
area for which he/she is responsible and to contact the
senior controller. An actual emergency always takes
precedence over an exercise. If necessary, the Exercise
Director may terminate the exercise so that resources can
be devoted to the real emergency.
Termination. Information concerning the procedures and
protocol for terminating an exercise are included in the
controller package and reviewed at the pre-exercise
briefing. Termination of the exercise at any time is
under the authority of the Exercise Director. Upon
notification from the Exercise Director or a lead
controller, the controllers should immediately announce
the termination of the exercise, record the time, and
ensure that responder exercise activity ceases.
In general, an exercise will be terminated when one of the
following conditions is met.
Exercise objectives have been met and the pace of play
indicates that major events have been drawn to a logical
conclusion.
Enough time has elapsed to allow the objectives to be
demonstrated.
An actual emergency occurs.
SAMPLE SUSPENSION AND TERMINATION INSTRUCTIONS
The exercise is scheduled to begin at 8:00 a.m. Mountain Standard
Time (MST) or 10:00 a.m. Eastern Standard Time (EST). No
responders should be pre-positioned, and response should be in
accordance with established policies and procedures. The
exercise is scheduled to run 6 hours with termination at 2:00
p.m. MST/4:00 p.m. EST. Each emergency response facility
participating in the exercise should conduct a critique of their
involvement immediately following the exercise. All controllers
and evaluators are expected to take notes of items identified by
the exercise responders. If controllers or evaluators are asked
for their impressions of how things went, specific issues or
problems should not be discussed.
The exercise may be terminated by the Exercise Director when
exercise objectives have either been demonstrated or given an
adequate opportunity to be demonstrated. Following consultation
with the senior controller and lead exercise evaluator, the
Exercise Director will make the announcement concerning exercise
termination.
If an actual emergency occurs, the exercise may be suspended or
terminated at the discretion of the Exercise Director. DOE/NNSA,
Federal, State, Tribal, local, or DOE/NNSA contractor
authorities, depending on the nature of the incident, may
recommend termination to the Exercise Director.
If the exercise is suspended, the controllers will instruct the
responders to safely stop in place. The conditions for restart
of the exercise will be determined by the exercise director in
consultation with the lead control personnel from the
participating organizations. The controllers will be instructed
on restart conditions.
Figure B-2. Sample Suspension and Termination Instructions
Sometimes, facility locations and offsite organizations have
specific objectives (e.g., recovery and reentry) that are
not applicable to other exercise locations. If that is the
case, the senior controller should determine the extent of
play necessary to permit those locations/agencies to meet
their objectives. In such cases, termination of an exercise
may be staggered for different groups of responders.
Premature termination of an exercise presents a problem
among response organizations in that it may preclude them
from meeting their objectives. To prevent premature
termination, the Exercise Director should obtain concurrence
from lead controllers that objectives have been demonstrated
or sufficient opportunity has been provided for the
objectives to be demonstrated prior to terminating the
exercise.
B.3.3 Post-Exercise Activities
The two primary post-exercise duties for controllers are
facilitation of responder hotwash critiques immediately
following the exercise and participation in the exercise
evaluation process. Since the controllers ensured the pace
and focus of the exercise, they have unique understanding of
who performed response actions when and why and, therefore,
typically lead the hot wash critiques. Controllers
typically provide input to the critique and evaluation
process because of their in-depth familiarity with the
exercise activities and responder actions.
B.4 Evaluator Activities
Preparation is the key to effective evaluation. Persons
assigned as evaluators must be thoroughly prepared and
cannot simply show up on the day of the exercise.
Evaluations depend significantly on the way the
facility/site responds to an emergency and should be
based on the specific emergency management program and
specific scenario.
B.4.1 Pre-Exercise Activities
Evaluators should be trained and thoroughly prepared for
their assigned duties for each exercise. Understanding the
scope, exercise objectives, and evaluation criteria, and
being familiar with emergency plans and implementation
procedures will help ensure that the evaluators can
concentrate on observing the actions of the responders.
Evaluators have the following three primary responsibilities
prior to the exercise:
Obtain and review all emergency plans, procedures, and
checklists for the activities to be evaluated.
Understand the exercise objectives and know how they
relate to the evaluation criteria.
Attend the pre-exercise briefing for
controllers/evaluators. Also attend, if presented, the
exercise safety briefing for all participants that is
separate from the other pre-exercise briefings.
Additional information about the site or facility being
evaluated should be provided to those evaluators who work at
another facility or site. This information should include
the ERO structure, procedures, notifications, communication
systems, Incident Command System, facility walk-downs, and
maps.
Emergency Plans and Implementation Procedures
Evaluators should understand the Emergency Plans and
implementation procedures being used by the responders
they are evaluating. The evaluation includes an
assessment of responder implementation of the plans and
procedures, as well as an assessment of the adequacy of
these plans and procedures.
Emergency Plans and Procedures are frequently updated.
Failure to review plans and procedures may result in
evaluator errors and skewed evaluation results. The person
assigned as lead evaluator should ensure that evaluators
receive plans and procedures far enough in advance to allow
for a thorough review and evaluators assigned to the
exercise know the importance of the plan and procedure
review.
Exercise Objectives, Criteria, and Checklist Relationship
Evaluators must understand the relationships between the
exercise scope, objectives, evaluation criteria and the
evaluator checklists prior to the exercise. This
relationship provides the evaluator with insight on what
has to be done, which items are critical, and how
observations should be classified.
Exercises test response capabilities according to a set of
measurable exercise objectives. Each objective is
associated with a standard of performance or criterion,
which must be met to demonstrate that objective. Checklists
are derived from these criteria to assist evaluators in
measuring performance. The relationships between these
elements and how they fit together in the evaluation process
are described in the following paragraphs.
Generic exercise evaluation criteria are provided for each
Program Element in Chapter 4 Appendix D of DOE G 151.1-3,
and should be used to develop facility- and exercise-
specific evaluation criteria as part of the exercise
development process. Evaluators use exercise-specific
criteria to determine whether exercise objectives have been
adequately demonstrated. Figure B-3 exemplifies how
evaluation objectives, criteria, and checklist items are
tied together in evaluation materials.
Though checklists vary from site to site, they should
contain the following items:
Method for recording a chronology or timeline of observed
events
Standards related to a specific exercise objective
References for that standard
Criteria associated with the standard
Activity-specific criteria (list of activities to look
for)
Method for documentation and comment on these specific
activities
Evaluation checklists for documenting specific activities
may vary from yes-or-no answers for activity-specific
criteria to numerical evaluations and/or evaluator comments.
Standardized forms may simplify the process of documenting
observations and analyzing the combined results. A simple
form with brief instructions and space to list identified
concerns works well, as does a checklist that uses a rating
system for the objectives. Evaluation forms should be as
straightforward as possible and should have space to
identify the evaluator, location, activity observed,
responders observed, and the time and date.
Objective: Activate the EOC, achieve operational status, and
staff the EOC in accordance with xxxxx procedures.
Criteria:
P/E8.1 A facility is available for use as a command
center by the Emergency Director (ED) and the members
of the ERO during an emergency response.
P/E6.7 The ERO activation is based on actual or
potential emergency conditions.
P/E6.8 Initial response functions are performed by on
shift operations staff.
P/E6.9 The ERO is functionally staffed and activated in a
timely manner; key emergency response facilities are
operational within an hour after declaration of an
Operational Emergency.
References: Site Emergency Plan, Site EOC Implementing Procedure
CHECKLIST
Criterion P/E8.1 -
____A facility is available for use as an EOC.
Criterion P/E6.7 -
____ERO activation is based on criteria specified in the
Emergency Plan
Criterion P/E6.8
____On shift operations staff has completed initial response
functions according to the Emergency Plan.
Criterion P/E6.9 -
1. The following positions were staffed within 1 hour:
___ Emergency Director
___ Federal Communicator
___ Plume Modeler
___ . . . per procedure
2. The Emergency Director:
___ Completed a turnover from the shift supervisor and
assumed overall direction and control within
15 minutes of arrival at the EOC
___ . . . per procedure
OBJECTIVE STATUS
MET ______ NOT MET______ NOT
OBSERVED______
JUSTIFICATION:
Figure B-3. Sample Evaluation Form
Evaluator Package. Evaluators are issued materials for
review prior to the exercise evaluator briefing. The
evaluator package is a subset of the items provided in the
EXPLAN, as well as specific evaluator instructions and other
items. Because the EXPLAN can be very large, evaluators
should reorganize the information or remove sections from
the EXPLAN for field use so that the information critical to
their assignment is readily accessible. They should bring
the package to the exercise-specific briefing and be
prepared to discuss any questions related to the packages.
The evaluator package should include the following
information:
Cover letter discussing scenario confidentiality and
providing a point-of-contact
Chart or listing of the evaluation organization
Plans, procedures, and checklists used by the
organization or people evaluated
Detailed instructions for the evaluator, including a
schedule of events
Exercise scope
Exercise objectives and evaluation criteria
Evaluation checklist(s) and materials
Scenario material
Site map(s)
Instructions. Evaluators are provided with general and
specific instructions for supporting pre-exercise setup,
exercise conduct, and exercise evaluation (see Figure B-4).
Information provided in evaluator instructions may include
the following:
Location and layout of the facility or function to be
evaluated
Expected time of responder arrival
Logistical information
Communication contacts and equipment
Applicable plans and implementation procedures
Potential problem areas
Approved simulations
Each evaluator should review and become familiar with the
specific information provided in the package.
SAMPLE EVALUATOR INSTRUCTIONS
EOC EVALUATOR ________________________
1.Participate in the evaluator pre-exercise briefing on May
10, 2xxx, at 11:00 a.m. in the (location).
2.Report to the visitor center not later than 6:30 a.m. on the
morning of the exercise for transportation to the EOC.
Bring your evaluator package with you.
3.You will receive your lunch and evaluator identification
at the EOC.
4.Check in with the lead evaluator by telephone (X-1234) when
you arrive at the EOC.
5.Coordinate with the EOC controller for conduct of the
responder critique after the exercise.
6.Report to the at 5:00 p.m. for the evaluator
debriefing. Bring your evaluation sheets, checklists, and
notes from the critiques. Dinner will be provided. The
meeting will not go past 10:00 p.m.
7.Attend the DOE management debriefing at 8:00 a.m. the
morning after the exercise. The debriefing will be
conducted at the Federal Building, Room 123.
Figure B-4. Sample Evaluator Instructions
Exercise-Specific Training. Exercise-specific training is
conducted prior to the exercise and assists in preparation
of the evaluators to perform their functions. It may be
combined with the controller training and includes a
detailed briefing on exercise activities and the scenario.
This training provides the opportunity for evaluators to ask
questions and to ensure they completely understand their
roles and responsibilities. Evaluator questions should be
addressed and information clarified so that evaluators feel
confident they can effectively perform their assignments.
B.4.2 Activities during the Exercise
During the exercise, an evaluator's primary
responsibility is to document observations of responder
activities. This includes maintaining a chronology of
events and using the checklists to ensure that
accomplishment of key actions has been recorded.
Evaluators should report to their appropriate staging areas
with any notes, evaluation criteria forms, evaluator
identification, safety equipment (hard hats, safety
glasses), and other evaluation materials. In addition,
evaluators need to ensure communication arrangements are
adequate and verify that any equipment they will be using is
in working order.
During the exercise, evaluators should address any questions
or needs for clarification of information to the
controllers. They should not interact with responders to
preclude prompting responders or interfering with responder
performance.
Realism versus Simulation. The purpose of an exercise is to
demonstrate and evaluate response capabilities under
simulated conditions. Even though responders know that the
events are simulated and that an actual emergency is not
occurring, realistic presentation of information can create
an atmosphere that parallels that of a real emergency. The
exercise should be planned so that events and required
responses are as realistic as possible, thereby mimicking
the sense of stress inherent in any actual emergency
situation.
Whenever possible, every activity and response action should
be carried out exactly as it would be if the events were
real. Controllers should provide information to responders
in a form and manner consistent with what would occur during
an emergency and as a result of specific events or actions
taken by the responders. Responders must earn information
(e.g., attempt to check instruments, perform meter readings,
or take vital signs) before they are given the information.
However, responder(s) should walk or talk through, rather
than actually perform response actions to restore or realign
equipment using panel switches to avoid changing critical
process or facility equipment alignments and parameters.
Evaluators should evaluate methods used by controllers in
acting on or the disseminating scenario information.
Free Play. During an exercise, responders may interject
mitigating actions that are not included in, but can be
accommodated by, the scenario. In some cases, the scenario
timeline will be modified to accommodate this free play.
The controller team is responsible for controlling free play
Evaluators should note any free play activities in the
exercise and the actions taken by the responders. Free play
may indicate a better understanding of emergency management
and response activities by the responders than the exercise
planners. Evaluators need to be aware of what is going on
between controllers and responders during free play so they
can document the actions.
Actual equipment and procedural problems during the conduct
of an exercise interject a form of free play. Responders'
solutions to actual equipment or procedural problems on a
real-time basis during the exercise afford a valuable
opportunity to evaluate the conduct and training of the
responders. Controllers should allow responders to solve
such problems unless safety is compromised or exercise
limitations are exceeded.
During the exercise, the primary duty of evaluators is to
document responder performance. After the exercise, that
data will be used to determine whether the exercise
objectives were demonstrated.
Scenario Confidentiality. Scenario information should be
closely guarded to ensure its confidentiality. If
responders are aware of the scenario beforehand, it will
skew any assessment of the emergency response capabilities.
The following are some guidelines for evaluators:
Evaluators should be careful of what they say and to whom
because it may be overheard.
Evaluators should be careful when positioning themselves
to observe an activity to ensure they do not give away
information by their actions.
Evaluators should ensure that no one sees their scenario
material or comments. They should never lay their
scenarios, notes, or messages in a location where
responders can read them.
Documenting the Exercise.
Evaluators observe and document responder activities
during the exercise. It is essential that evaluators
keep accurate records and notes because these will form
the basis for evaluation of performance.
The value of exercise evaluation is the ability to provide
constructive feedback (positive or negative) to improve and
enhance the effectiveness of an organizations response to
emergencies. Accurate and detailed documentation is
critical in facilitating a full record of all the events in
an exercise and an understanding of responder actions.
Evaluators document the exercise by maintaining a chronology
of important events, decisions, and actions in their area.
Evaluators should document key activities for later
evaluation, especially those that require a timely response.
A list of these important events is included as part of the
EXPLAN - in the timeline or MSEL. Highlighting or noting
events that occur in an evaluators assigned location(s) is
an effective way to track responder activity.
Evaluators should review their chronologies and notes
immediately following termination to ensure an accurate
reconstruction of events and activities for discussion at
critique and evaluation sessions. Evaluation materials, as
well as critique notes and forms, become part of the
exercise documentation. Checklists and evaluation forms
should be completed as thoroughly and accurately as
possible.
Evaluation Basics. Experienced evaluators use the following
techniques for effective observation and evaluation:
Use checklists to confirm that exercise objectives are
met.
Take detailed notes concerning significant activities
observed, including the time of occurrence.
When more than one evaluator is assigned a facility/area,
divide responsibilities to ensure detailed observation of
responder activities.
Stay in close proximity to responder decision makers.
Focus on critical activities (e.g., dose assessment
decisions, protective action decisions, command and
control issues).
Although numerous events may occur simultaneously,
evaluators do not need to record all of the action. Knowing
which events are important eliminates superfluous
information and provides the kind of data most useful for
exercise evaluation. Important events to record include the
following:
Initiating scenario events (including when responders
first detect abnormal conditions)
Emergency facility activation and staffing completion
Reactions of responders to the scenario
Key decisions made by managers and the time they were
made
Deviations from plans and implementation procedures
Times when mitigating actions were completed
Locations for Monitoring. Evaluators should be located so
that they can observe responder actions and hear
conversations without interfering with those activities.
Certain conditions may warrant more than one evaluator being
located in a setting or area.
What information is critical to collect? Individuals who
prepare the AAR will analyze the results provided by all
evaluators to achieve an integrated evaluation of response
capabilities. Their analysis will focus on the measures
taken to mitigate the simulated emergency, the timing of key
events, decisions made, and actions taken. Potential
problem areas include the following:
Lack of timeliness in mitigating actions
Ineffective communication among responders and
organizations
Inadequate direction and coordination of field activities
Inability to monitor and assess scenario events
Ineffective command and control at the scene or response
facility
Control problems that hinder conduct of the exercise
Responder deviations from plans and implementation
procedures
Unclear plans or procedures that hinder responder efforts
Facility or equipment shortcomings that hinder responder
efforts
Evaluator Do's and Don'ts. Evaluators should know that
scenario data and conditions must not be changed without
obtaining the permission of the Exercise Director or other
designated persons. Evaluators should not interfere with a
responder's action, unless there is safety issue.
Responders should be free to make their own decisions and
should act on those decisions without interference. Listed
below are reminder do's and don'ts for evaluators.
Do's:
Be familiar with other controllers and evaluators.
Remember that there may be two time frames, a scenario
time and a real time. Scenario time may compress events
so that several days are played in a few hours.
Note any communications passed between time zones.
Identify the participants by title and function.
Be easily identifiable. Wear the prescribed identifier
(e.g., arm band, shirt, or name tag).
Position yourself to maximize your effectiveness.
Locate the telephone or radio (for field teams) you will
use and know how to use it.
Be sure you understand the scenario. Know precisely what
level of simulation is required and acceptable.
Work with the other evaluators. Make sure they are
reasonably aware of your actions and those of the
responders.
Make notes on responder's strengths and weaknesses
related to the activities, as well as areas for
improvement. Use critique sheets.
Attend responder hot wash critique to document
observations.
Attend the post exercise critique session to provide your
comments (if appropriate) and recommendations to the
Exercise Director.
Complete evaluation forms as soon as possible following
termination of the exercise, while details are still
clear in your mind. Identify your observations.
Collect copies of exercise-generated documents such as
notifications forms, media releases, employee
announcements, consequence assessments, etc.
Don'ts:
Don't leave your post at key times.
Don't ever prompt a responder!
Don't get in the way.
Don't answer questions from responders; refer them to the
controller.
Termination of the Exercise. Upon notification from the
Exercise Director or the senior controller, controllers will
announce the termination of the exercise. Evaluators should
note the time and circumstances associated with the
termination.
B.4.3 Post-Exercise Activities
The primary post-exercise duties for evaluators are
documentation of responder hotwash critiques immediately
following the exercise, participation in the exercise
organization critiques, and the exercise evaluation and
report-writing processes. Evaluators will work with other
members of the exercise organization during the evaluation
process to complete the picture of responder actions and
assess whether objectives were met, and what improvements or
corrective actions are needed. These evaluation activities
are covered in Section B.5, below. Specific instructions,
guidelines, and schedules for evaluators will be in their
evaluator packages.
B.5 Exercise Critique and Evaluation Process
Controllers and evaluators participate differently in the
exercise evaluation process. While the majority of
controller activity occurs during the exercise, evaluators
perform the majority of their functions after the exercise
is completed. Both groups make significant contributions to
the evaluation process.
B.5.1 Evaluation Input
In addition to evaluator observations and documentation, the
following sources of information may be used to evaluate the
exercise.
Self-critique forms
Exercise critique comments
Exercise evaluation materials completed by controllers
Observations contained in the post-exercise reports
submitted by participating agencies
B.5.2 Critique of the Exercise
A series of formal critiques is conducted after the exercise
to provide participants (responders, controllers, and
evaluators) the opportunity to identify and discuss
observations (both positive and negative).
Responder Hotwash/Critique. This critique occurs
immediately after the exercise and is facilitated by the
controller team at each location. The purpose of the
critique is to provide a forum for constructive feedback on
the exercise by the responders. The identification of both
positive and negative observations provides a starting point
for improving emergency response capabilities. This is a
unique opportunity for responders to discuss the event and
to provide their own perspectives on the activities.
Controllers may partake in the discussion of the
observations. Evaluators usually remain silent and document
the observations and feedback from the responders.
The critique should be performed while exercise activities
are still fresh in the minds of the responders, controllers,
and evaluators. Responders may identify any weaknesses,
shortfalls, or improvement items. They evaluate their
plans, procedures, and task checklists for specific response
organization positions, equipment and supplies, facility
layout, and performance. For smaller exercises, the
facility director (exercise responder) often conducts the
critique, but it may also be under the direction of the
facility lead controller. Controllers should answer
questions on the timeline and scenario. This session can
also be used to clarify and verify any information on which
there were questions. Responders usually have a basic
understanding and evaluation of their job performance during
the exercise.
Controllers and evaluators should not provide the
responders with details of any observations during this
critique. Controller input should be limited to feedback
concerning the actual event scenario, as opposed to the
outcome of exercise.
Responders should be reminded that all controller/evaluator
observations are preliminary and may be revised based on
information from other evaluators.
If an evaluator or controller did not observe specific
aspects of an organization's performance, the exercise
responders may be asked to comment. Since it is critical
that the evaluators not prompt or coach responders during
the exercise, the evaluator should raise all questions of
this nature through the controller after the exercise
activities have been completed. These aspects should be
indicated in the evaluation as being provided by responders.
A responder self-critique form can be used for documenting
responder information about the exercise. Normally, the
controller distributes these forms immediately before the
critique begins. They should be collected after the
critiques along with all attendance or participation
rosters. Controllers should emphasize to responders that
the self-critique forms provide the opportunity to candidly
comment on emergency response activities and effectiveness
of the exercise.
Evaluation Critique. This critique session generally occurs
the day following the exercise and includes participation by
all controllers and evaluators. This critique should
provide the forum for discussion and correlation of
individual observations, the formulation of exercise
findings, determination of objectives demonstrated, and
determination of overall exercise performance. Preliminary
discussion on recommendations for corrective and improvement
actions should be initiated.
Formal critique sessions are usually several hours in length
and address, at a minimum, the following elements.
Reconstruction and review of scenario events and
shortcomings in the scenario or exercise conduct.
A comparison of anticipated versus actual responder
activities.
An assessment of performance based objectives and
criteria.
An assessment of the adequacy of plans and procedures.
An assessment of the adequacy of facilities, equipment,
and communications.
The first part of the critique is devoted to reconstruction
of scenario events and response activities. Timelines
should be reproduced for major evaluations, such as the
troubleshooting and restoration of a needed piece of vital
equipment. At this time, evaluators will organize and
consolidate their documented observations. The controllers
will provide input to the evaluators' documentation.
After this initial documentation is complete, the lead
evaluator for the exercise will facilitate a review of the
events (using the timeline) to document the interactions
between response organizations. This is generally time-
consuming, but it provides the information required to check
the communication process among all response organizations,
resulting in a consolidated exercise timeline of events that
actually occurred.
After this process is completed, the individual evaluators
should have sufficient information available for determining
whether the responders demonstrated the exercise objectives.
Although the evaluation is primarily concerned with the
exercise objectives, collateral observations of responder
activities that should be mentioned (positive and negative)
are documented.
Each evaluator should develop a rationale for the evaluation
from their respective point of view. This assists the
evaluation report writer in assessing conflicting
information that may occur from different sources. When the
exercise demonstration is substantially at variance with
what was expected, the evaluator should describe this in
enough detail to provide a sense of what occurred.
The critique should end with a discussion of the preliminary
results of responder performance for each exercise
objective. Evaluator notes and materials should be
collected at the conclusion of this session.
Senior Management Briefing/Critique. Key participants
should attend a senior management critique, including:
management-level responders, the exercise director, the lead
controller(s) and the lead evaluator(s). The overall
exercise performance, significant observations, findings,
and preliminary corrective and improvement actions should be
addressed.
B.6 Exercise Report
The AAR is prepared by the evaluation team to document
evaluation of overall exercise performance. This report is
the responsibility of the lead evaluator. Information from
the formal evaluation and critique process provides the
supporting documentation necessary to generate the exercise
report. Additional exercise reports may also be prepared by
any external Departmental organizations that evaluated the
exercise.
The AAR covers the schedule, scenario, participants'
activities, observations, and recommendations for corrective
actions. The AAR considers the observations and evaluations
made by the evaluators, controllers, responders (self-
critiques), and other participating organizations. The AAR
may contain the following.
Executive Summary
A narrative summary with introductory and general
statements noting exercise scope, purpose, objectives,
participants, and an overall performance (rating may or
may not be assigned) of the exercise.
Detailed findings for each objective, including positive
and negative comments regarding the effectiveness of
emergency planning and preparedness elements.
Recommendations for correcting negative findings.
Once the AAR has been drafted, the evaluators should review
the report for accuracy. Evaluator findings will in turn be
reviewed to ensure responders were measured against the
evaluated organization's plans and procedures. Because
perceptions differ, the report writers may find it necessary
to adjust or level various findings to achieve
standardization and consistency within the AAR. The lead
evaluator should approve the leveling of any findings.
4. READINESS ASSURANCE
4.1 Introduction
The purpose of this chapter is to assist DOE and NNSA field
elements in complying with the DOE O 151.1C requirement that
a Readiness Assurance program be in place to assure that
emergency plans, implementing procedures, and resources are
adequate and sufficiently maintained, exercised, and
evaluated and that improvements are made in response to
identified needs.
Readiness assurance programs provide assurances that the key
activities of emergency management (planning, preparedness
and response) are effective in maintaining an adequate and
reliable response. Readiness assurance performs an
essential role in the development, management, and
improvement of emergency management programs. Its
structured process of evaluations followed by a rigorous
implementation and tracking of program improvement ensures
an efficient and timely progression toward a high quality
emergency management program. For this reason, this chapter
of the Emergency Management Guide (EMG) will stress the
importance of and provide guidance for the implementation of
a formal, structured, and reliable readiness assurance
program for each DOE/NNSA facility/site and activity.
The readiness assurance requirements in DOE O 151.1C and the
guidance contained in this chapter are consistent with
DOE P 226.1, Department of Energy Oversight Policy, and
DOE O 226.1, Implementation of Department of Energy
Oversight. DOE P 226.1 establishes that all levels of the
Department must have an assurance system to pursue
excellence through continuous improvement. DOE O 151.1C
requires all levels of the Comprehensive Emergency
Management System, facility/site or activity (contractor),
Cognizant Field Element, and Cognizant Secretarial Officer
to conduct annual self-assessment of their programs. This
chapter of the guidance concentrates on acceptable methods
of implementing those requirements at the facility/site or
activity level, with Section 4.6 specifically addressing
self-assessments.
In addition, DOE O 151.1C follows the DOE Oversight Model
(DOE P 226.1) with readiness assurance requirements for the
Cognizant Field Element and Cognizant Secretarial Officer to
ensure the continuous improvement of the Comprehensive
Emergency Management System at all levels of the Department.
Teams from the Cognizant Field Element and/or Cognizant
Secretarial Office levels most often employ the structured
approach to emergency management evaluations discussed in
Section 4.4. The DOE Office of Health, Safety, and
Security, Office of Emergency Management Oversight (HS-63)
conducts independent oversight of emergency management
programs.
A potential readiness assurance effort is the use of an
accreditation program at the site level. Under the auspices
of the Emergency Management Issues Special Interest Group
(EMI SIG), an Emergency Management Accreditation (EMA)
Program is under development. Accreditation provides an
evaluation of the site emergency management program by
independent third-party evaluators. The accreditation
process focuses attention on areas that need improvement as
well as recognition of excellence. The EMA Program goes
beyond the minimum DOE O 151.1C requirements for each
program to conduct an annual self-assessment of the
emergency management program. The accreditation program is
optional -- sites may or may not elect to pursue
accreditation. After successful conduct and validation of a
pilot accreditation assessment, corresponding guidance will
be developed.
Section 4.2 presents a discussion of the general approach to
readiness assurance evaluations and continuous improvement.
Section 4.3 contains an approach for implementing a formal
readiness assurance program of evaluations, improvement, and
documentation. The discussion in Section 4.4 introduces
performance-based evaluations and their application to the
emergency management program elements, using a standard set
of evaluation criteria. The process of evaluating programs
is detailed in Section 4.5, self-assessments and post-
accident evaluations are addressed in Sections 4.6 and 4.7,
respectively, and a basis and approach for developing
performance indicators for emergency management programs is
presented in Section 4.8. Format and content guidance for
Emergency Response Assurance Plans (ERAPs), criteria for
performance-based evaluations, and a systematic approach for
performing self-assessments are contained in Appendices C,
D, and E, respectively.
This guidance is designed primarily for facilities/sites and
activities required to implement an Operational Emergency
Hazardous Material Program and is directed at operations and
emergency management staff at Field Elements and operating
contractor organizations responsible for DOE and NNSA
facilities/sites and activities. The guidance focuses
specifically on the personnel responsible for implementing
readiness assurance programs and DOE/NNSA personnel
responsible for performing evaluations and monitoring
corrective actions while overseeing emergency management
programs.
4.2 General Approach
DOE O 151.1C defines the framework of a readiness assurance
program consisting of evaluations, improvement, and
documentation. The Order specifies the responsibilities of
the facility/site or activity in performing internal
evaluations (self-assessments) to monitor their own
programs, in addition to their responsibilities to respond
to the external oversight evaluation activities of DOE/NNSA
Headquarters and Field elements. The facility/site or
activity must ensure that appropriate and timely
improvements are made in response to needs identified
through these evaluations, and through other activities such
as training and drills. A formal tracking system monitors
the implementation, verification, and validation of
improvements made through corrective actions developed for
findings from all sources. A formal program must be
implemented that takes advantage of lessons learned from
DOE/NNSA programs, as well as other similar government and
private activities. Finally, facilities/sites and
activities are required to develop or contribute to an ERAP
that documents the readiness of the emergency management
program based on emergency planning and preparedness
activities and the results of the readiness assurance
program, including evaluations and improvements. Thus, the
key elements of a readiness assurance program to be
discussed in this guide are the evaluation program, the
improvement program, and documentation of readiness
assurance results and forecast of activities in the ERAP.
Managing a readiness assurance program requires that the
emergency management organization take a systematic and
structured approach to carrying out and/or integrating the
results of program evaluations, program improvement, and the
program planning reflected in the ERAP. Readiness assurance
evaluation programs must be managed intentionally to test
and evaluate all emergency management functions and
activities. In addition, the evaluation program should be
based on identified needs each year, focusing on areas where
the organization has shown that it needs improved
performance. The improvement program must ensure that
corrective actions are rigorously developed and implemented,
verified and validated, to correct identified problems,
pursue lessons learned, and acquire relevant experience from
other sites and industries. Annually, the organization
should assess its overall readiness and identify resources
needed to address improvements, as necessary.
As will become evident in subsequent discussions, no single
evaluation tool provides the complete readiness evaluation
required for readiness assurance. Also, the evaluation
tools by themselves are not sufficient without an effective
improvement process. Finally, management commitment and
support is essential for committing resources necessary to
meet requirements, conduct evaluations, correct deficiencies
and weaknesses, and institute an effective lessons learned
program. The elements of the readiness assurance program
give the emergency management organization a framework to
determine what is needed to ensure a viable emergency
management, supported by convincing evidence. Hence, the
readiness assurance program should be a comprehensive
system, a collage of redundant and complementary
evaluations, which provide both general and focused program
validation, combined with a reliable, continuous improvement
process.
4.3 Readiness Assurance Program
The DOE/NNSA emergency management Order requires that each
facility/site or activity implement a readiness assurance
program consisting of the following three components:
evaluations, improvements, and documentation. The
discussion in this section will address this framework as
follows:
Evaluation Program: Provides guidance on what to
evaluate and methods of evaluation.
Improvement Program: Provides guidance on the concepts
of corrective actions, and lessons-learned, and the
importance of management support.
Documentation: Provides guidance on development and
submittal of ERAP.
In the sections that follow, each component of the readiness
assurance program will be addressed, providing guidance for
developing and maintaining a rigorous continuous improvement
process.
4.3.1 Evaluation Program
The first component of readiness assurance is the evaluation
program. The purpose of evaluations is to identify problems
in the emergency management program, usually focused on the
DOE/NNSA emergency management Program Elements and their
associated activities and functions. If problems are found,
then improvements can be accomplished. This process is
repeated periodically, so that continuous improvement
becomes a constant component of the emergency management
program. When no significant problems are identified
(i.e., no Deficiencies or Weaknesses), then the results
provide periodic assurances that emergency capabilities are
sufficient to implement emergency plans in response to
Operational Emergencies (OEs).
Evaluations of an emergency management program are focused
on three general areas:
Technical planning basis, plans, procedures and
supporting analyses
Preparedness activities
Response performance
The first area focuses on the technical planning basis for
the emergency management program and emergency plans,
procedures, and supporting analyses developed to implement
the program. The second area includes the planning and
preparedness activities, which support and maintain the
program. Finally, the response performance involves the
implementation of the emergency plan and procedures in
applying the response capabilities to mitigate consequences
and protect people and the environment. Evaluations in the
first two areas involve an assessment of projected response,
based on observations resulting from document and analyses
reviews, training data analysis and validation tests, the
exercise program, corrective action tracking, etc. The
response performance involves direct observations of actual
integrated response to a simulated emergency situation.
Evaluations of projected response based on plans,
procedures, supporting analyses, and preparedness activities
can be comprehensive, since the documented response planning
and the preparedness activities are intended to apply to the
full planning basis of the program. However, evaluators are
limited in their ability to assess, and hence, predict
actual responder performance since they observe no
demonstration of trained responders. They must simply rely
on their assessment of the training responders have received
and the plans/procedures they are to follow. In contrast,
given the necessary resources, evaluations of demonstrated
response are unlimited in reviewing and assessing all
aspects of the responder performance. However, they are not
comprehensive since the response evaluated is focused only
on one scenario from the technical planning basis. The
value of these two evaluation techniques (i.e., projected
and actual response) lies in their complimentary aspects,
one compensating for the weaknesses in the other. Both
techniques should be conducted as part of an effective
readiness assurance program.
Tools or processes for performing formal evaluations can
differ for each of the program components discussed above,
as follows:
Plans, procedures, supporting analyses, and preparedness
activities are evaluated using the following evaluation
tools:
Program Evaluations
Performance Indicators
Response performance is evaluated using:
Limited Scope Performance Tests (LSPTs)
Exercise Evaluations
Performance Indicators
Program Evaluations. Program evaluations involve a
comprehensive examination and assessment of the readiness of
an emergency management program, based on data collected
from the following sources: documents, response tools,
interviews, preparedness observations and data/records
analyses. None of these sources or associated collection
methods provides the evaluator with a complete picture of
the emergency management program, as indicated below:
Document reviews Expected/projected performance only.
Operation/utilization of response tools Demonstrates
that tools are in place, but not their correct use in a
response.
Interviews with responders Less stressful environment
than a response.
Observations of training and drills Lack of realism;
less stressful environment.
Analysis of preparedness data (e.g., training and
participation records) Verification of training and
participation not a validation of training effectiveness.
Performance Indicators. Performance indicators also
represent an evaluation tool for monitoring, tracking, and
analyzing specific parameters that reflect the
characteristics of a programs preparedness activities.
They can provide a timely indication of problems developing
in program readiness, if key parameters have been identified
that might predict performance degradation. Performance
indicators, which reflect response performance obtained
directly from observations during exercises, can also be
used to track and identify unacceptable trends associated
with responder activities. Section 4.7 addresses
performance indicators in more detail.
Limited Scope Performance Test (LSPTs). The LSPT is a
technique used by evaluators to observe and evaluate
performance in a single emergency response function or task.
The test is conducted in isolation, separate from any other
response functions or activities, usually in a classroom-
like setting. LSPTs involve an evaluation of the
performance of a small group of responders or may focus on
individual responders. Generally, scenario information is
presented to the responder(s) who in turn are asked to
demonstrate knowledge of procedures, familiarity with
interfaces, required communications, resources, and decision-
making capabilities. Example applications of the LSPT may
include:
Assessing the capability of site personnel who are
responsible for emergency categorization and
classification to make decisions and communicate those
decisions as required by site procedures and Emergency
Action Levels (EALs).
Testing select managers on ability to develop and
implement a reentry plan in response to a security
initiated chemical release on site.
Testing personnel on ability to select the correct EAL
given a set of emergency conditions.
This technique provides an effective means for validating
the training that a specific responder has received or, more
generally perhaps, the training program for the specific
response capability.
Exercise Evaluations. Exercise evaluations involve the
evaluated observations of an integrated response to a
simulated emergency situation, where multiple organizations
or activities must function together to mitigate the
emergency. Exercise objectives ensure that the opportunity
for evaluation exists by focusing on specific activities or
functions. Specific objectives provide the basis for
evaluating/validating the performance of response
capabilities. Exercise objectives clearly state what is to
be demonstrated; they are specific, attainable, and
measurable. Hence, the response capabilities to be
evaluated are limited in that only one specific scenario is
simulated.
There are several variations of the exercise that differ in
scope and focus. For a more detailed discussion of these
variations see DOE G 151.1-3, Chapter 3. A brief discussion
of the application of these exercises is included below:
Tabletop Exercise - Tabletops can be applied to any
working group(s) or team(s) whose successful performance
depends on the timely and appropriate interaction of all
of the participating individuals or teams involved an
emergency response. The scope is usually limited to the
participants present and the focus is on the decision-
making activities of those interacting. Control cells
are used to simulate non-participants.
The EMI SIG has developed A Guide for Conducting
Emergency Response Tabletop Activities, March 1998, and
is available at http://www.orise.orau.gov/emi.
Additionally, the Office of Independent Oversight and
Performance Assurance (HS-63) issued an Emergency
Management Tabletop Performance Test Inspectors Guide,
available through the website at
www.ssa.doe.gov/Sp40/sp43/docs.html.
Facility & Site Exercises - A formal exercise program is
established at a DOE/NNSA site to validate all elements
of the emergency management program over a five-year
period. Each exercise should have specific objectives
and be fully documented. Each individual facility
exercises its facility-level emergency response
capability annually, including at least a facility-level
evaluation and critique. Each facility has an external
Departmental evaluation at least every three years. Site-
level Emergency Response Organization (ERO) elements and
resources participate in at least one exercise annually.
This annual site-level exercise is designed to test and
demonstrate the site's integrated emergency response
capability. For multiple-facility sites, the basis for
the annual site-level exercise must be rotated among
facilities. At least once every three years offsite
response organizations are invited to participate in a
site-level full-participation exercise.
The evaluations of these exercises represent essential
contributions to the readiness estimate for the emergency
management program. As a readiness assurance technique,
the evaluation of a well-controlled and designed exercise
(especially a full-participation exercise) most nearly
estimates the readiness of the program to accomplish the
goals of emergency response
to mitigate consequences
and protect people and the environment.
No-Notice Exercises (NNXs). The basic objective of a NNX
is to test/assess the ability of initial responders of
the ERO to respond to a simulated OE under no-notice
conditions similar to those encountered during an actual
emergency. The duration of the NNX is normally limited
to about 2-3 hours. The absence of scenario-specific
consequence data for use in the exercise is a major
factor limiting the duration of responder activities.
NNXs are currently focused on initial activation,
mobilization, and response activities.
The NNX is designed to require minimum resource
expenditure and cause only limited disruption of
facility/site or activity operations. In addition to the
participation of initial responders, the site/facility or
activity organization assigns a trusted agent to assist
in the identification of a credible emergency scenario
and to provide facility/site- or activity-specific
information. The Headquarters (HQ) DOE/NNSA Office of
Emergency Management (NA-41) schedules (with
facility/site or activity concurrence), conducts, and
documents the NNX and its evaluation, including the
development and coordination of the exercise design
package, providing an exercise director and
controllers/evaluators, conducting participant and formal
controller/evaluator critiques, and producing an After
Action Report (AAR). Although the NNX program began as a
HQ initiative, the approach can be readily adapted by a
contractor organization as a tool in their self-
assessment program.
The evaluation program should focus the available evaluation
tools discussed above on the specific issues/problems
identified through previous evaluations, tracking
performance indicators, or other preparedness activities
such as training and drills. Each tool has advantages and a
scope that is appropriate for specific aspects of the
emergency management program; each also has limitations that
should be considered when applying each to an issue. Table
4-1 provides some indication of the focus and limitations of
these tools.
According to DOE O 151.1C, DOE/NNSA facilities/sites and
activities are expected to implement an emergency management
self-assessment program to identify problem areas (Cf.
Section 4.6). An effective self-assessment can include more
that a single annual program evaluation or exercise
evaluation. A broader interpretation includes the usual
internal evaluations together with data collected from
various observations, review and monitoring activities over
course of the year. This involves a systematic selection of
the areas and program elements for emphasis. A robust self-
assessment program, in addition to likely becoming the major
component of the readiness assurance program in the future,
will assist in ensuring that problems are self-identified
and corrected, without requiring the influence of external
evaluators.
Table 4-1. Focus and Limitations of Evaluation Tools
Method Focus Limitation
LSPT Single function or Generally does not
venue performance involve the full
ERO, so not all
functions and
interactions are
evaluated; isolated
decision-making.
Tabletops Integrated decision- Focused on decision-
making making, not on
resulting
performance
Facility & Site Onsite integrated Interactions with
Exercises response offsite responders
not observed
Full Integrated offsite Response to only
Participation and onsite response one scenario
Exercise
NNX ERO Lack of consequence
data to expand
exercise play
Program Comprehensive No performance
Evaluation response planning observed
Self-Assessment Detailed, Limited evaluation
comprehensive by owners of
review program
4.3.2 Improvement Program
A strategic objective of an emergency management readiness
assurance program should be continuous improvement. A
strong, reliable readiness assurance program will help an
organization ensure that appropriate and timely improvements
are made in response to needs identified and will provide
the organization with a direction and a path forward to
achieve an effective and efficient emergency management
program. The two key elements of a readiness assurance
improvement program include:
Corrective Action Program
Lessons Learned Program
Corrective Action Program. A corrective action program is a
readiness assurance process for continuously improving an
emergency management program. This continuous improvement
results from reliable implementation of corrective actions
for findings (e.g., Deficiencies, Weaknesses) in all types
of evaluations, including both internal self-assessments and
external evaluations.
A Corrective Action Plan (CAP) is the formal documented
response to findings that have been identified in program
and exercise evaluations or through observations in other
preparedness activities, such as training and drills. The
corrective action itself consists of the means, measures,
and methods proposed by the emergency management
organization for addressing and fixing the identified
problem area.
In preparing/writing a CAP, the conditions, circumstances,
situation, and causal factors that led to the finding should
be described. A description of the specific corrective
action(s) that will be taken to remove the cause of the
problem and to resolve the finding must be addressed. The
extent and prevalence of the same or similar problem area
should also be indicated in the write up. Completion of
corrective actions must include a verification and
validation process, independent of those who performed the
corrective action, that verifies that the corrective action
has been put in place and validates that the corrective
action has been effective in resolving the original finding.
A general description of the conduct of an independent
corrective action effectiveness review for verification and
validation should be included with the CAP.
Corrective actions that address revision of procedures or
training of personnel are particularly urgent and should be
assigned a high priority and completed before the next
evaluation of the program. In contrast, a corrective action
may require some time before it is finally in place. As
necessary, non-permanent interim measures should provide
control over a deficient situation or condition to limit the
hazard or the possibility of emergency response failure.
Any interim measures or compensatory actions should be
described in the CAP.
The recommended content of a CAP is summarized below:
Conditions, circumstances, situation, and/or causal
factors that led to the finding
Extent and prevalence of similar or the same/repeat
problem area
Specific corrective action(s) that will be taken to
remove the cause of the problem and to resolve the
finding
Interim measures or compensatory actions taken
General description of the conduct of an independent
corrective action effectiveness review for verification
and validation
These recommendations can be used for self-assessment
corrective actions and for corrective actions where the
required content of the formal corrective action is
unspecified. Detailed requirements associated with the
management of specific Departmental corrective actions are
contained in DOE O 414.1C, Quality Assurance.
Each emergency management organization should maintain (at
least) an internal system for ensuring that corrective
actions are acknowledged and progress in implementing
corrective actions is tracked. Tracking systems should
enable managers to identify corrective action status
(open/closed), to assign responsible staff, to ensure that
scheduled commitments for corrective action are met and that
the effectiveness of corrective actions is validated.
Tracking systems may also enable organizations to sort
corrective actions and collect data by category and cause.
This can help organizations to more effectively focus on
prevention and improvement. Data concerning corrective
actions may also be monitored in the organizations site
performance measurement program.
Program responsibilities for acting on corrective actions
are stated in DOE O 470.2A, Security and Emergency
Management Independent Oversight and Performance Assurance
Program, Attachment 2, Contractor Requirements Document, as
follows:
When notified of a significant vulnerability, responsible
contractor organizations must:
Promptly identify and implement immediate corrective
actions to mitigate the identified risk or
vulnerability.
Develop and implement long-term actions to eliminate
the vulnerability or reduce the level of risk to an
acceptable level as soon as possible.
Contractors must prepare, implement, and track to
completion approved preliminary, interim, and final CAPs
that address issues and concerns identified during the
assessments and evaluations.
CAPs are to be based on analysis of underlying causal
factors to determine if systemic program weaknesses
exist.
[A more detailed description of contractor assurance
programs, including information related to issues management
and corrective action systems, can be found in
DOE Order 226.1, Implementation of Department of Energy
Oversight Policy.]
Lessons Learned Programs. Decision-making, planning, and
execution of work should be founded on the best professional
and industrial practices available. DOE/NNSA management has
placed significant emphasis on the concept of lessons
learned across multiple health and safety disciplines, to
ensure that knowledge and experience is shared among
individuals and organizations in order to benefit from the
experiences of others. Broad application of the lessons
learned concept is important to the Departments commitment
to maintain effective Integrated Safety Management Systems
(ISMS). Outside the DOE, corporations, government agencies
and departments, and the military are actively using lessons
learned information to help them achieve their varied goals
and missions.
The emergency management community is committed to
benefiting from the experiences, good and bad, of our peers,
both from within and outside the DOE/NNSA complex. A lesson
learned may be a good practice or innovative approach that
is captured and shared to promote repeat application. A
lesson learned may also be an adverse practice or experience
that is captured and shared to avoid recurrence. A lessons
learned program is a principal component of emergency
management organizations whose culture is committed to
continuous improvement. As such, an emergency management
lessons learned program should strive to:
Reduce the number of problems encountered by sharing
information.
Improve program efficiencies and effectiveness by
exchanging information and experience with others in
emergency management.
The following are important functions of an effective
emergency management lessons learned program:
1)Identify: Mechanisms should be in place to identify
lessons learned
2)Document: A process for documenting lessons and success
stories
3)Validate: Validate the lessons learned to ensure each is
meaningful and not repetitious
4)Store: A database for capturing and storing lessons
learned information
5)Share: Forum for sharing information between
organizations, both within the site organization and
within the broader DOE community. The EMI SIG provides
such a forum for the DOE complex. [The DOE Society for
Effective Lessons Learned Sharing (SELLS) also provides
complex-wide forum for more generally applicable lessons
learned to be found at:
http://hss.energy.gov/CSA/Analysis/ll/sells.]
6) Evaluate: Establish a formal method or process to
evaluate the applicability of lessons learned to the site
or its facilities.
7) Incorporate and utilize: Incorporate the actions to
address applicable lessons learned into the site/facility
corrective action tracking system. Use lessons learned
to improve the program.
8)Follow-up: Follow-up process is implemented to ensure
actions are taken
In summary, an effective readiness assurance program
includes a system for incorporating and tracking lessons
learned from training, drills, actual responses, and a site-
wide lessons learned program. DOESTD750199, The DOE
Corporate Lessons Learned Program, provides guidance on use
of the system. Additional information on Lessons Learned
programs can be found in the SELLS website within DOE.
4.3.3 Documentation of Readiness Assurance
ERAPs are documented assessments of the development,
implementation, and maintenance of Emergency Management
Programs. The ERAP is also a planning tool to identify and
develop needed resources and improvements. An ERAP
highlights significant changes in emergency management
programs (i.e., planning basis, organizations, and
exemptions) and compares actual achievements to goals,
milestones and objectives. The information reported in the
ERAP should provide assurances to the organizations
management as well as DOE/NNSA Headquarters that emergency
management programs are ready to respond. The ERAP is
designed to be an emergency preparedness management tool for
all levels of management.
While the provisions of the facility/site or activity
readiness assurance program are documented in the emergency
plan, the ERAP documents the annual assessment of readiness
assurance activities. The ERAP provides detailed
information on an annual basis about how continuous
improvement in the emergency management program is being
achieved and how the complimentary tools of program and
exercise evaluations are being used to ensure that the
emergency management program is ready to respond.
Following the direction of the Government Performance
Results Act (GPRA 31 U.S.C. 1115 and 1116), the time
period for ERAP coverage was reduced to the immediate past
fiscal year and the new fiscal year in DOE O 151.1C. The
ERAP includes the goals for the immediate past fiscal year
and compares those what was accomplished during the past
year, and identifies the goals established for the new
fiscal year. For example, a facility/site ERAP submitted on
9-30-07 would compare the progress made during FY07 (10-1-06
to 9-30-07) against the goals that had been set for FY07, as
well as identify the goals that were set for FY08 (10-1-07
to 9-30-08).
In addition, the ERAP should contain the results of
emergency preparedness activities, external
evaluations/assessments, self-assessment activities,
exercise after-action reports, corresponding corrective
action plans, improvements based on the lessons learned
program, and summary information about the facility/site or
activity emergency management program in sufficient detail
to be understood by managers that are not in direct contact
with the program. Appendix C contains format and content
guidance for the ERAP. The following general guidance
regarding content should also be applied:
The level of detail should not be voluminous.
Include information that will help support the
improvements needed for the program.
Details about daily processes are not relevant.
Ensure that information included is complete and
accurate.
The most detailed information should be from the past
year.
DOE O 151.1C contains specific requirements addressing
submittal and review and the general requirements for the
ERAP summarized as followed:
The facility/site or activity contractor is responsible
for preparing the ERAP.
An ERAP may require review for classified or controlled
information prior to submittal.
It must be submitted to the cognizant Field Element
manager for approval.
The cognizant field element manager should review the
ERAP; comments should be addressed to the contractor
representative.
ERAPs should be consolidated for facilities/sites and
activities under the supervision of the Cognizant Field
Element Manager.
The consolidated ERAP should also contain the same type
of discussion regarding the emergency management program
at the Cognizant Field Element level as is contained in
the facility/site or activity ERAP
The consolidated ERAP should be submitted to the
Associate Administrator for Emergency Operations and the
responsible Program Secretarial Officer each year
The Associate Administrator for Emergency Operations will
prepare, in coordination with the responsible Program
Secretarial Officers, an annual report summarizing the
status of the DOE Emergency Management System.
4.3.4 Management Commitment
Management leadership, commitment, and active involvement
are essential for emergency management program improvement.
Management should be made aware of the requirements and
performance expectations in order to integrate these with
the strategic plans of the organization. Site and facility
managers should be kept informed and involved in the
emergency management program.
Management should review the emergency management program at
planned intervals to ensure its continuing suitability,
adequacy, and effectiveness. This review should include
assessing opportunities for improvement and the need for
changes. Records from management reviews should be
maintained. Input to management reviews should be provided
by personnel responsible for emergency management and should
include information related to:
Exercise performance
Results of internal and external evaluations
Findings involving the emergency management program
Complaints or significant communications with offsite
agencies
Status of corrective actions
Emergency management performance measures and progress in
meeting targets
Changes in regulatory and statutory requirements that may
impact resource needs of the emergency management program
Changes at the facility/site that may impact preparedness
or response
Recommendations for improvement
An annual ERAP report
Site and facility management should ensure that necessary
interface and cooperation is maintained between emergency
management and the various departments at the facility/site.
The nature of emergency management requires effective
ongoing coordination and cooperative interfaces with
organizational groups such as medical, fire/Hazardous
Materials (HAZMAT), training, environmental health and
safety, health physics, engineering, information systems,
security, public affairs/media relations, etc.
Line management participation is critical to ensuring that
corrective actions are handled efficiently and effectively
and that applicable lessons learned information is
effectively distributed and implemented throughout the
organization. Expectations of upper management must be
communicated to line management. Performance measures
established for the emergency management program should be
linked with performance expectations of senior management
and their commitment to the program.
Sufficient resources must be budgeted to maintain and
improve the program. In considering resource needs to
maintain emergency management and meet expectations for
performance improvement, the following should be factored
into requests to management:
Current resources
Changes at the facility/site that may impact preparedness
or response. These may include physical, information
systems, communication systems, organizational, and
financial changes
Changes in regulatory or statutory requirements
Changes in response capabilities of external EROs
Program effectiveness and achievement of performance
goals over the past year
Lessons learned (facility-specific, DOE/NNSA Complex and
industry)
New performance goals
Corrective action commitments
Continued improvement and success of an emergency management
program can only be ensured through management commitment to
maintain a response capability that is ready to respond
promptly, effectively, and efficiently to emergencies at
DOE/NNSA facilities/sites and activities.
4.4 Evaluations
An emergency management program consists of diverse
functions and activities whose collective objective is to
ensure that response capabilities will be ready when needed
and will be applied promptly, effectively, and efficiently.
The successful accomplishment of this objective involves
groups of emergency management functions and activities.
These groups are included in the Program Elements of the DOE
emergency management system. The evaluation of emergency
management programs will focus on these elements as a
convenient means for organizing the evaluation methodology
and processes to be discussed in this EMG.
The general, overall performance goal (or mission) of
DOE/NNSA emergency management programs can be summarized as
follows:
DOE emergency management programs will be ready at all
times to promptly, effectively, and efficiently apply the
necessary resources to mitigate consequences and protect
its workers, the public, the environment, and national
security in the event of an Operational Emergency
involving DOE/NNSA facilities/sites or activities.
The goal expresses the basic attributes of a DOE/NNSA
emergency management program that determine its expected
performance:
DOE emergency management programs will be ready at all
times . . .
DOE emergency management response is a standby system,
which must be ready (i.e., ready on demand) to respond
to an emergency event or condition when called upon.
Planning, preparedness, and readiness assurance are
continuous, ongoing activities of emergency management
programs that enable response capabilities to be ready to
respond . . . in the event of an OE.
. . . to . . . apply the necessary resources . . .
The emergency management response activities will apply
the necessary resources in the event of an OE.
The application of necessary resources is the OUTPUT of
the response activities of emergency management programs.
. . . promptly, effectively, and efficiently . . .
Resources will be applied promptly, effectively, and
efficiently.
Promptly: Produces the effect or result in a timely
manner, when needed
Effectively: Produces the desired effect or result
Efficiently: Produces the desired effect or result with
a minimum of effort, resources, or waste; works well
. . . to mitigate consequences and protect its workers,
the public, the environment, and national security . . .
The application of resources (i.e., OUTPUT) of the
emergency management response will mitigate consequences
and, in doing so, protect its workers, the public, the
environment, and national security.
The mitigation of consequences and protection of workers,
the public, the environment, and national security are
the ultimate objectives and desired OUTCOME of the
emergency response.
. . . DOE/NNSA facilities/sites or activities.
This guidance is focused on the response to OEs involving
DOE/NNSA facilities/sites or activities.
The performance goal expresses the performance
characteristics of the response OUTPUT and the desired
results of its application, the OUTCOME. Since the ultimate
objective of the emergency management program is to mitigate
consequences and protect, it is the success in accomplishing
this OUTCOME that should be evaluated.
However, only an actual emergency event can provide the
circumstances and situations that in reality require urgent
measures to protect workers, the public, the environment, or
national security. Only an actual emergency event will
produce realistic stress levels, challenging physical
environments, time constraints, real information
uncertainties, and operational problems. Hence, it is only
during an actual response that the OUTCOME of the emergency
response can be directly measured and judged and the true
readiness of an emergency management program estimated.
However, emergency events, especially OEs, are rare, and
most that have occurred are not sufficiently complex to
provide a comprehensive test of the overall program
readiness. Another method is needed for evaluating
emergency management programs and estimating program
readiness.
The aspect of the performance goal that can be tested and
evaluated on a routine basis is the capability of the
emergency response to be ready at all times to promptly,
efficiently, and effectively apply the necessary resources
(i.e., the OUTPUT). Thus, in the absence of adequate
response data from actual emergencies, a basic, practical
assumption of the methodology for evaluating DOE /NNSA
emergency management programs is the following:
An estimate of the readiness of the emergency
management response to apply the necessary resources (the
OUTPUT) will be the best available estimate (i.e., a
surrogate) of the readiness of the response to mitigate
consequences and protect personnel, the environment, and
national security (the OUTCOME).
This basic assumption represents the foundation of readiness
assurance evaluation methodology, which focuses on
performance-based evaluations of the OUTPUT of emergency
management programs.
The first evaluation option to consider is an evaluation of
performance during an integrated response to a simulated
emergency scenario in an exercise. This direct observation
of performance provides an excellent contribution to an
estimate of readiness, and multiple tests would address a
full spectrum of potential emergency events. However, this
option is limited by the substantial resource requirements
and its invasive effect on normal operations. These
constraints result in a limited number of site-level,
integrated operations-based exercises, usually one annually
(per the DOE O 151.1C requirement), which is insufficient by
itself to ensure that the response will perform as needed
during any emergency.
The additional requirement for each facility on a site to
conduct a facility-level operations-based exercise (per the
DOE O 151.1C requirement) provides the opportunity to
evaluate the performance of the facility-level Emergency
Response Organization (ERO) components. These direct
observations of performance at the facility-level provide an
excellent contribution to an estimate of readiness of the
emergency management program; multiple tests across the site
can address a full spectrum of potential emergency events
and response elements. However, essential site-level
response assets may be only simulated during these exercises
resulting in a test of integrated response limited to
facility-level assets.
Hence, the limitations of the direct observation method,
discussed above, lead to reliance on a readiness estimate
for the system that also depends on assurances
(i.e., convincing arguments to inspire confidence) that
the response will perform as required. These assurances are
based on a comprehensive evaluation of plans, procedures,
supporting analyses, and the conduct of preparedness
activities, COMBINED with evaluations of performance during
formal exercises.
The emergency management system requires this combined
approach for evaluating performance primarily because it is
a standby system involving diverse and integrated functions
and skills. Output failures in a continuously operating
system of similar complexity will generally be discovered in
real-time. In a standby system, failures will only be
discovered when the system is called upon to operate. As a
result, since demonstrations of response performance are
limited, the standby system requires supporting activities
involving planning, testing, and routine maintenance
(e.g., training) to prepare the system to perform
successfully on demand. Each activity has an essential role
in maintaining the readiness of the emergency management
program and the performance of each will reinforce
confidence in performance of the overall system. Hence, a
robust readiness assurance program incorporates a focused
group of diverse techniques for evaluating the actual or
projected performance of the response and the continuing
performance of its supporting and sustaining
activities/functions.
4.4.1 Emergency Management Program Elements
The DOE/NNSA emergency management system consists of four
core activities:
Emergency planning including the identification of
hazards and threats, hazard mitigation, development and
preparation of emergency plans and procedures, and
identification of personnel and resources needed for an
effective response
Emergency preparedness including the acquisition and
maintenance of resources, training and drills, and
exercises
Readiness assurance including evaluations, improvements,
and documentation to assure that stated emergency
capabilities are sufficient to implement emergency plans
Emergency response including the application of resources
to mitigate consequences to workers, the public, the
environment, and the national security, and the
initiation of recovery from an emergency
In order to more clearly indicate the roles that the fifteen
(15) Program Elements of the emergency management program
perform, they are separated into the following three groups:
Technical Planning Basis (Planning), Programmatic Activities
(Planning, Preparedness, Readiness Assurance), and Response
Activities.
As indicated above, these groups represent the core
emergency management activities and encompass the fifteen
(15) Program Elements of the DOE/NNSA emergency management
program:
TECHNICAL PLANNING BASIS
Hazards Survey/Hazards Assessment
PROGRAMMATIC (ongoing activities)
Program Administration
Training and Drills
Exercises
Readiness Assurance
RESPONSE (standby activities)
Emergency Response Organization (ERO)
Offsite Response Interfaces
Emergency Facilities and Equipment
Categorization and Classification
Communication and Notification
Consequence Assessment
Protective Actions and Reentry
Emergency Medical Support
Emergency Public Information
Termination and Recovery
The above grouping separates the program elements by role
into the technical planning foundation of the program, the
ongoing programmatic activities that sustain the program,
and the standby response elements that respond or
contribute to response as needed in an emergency.
Fundamental emergency management activities are clearly
distinguishable by the time frame during which they perform.
In general, ongoing programmatic elements include
activities that take place on a continuous, periodic, or as-
needed basis during normal conditions to maintain the
readiness of the program to respond to an emergency. In
contrast, response elements are normally in a standby mode
and are activated to respond to an emergency event or
condition. Response elements include the organizational
structure, functions, products, activities, response tools,
etc., that directly participate in or contribute to a
response to mitigate consequences of an emergency.
Within each of the program elements, specific activities,
tasks, products, etc., and their related plans, procedures,
and tools contribute to the successful accomplishment of the
intended OUTPUT product(s) in each specific area. Examples
of individual products can include: event classification,
press briefings, source term estimates, Protective Action
Recommendations (PARs), consequence versus distance
calculations, exercise program for the site, training plans,
effectively trained ERO personnel, detection equipment
requirements, meetings with public or offsite response
organizations, Emergency Planning Zone (EPZ), etc.
The intended OUTPUT from each function or activity is
evaluated in terms of the desired qualitative and/or
quantitative performance characteristics of each of the
products. These performance characteristics are determined
by the role that the OUTPUT plays in meeting the overall
emergency management performance goal. The resulting
evaluations will be referred to as performance-based
evaluations, where the focus of the evaluation methodology
is not on specific, prescriptive details of plans,
procedures, calculation techniques, administrative
structure, etc., but on the desired performance
characteristics of the resulting OUTPUT.
The collective performance associated with the program
elements supports the overall performance goal of emergency
management programs. In order to provide a logical
structure for evaluating the overall emergency management
program, an individual performance goal has been developed
for the OUTPUT of each program element. These goals were
derived from the requirements of DOE O 151.1C, COMPREHENSIVE
EMERGENCY MANAGEMENT SYSTEM, augmented with additional
guidance from the EMG (DOE G 151.1-series), other DOE-
directives, and other Federal, tribal, state, and local
regulations.
Program Element Performance Goals - Technical Planning Basis
1.Hazards Survey. An examination of the features and
characteristics of the facility/site or activity that
identifies the generic emergency events and conditions
and the potential impacts of such emergencies to be
addressed by the DOE Comprehensive Emergency Management
System. The Hazards Survey identifies key components of
the Operational Emergency Base Program that provide a
foundation of basic emergency management requirements and
an integrated framework for response to serious events
involving health and safety, the environment, safeguards,
and security. For facilities/sites and activities
involved in producing, processing, handling, storing, or
transporting hazardous materials that have the potential
to pose a serious threat to workers, the public, or the
environment, the Hazards Survey provides a hazards
screening process for determining whether further
analysis of the hazardous materials in an Emergency
Planning Hazards Assessment (EPHA) is required.
2.Emergency Planning Hazards Assessment (EPHA). Performed
for each facility/site and activity involving at least
one candidate hazardous material, as identified through
the hazardous material screening process and indicated in
the Hazards Survey. EPHAs involve the application of
rigorous hazards analysis techniques that provide
sufficient detail to assess a broad spectrum of
postulated events or conditions involving the potential
onsite release of (or loss of control over) hazardous
materials and to analyze the resulting consequences.
Each EPHA reflects both the magnitude and the diversity
of the hazards and the complexity of the processes and
systems associated with the hazards, and provides the
technical planning basis for determining the necessary
plans/procedures, personnel, resources, equipment, and
analyses [e.g., determination of an Emergency Planning
Zone (EPZ)] for the Operational Emergency Hazardous
Material Program.
Program Element Performance Goals - Programmatic Elements
1.Program Administration. Effective organizational
management and administrative control of the
facility/site or activity emergency management program is
provided by establishing and maintaining authorities and
necessary resources commensurate with the responsibility
to plan, develop, implement, and maintain a viable,
integrated, and coordinated comprehensive emergency
management program.
2.Training and Drills. A comprehensive, coordinated, and
documented program of training and drills is an integral
part of the emergency management program to ensure that
preparedness activities for developing and maintaining
program-specific emergency response capabilities are
accomplished.
3.Exercises. All elements of an emergency management
program are validated over a multi-year period through a
formal exercise program. The exercise program validates
facility- and site-level emergency management program
elements by initiating response to simulated, realistic
emergency events/conditions in a manner that replicates
an integrated emergency response to an actual event as
nearly as possible. Planning and preparation use an
effective, structured approach that includes
documentation of specific objectives, scope, time lines,
injects, controller instructions, and evaluation criteria
for realistic scenarios. Each exercise is conducted,
controlled, evaluated, and critiqued effectively and
reliably. Lessons-learned are developed, resulting in
corrective actions and improvements.
4.Readiness Assurance. A Readiness Assurance program
provides a framework and associated mechanisms to assure
that emergency plans, implementing procedures, and
resources are adequate and sufficiently maintained,
exercised, and evaluated (including evaluations and
assessments) and that appropriate, timely improvements
are made in response to needs identified through
coordinated and comprehensive emergency planning,
resource allocation, training and drills, exercises, and
evaluations.
Program Element Performance Goals - Response Elements
1.Emergency Response Organization. An Emergency Response
Organization (ERO), a structured organization with
overall responsibility for initial and ongoing emergency
response and mitigation, is established and maintained.
The ERO establishes effective control at the
event/incident scene and integrates local agencies and
organizations providing onsite response services. An
adequate number of experienced and trained primary and
alternate response personnel are available on demand for
timely and effective performance of ERO functions.
2.Offsite Response Interfaces. Effective interfaces are
established and maintained to ensure that emergency
response activities are integrated and coordinated with
the Federal, Tribal, State, and local agencies and
organizations responsible for emergency response and the
protection of workers, the public, and the environment.
3.Emergency Facilities and Equipment. Facilities and
equipment adequate to support emergency response are
available, operable, and maintained. Specifically, an
adequate and viable command center is available as
necessary and personnel protective equipment is available
and operable to meet the needs of the responders.
4.Categorization and Classification. Major unplanned or
non-routine events or conditions involving or affecting
DOE or NNSA facilities/sites or activities by causing or
having the potential to cause serious health and safety
impacts onsite or offsite to workers or the public,
serious detrimental effects on the environment, direct
harm to people or the environment as a result of
degradation of security or safeguards conditions, or
release of (or loss of control over) hazardous materials,
are recognized promptly, categorized, and declared as
OEs. In addition to being categorized as OEs, events
involving the actual or potential airborne release of (or
loss of control over) hazardous materials from a
facility/site or activity also require prompt and
accurate classification based on health effect thresholds
(for initiating protective actions) measured or estimated
at specific receptor locations (i.e., facility and site
boundaries, etc.) Predetermined conservative onsite
protective actions and offsite protective action
recommendations are associated with the classification of
these OEs.
5.Notifications and Communication. Prompt, accurate, and
effective initial emergency notifications are made to
workers and emergency response personnel/organizations,
including appropriate DOE or NNSA elements, and other
Federal, Tribal, State, and local organizations and
authorities. Accurate and timely follow-up notifications
are made when conditions change or when the
classification is upgraded or the emergency is
terminated. Continuous, effective, and accurate
communications among response components and/or
organizations are reliably maintained throughout an OE.
6.Consequence Assessment. Estimates of onsite and offsite
consequences of actual or potential releases of hazardous
materials are correctly computed and assessed in a timely
manner throughout the emergency. Consequence assessments
are integrated with event classification and protective
action decision-making, incorporate facility and field
indications and measurements, and are coordinated with
offsite agencies.
7.Protective Actions and Reentry. Protective actions are
promptly and effectively implemented or recommended for
implementation, as needed, to minimize the consequences
of emergencies and to protect the health and safety of
workers and the public. Protective actions that can be
implemented individually, or in combination, to reduce
exposures from a wide range of hazardous material types,
include evacuation, sheltering, decontamination of
people, medical care, ad hoc respiratory protection,
control of access, shielding, radio protective
prophylaxis (e.g. administration of stable iodine,
chelating agents, or diuretics), control of foodstuffs
and water, relocation, decontamination of land and
equipment, and changes in livestock and agricultural
practices. Protective actions are reassessed throughout
an emergency and modified as conditions change. Reentry
activities are planned, coordinated, and accomplished
properly and safely.
8.Emergency Medical Support. Medical support is provided
for workers contaminated by hazardous materials.
Arrangements with offsite medical facilities to
transport, accept, and treat contaminated, injured
personnel are documented.
9.Emergency Public Information (EPI). Accurate, candid,
and timely information must be provided to workers, the
news media, and the public during an emergency to
establish facts and avoid speculation. Emergency public
information efforts must be coordinated with State,
Tribal, and local governments, and be part of Federal
emergency response plans as appropriate. Workers and the
public are informed of emergency plans and planned
protective actions before emergencies.
10. Termination and Recovery. An OE is terminated only
after a predetermined set of criteria has been met and
termination has been coordinated with offsite agencies.
Recovery from a terminated OE involves communication and
coordination with Tribal, State, local, and other Federal
agencies; planning, management, and organization of the
associated recovery activities; and ensuring the health
and safety of workers and the public.
The performance goals are deliberately general and broadly
based in order to apply to all Operational Emergency
Hazardous Material Programs across the DOE complex. The
performance described implies no facility/site- or activity-
specific characteristics, such as diversity and number of
hazards, size of site, number of facilities, ERO
composition, etc. Each goal characterizes the performance
of a group of emergency management functions/ activities
that support the functional area indicated by the program
element title. The collective performance described in the
goal does not imply that the individual functions or
activities associated with the program element are
necessarily dependent on one another so that a single
failure or several failures in the functional area fail the
whole element.
4.4.2 Program and Exercise Evaluations
Two methods are recommended for obtaining complementary
estimates of the readiness of an emergency management
program to respond, namely, program and exercise
evaluations:
A program evaluation involves an assessment of an
emergency management program based on a comprehensive
examination and evaluation of response plans and
procedures, administrative control mechanisms, planning
basis and supporting analyses, response tools
(e.g., computer models), resource availability, training
activities and results, training validation
(i.e., individual testing of trained personnel), overall
exercise program, organizational factors, etc.
An exercise evaluation involves an assessment of an
emergency management program based on the observation and
evaluation of the demonstrated integrated performance of
response capabilities during simulated emergency event(s)
or condition(s). The exercise evaluation is usually
combined with an evaluation of the validity of the
specific exercise (i.e., Conduct of Exercise) as a viable
test of the readiness of the program to respond.
The technical planning basis and programmatic activities are
generally evaluated during a program evaluation. The
evaluation focuses on observables associated with the INPUT
to the activity, the characteristics and conduct of the
ACTIVITY itself, and the OUTPUT of the activity. Response
activities are also evaluated during program evaluations.
However, since there is no observation of response to an
emergency event (as in exercises), the evaluation must
depend on expert extrapolation from the response planning,
procedures, and available tools to an anticipated or
projected response. The evaluation consists of a judgment
of the anticipated or projected performance based solely on
review and analyses of the planning and preparedness
activities.
Evaluation of response activities during a drill or exercise
involves an assessment of the demonstrated performance of
integrated response capabilities, based on observation and
evaluation of the actual OUTPUT from each program element.
Exercise evaluation will determine whether and how well
response functions and activities are performed based on
observations during a specific (simulated) emergency
scenario. [Note that some programmatic activities are
evaluated during an exercise to determine the reliability of
the planning and conduct of the exercise. This assessment
(i.e., Conduct of Exercise) provides an indication of the
value of the observed performance during a specific exercise
in estimating the expected response during an actual
emergency.]
ERO personnel perform response functions and participate in
activities during an exercise; actual failures in
performance can be observed. Similarly, during an
assessment of a programmatic element (e.g., Training and
Drills) in a program evaluation, inadequate or ineffective
training of response personnel might be discovered through a
record search or individual testing. This will also be an
actual failure, where accomplishments of the training
program do not meet emergency management program
expectations. In contrast, during a program evaluation of a
response element, the plans, procedures, and tools for a
specific activity are evaluated to determine whether they
are accurate, clear, unambiguous, effective, timely, etc.
If they were found to meet expectations by an evaluator,
then, based on the materials evaluated or interviews with
the ERO personnel, the desired OUTPUT of the activity during
a response would be satisfactory. In other words, no
projected failure would be suggested by the program
evaluation. On the other hand, if the procedures were
incorrectly written or the expected OUTPUT not appropriate,
then a failure might be projected (considered likely) during
an emergency response.
It is common practice during program evaluations to
construct LSPTs to be administered to ERO staff in order to
validate their proficiency in performing certain response
tasks (e.g., classification, protective action decision-
making) for which they would be responsible during an
emergency response. These tests validate the effectiveness
of the training received and are essential for evaluating
the performance of the Training and Drills, a programmatic
element. However, they should be interpreted as one
indicator of projected performance, but not as sufficient to
characterize expected performance as is a well-planned and
conducted, quality exercise or during an actual response to
an emergency.
As stated in the beginning of this section, two methods are
recommended for obtaining complementary estimates of the
readiness of the response capabilities of an emergency
management program, namely, program and exercise
evaluations. Neither type of evaluation is sufficient by
itself to provide a complete measure of the readiness of the
emergency management program. Program evaluation is
comprehensive in terms of hazards and activities. It
provides a measure of the applicability of the planning
associated with the program, as well as an evaluation of the
adequacy of programmatic activities that ensure the
maintainability of the program in sustaining a high level of
proficiency over time. Program evaluation, however, lacks a
true evaluation of actual, integrated performance of
response capabilities as observed in a real or simulated
emergency event.
Exercise evaluation, on the other hand, provides such an
evaluation of response performance during a simulated
emergency scenario. However, since the scenario involves
only selected response capabilities and personnel
appropriate to the emergency event, the evaluation of the
exercise is insufficient to generalize the overall
capabilities of the response. The exercise is a snapshot in
time that captures and evaluates performance in response to
the specific event. Each scenario demands a unique,
detailed response from emergency management functions and
activities (e.g., the specific EAL; the associated
protective actions; the consequence assessment calculations;
the procedures used; the medical situation encountered; the
specific content of news releases). Although the processes
for some general response functions/activities are
accomplished similarly for each type emergency (though the
detailed results are unique), the exercise evaluation does
not provide a comprehensive, overall assessment of the
adequacy of the emergency management program in responding
to a spectrum of emergency events or in maintaining the
readiness of the program over time.
Finally, the readiness of an emergency management program is
best estimated by an assessment of the combination of
complementary results from program and exercise evaluations
and, when available, post-emergency evaluations. However,
in order to perform these evaluations, performance-based
criteria must be developed to provide a means for
consistently judging the critical aspects of emergency
management program element performance. In the next
section, performance-based criteria for evaluating emergency
management program elements are introduced.
4.4.3 Evaluation Criteria
Generic performance-based criteria have been developed to
support the evaluation of specific emergency management
performance goals (Cf. Section 4.4.1) for each program
element of an Operational Emergency Hazardous Material
Program. The criteria are based generally on the
requirements of DOE O 151.1C, augmented with guidance from
the DOE G 151.1-series, and other Federal, State, and
general local requirements applicable to DOE emergency
management programs. Consistent with the performance goals,
the criteria are also intentionally generic and broad-based
to accommodate the diversity of hazards and response
capabilities associated with facilities/sites and activities
within the DOE/NNSA complex, and to provide a consistent
framework for judging emergency management performance
complex-wide.
Fifteen sets of criteria, grouped by program element, are
given in Appendix D of DOE G 151.1-3. Each criterion is
labeled to differentiate among those assessed during to
program (P) evaluations, exercise (E) evaluations, or both
(P/E). If program and exercise evaluations are to be
treated as complementary estimates of program readiness, it
follows that response elements must be judged using the same
performance goal and performance-based criteria (P/E).
In several response program elements, there are number of
instances where selected ongoing or programmatic activities
are included that maintain the response aspects of the
specific program element. For example, the activity that
ensures that ERO rosters are kept current with trained and
qualified responders is a routine function that maintains
the readiness of the responder organization. A routine test
and maintenance program for emergency facilities and
equipment maintains the readiness associated with the
Emergency Facilities and Equipment. These types of
activities should be considered part of a comprehensive
evaluation of programmatic activities during a program
evaluation. Criteria applicable to these programmatic
functions (P) are indicated as such in Appendix D.
Exercise evaluators should not use the criteria contained in
Appendix D as given. Program-specific expectations and
characteristics should be developed for each emergency
management program, based on facility/site- or activity-
specific hazards and associated program capabilities
(e.g., derived from existing plans and procedures). From
these attributes, generic criteria can be restated in the
context of the specific program. This facilitates the
evaluators task by bounding the general intent and scope of
the function or activity, as expressed in the generic
criteria, and focusing on the key program-specific
attributes incorporated in the revised criteria. In
contrast, during a program evaluation, the generic criteria
for programmatic and response elements are used as the
standard against which the plans/procedures and preparedness
activities are judged in the context of the facility/site-
or activity-specific hazards, associated program
capabilities, and commensurate with hazards concept. As a
result, the program-specific criteria for a program element
that reflect the hazards, plans, size of the ERO, and other
characteristics, may be a subset of the generic criteria
given in Appendix D.
4.4.4 Findings
Program and exercise evaluations consist of numerous
individual judgments related to the adequacy of projected or
demonstrated performance in specific emergency management
functional areas. These judgments are expressed in terms of
meeting or failing to meet expectations expressed by a
specific evaluation criterion. The necessary information
for making such judgments is obtained from numerous data
sources, including: document and record reviews, direct
observations, personnel interviews, selected testing of
personnel performance, and critiques.
Inadequate or failed actual or projected performance
identified during an evaluation is referred to as a finding.
A finding describes a failure related to a criterion.
Findings are ranked as either a Weakness or Deficiency to
reflect the significance of the failed criterion in
adversely impacting actual or projected performance
(i.e., OUTPUT) of the program element. A Deficiency has a
more significant impact on the program element OUTPUT than a
Weakness. The Deficiency also indicates a greater urgency
associated with implementing the appropriate corrective
action and fixing the problem than associated with a
Weakness.
For either a program or exercise evaluation, a Deficiency is
defined as follows:
A Deficiency is an actual or projected failure to meet an
evaluation criterion, thereby directly impacting the
associated basic emergency management activity, such as
planning, preparedness, readiness assurance, or response.
A direct impact on an emergency management activity
resulting from a failure to meet a single evaluation
criterion will, by itself, adversely affect the associated
performance-based OUTPUT of the program element. The
adverse effect of the failure should be significant and
readily apparent. For example, in Consequence Assessment (a
response element), failure to conduct a Timely Initial
Assessment (TIA) of the consequences of an emergency has a
direct impact on the response activity, since consequence
assessment results serve as a basis for the initial timely
decision-making following pre-planned (default) conservative
decisions. Failure of the Hazards Survey/EPHA to analyze
potential emergency events or conditions representing a
spectrum of severity has a direct impact on the planning
activity, because the Hazards Survey/EPHA serves as the
comprehensive planning basis for the emergency management
program.
For either a program or exercise evaluation, a Weakness is
defined as:
A Weakness is an actual or projected failure to meet an
evaluation criterion, thereby contributing to a direct
impact or indirectly impacting the associated basic
emergency management activity, such as planning,
preparedness, readiness assurance, or response.
A failure that contributes to a direct impact on or
indirectly impacts an emergency management activity results
from a failure to meet a single evaluation criterion that
adversely affects a supporting or auxiliary role in the
accomplishment of the emergency management activity. For
example, failure to maintain a controlled document system as
part of Program Administration (a programmatic element)
contributes to a direct impact on the planning activity,
because the latest procedures or hazards information might
not be available during a response. Failure to plan for
extended operations of the ERO (a response element) has an
indirect impact on the response activity, since members of
the ERO need respite from their assigned activities; fresh
personnel are more likely to make better decisions than
tired personnel.
When determining direct, contributing to direct, or indirect
impacts on emergency management activities, the following
should be considered. The impact in a specific functional
area is on the OUTPUT of the activity. The impact cannot be
judged on the actual or projected OUTCOME of the emergency
management programs response, since there are no measures to
realistically judge the OUTCOME during either a program or
exercise evaluation. A direct impact cannot be judged on an
unsupported evaluators prediction of an adverse direct
impact. A finding that represents an indirect impact based
on its intended function cannot become a direct impact
simply because the evaluator can conceive of a scenario that
explains how the failure could ultimately combine with other
failures to produce a direct adverse impact. Such
extrapolations simply cannot be supported by evidence
gathered during the evaluations.
Finally, the recommended approach [Cf. DOE G 151.1-3,
Chapter 3] for developing exercises that test emergency
response capabilities identifies a specific set of exercise
objectives to be achieved. The finding definitions should
be used only to address failure of a specific evaluation
criterion. However, although some exercise objectives may
correspond to just one evaluation criterion, a more likely
situation would have multiple criteria associated with an
exercise objective. In that case, the definitions of
Deficiency and Weakness should NOT be applied to the failure
of the exercise objective, but to each of the evaluation
criteria. When multiple evaluation criteria are reflected
in an exercise objective, then the failure of the objective
should be judged by the combined (rolled-up) assessment of
the total results for the objective, namely, those criteria
that were achieved versus those that were not.
Finally, findings will be directly associated with a
specific emergency management program element and hence will
point to problems specific to the element through the
characterization of the evaluation criterion. However, to
identify problems that are systemic across elements, it is
useful to determine the general characteristics of the
failure. In the following section, the format for labeling
or binning findings is presented in terms of generic faults.
Such a system can facilitate analyses of a collection(s) of
findings by emphasizing the generic (observed) fault that
may have caused the failure of diverse criteria across
program elements.
4.4.5 Generic Faults
In this section, a technique is presented for analyzing
findings across program elements or from multiple
evaluations. The technique is most appropriate for large
numbers of findings. Large sites with multiple facilities
or HQ offices with responsibilities over multiple sites and
facilities could potentially benefit most from the resulting
analyses. Smaller sites with limited numbers of facilities
and findings can usually analyze failures by a simple review
(i.e., reviewing/analyzing each individual finding and then
generalizing about cross-cutting faults.).
To assist in revealing systemic (i.e., cross-cutting)
problems common to multiple elements of a specific program
or even DOE complex-wide, findings from program and exercise
evaluations can be grouped using a generic fault system.
These generic faults characterize identified failures that
represent actual or projected failure to meet criteria. In
order to facilitate analyses, generic faults have been
developed to further clarify and refine the characterization
of specific failures that reflects as nearly as possible the
bases for the findings. The method provides the means for a
structured and comprehensive examination of findings both
within sites and DOE-wide.
Findings associated with diverse functional areas identified
during an evaluation can be further characterized by using
specific generic faults. For example, confusing or
inadequate emergency plans/procedures identified in several
response program elements during a program evaluation might
be represented by a generic fault associated with the
adequacy of plans and procedures. Evaluation of findings
for this specific emergency management program may indicate
the need for a re-look at how plans and procedures are
written. Similarly, if communication failures occur in
various response elements during an exercise evaluation,
then the assigned generic fault might be related to general
communication and information flow. In this case, an
analysis of multiple findings might reveal a general
weakness in the communication skills of the ERO members or
in the protocols used.
The following are candidate Generic Faults for use in
characterizing findings from evaluations:
1.Implementation: Failures related to the translation of
DOE requirements, policy or guidance into specific plans,
procedures and preparations for conducting effective
emergency management activities planning, preparedness,
readiness assurance, and emergency response. Site plans
and procedures do not make provision for meeting specific
Order requirements. Site procedures, documents, and
preparations do not exist for the effective
accomplishment of emergency actions as described in the
site Emergency Plan.
2.Performance: Failures related to the actions of staff in
carrying out preparedness or response actions. Personnel
fail to carry out an approved response procedure.
Failure to accomplish planning/preparedness activities
necessary to support effective response, if the
inadequacy is due to personnel performance and not to
some other cause, such as inadequate procedures or
resources.
3.Roles/Responsibilities: Failure of an assigned
individual to assume a specific role; an unclear, missing
or conflicting formal designations of responsibility for
carrying out emergency management functions; or, the
exercise of specific capacities or functions by assigned
personnel in an emergency response.
4.Interfaces/Coordination: Failures in links or
connections that are established between different
organizations to support emergency planning and
preparedness: inadequate cooperative/joint planning or
lack of agreement on how to respond to certain
circumstances. Failures might involve incompatible
terminology, uncoordinated response actions, disputed
authorities, or ineffective division of roles and
responsibilities.
5.Decision Making: Failures related to the exercise of
management-level decision-making authority for the
response process as a whole, or any part of it. Failure
to make sound decisions based on available information,
and to any procedures, processes, or criteria that would
lead to unsound decisions regarding important response
actions.
6.Control: Failures related to management-level direction
of response actions and allocation of resources,
continuity and effective use of authority and
responsibility.
7.Communication: Failures in the act of transfer of
information between response facilities, personnel, and
organizations as part of emergency response; failures or
inadequacies in the communication of facts, status,
decisions, directions or actions. Not including
communication equipment failures.
8.Plans and Procedures: Failures related to plans and
procedures, in hard copy or electronic form, which
provide direction to planning, preparedness and response
activities. The plans or procedures do exist, but are
found to have factual or editorial errors that would
degrade the effectiveness of response, are not internally
consistent, or are not current.
9.Documentation: Failures related to reports, agreements,
and other documents, excluding Plans/Procedures and
Records, in hard copy or electronic form, which provide
technical planning basis and support/direction to
planning, preparedness and response activities. EPHAs,
training materials, lesson plans, and training manuals;
understandings and agreements; protocols
(e.g., communication, medical, control); or other
emergency management documents are found to have factual
or editorial errors that would degrade their
effectiveness, be internally inconsistent, or not
maintained current.
10. Records: Failures related to logs, event
chronologies and data sheets generated during emergency
response or exercises; medical records; equipment and
facility test and maintenance records, staff training and
qualification records; or documentation of meetings,
planning decisions, or any other past events/ occurrences
that form part of the planning basis, in hard copy or
electronic form. Factual errors degrade the
effectiveness of the recorded data, the recorded
information is internally inconsistent, or the record is
not maintained current during the time frame/activity
required.
11. Personnel: Failures related to the number,
qualifications, or capabilities of staff performing
planning, preparedness, readiness assurance, and response
functions. Failures reflect lack of qualified staff to
effectively carry out a function, or the assignment of
personnel with inappropriate skills, training, or
qualifications for the job.
12. Equipment: Failures related to the physical and
other assets available to support the emergency
management function: deficiencies or failures in
dedicated response equipment, such as radios, radiation
survey instruments, computers, protective equipment, or
facilities from which to conduct emergency response
activities.
13. Backup (Alternate): Failures related to the
secondary or backup facilities, equipment, staff, or
other resources/capabilities for response: lack of or
inadequate backup for any key person, facility or
resource; failure to adequately train backup personnel or
to maintain the alternate response facilities in a usable
state.
These generic faults should be clearly distinguished from
root causes. The root cause is the basic reason for the
criterion failure for which no underlying cause could be
identified. The root causes generally reflect inadequate
performance of programmatic activities, such as training or
program administration, which maintain the program. In
contrast, the generic fault is the further characterization
of the observed failure; there will likely be a root cause
for identified generic faults.
As a practical limit, the evaluator can assign up to three
faults to each finding, as necessary. The assignment of
multiple faults can represent situations, such as:
Ambiguity or uncertainty in characterizing the failure
Multiple generic faults are identified
Primary and secondary (and maybe tertiary) generic
fault(s) are identified
The selection of the appropriate fault(s) should not
represent a major task. If the finding description or
summary is written clearly and contains complete information
for interpreting and characterizing the situation, selected
faults(s) should be readily apparent and referred to within
the finding description itself. For example, documents to
be used for response are found to be old versions. The
evaluator must distinguish between Implementation (#1) and
Performance (#2). If an adequate document control system
exists, and it is not followed, then Performance applies.
On the other hand, if no system is in place, then
Implementation is appropriate. The finding descriptions
must certainly contain sufficient information to distinguish
between the two possibilities, since different corrective
actions may be required.
4.4.6 Program Readiness
An emergency management program consists of diverse
activities, functions, and tools that operate either in a
periodic, ongoing mode to ensure the readiness of the
program to respond (through planning, preparedness,
readiness assurance) or in a standby mode to respond to an
emergency when called upon. A comprehensive estimate of
overall program readiness to respond to an emergency cannot
be obtained from a simple (linear) combination of evaluation
results related to the individual program elements
(e.g., average of all program element performance
estimates). A true determination of the readiness of the
overall emergency management program must account for the
relationships between the technical planning basis,
programmatic and response elements and on the relative
contribution of each type to the estimate. Only by
combining applicable programmatic element evaluations with
response element evaluations can an estimate be obtained
from program evaluations complementary to observed
performance during an exercise. It is the method for
combining/integrating programmatic and response element
evaluation results that will determine the structure and
methodology for obtaining overall readiness estimates.
Finally, until a practical framework for integrating the
results within and between each type of evaluation is
developed, program and exercise evaluations will necessarily
represent standalone estimates of program readiness, each
with its own attributes and limitations.
4.5 Evaluation Process
Evaluations of DOE/NNSA emergency management programs can be
characterized by the organization that sponsors the
evaluation and the relationship of the evaluators to the
development and maintenance of the program. Evaluators who
are neither responsible for the development or maintenance
of the program nor associated with the managing organization
conduct an external evaluation. An internal evaluation or
self-assessment is an evaluation sponsored by an
organization to review and evaluate its own emergency
program.
The process of evaluating emergency management in a program
evaluation, either external evaluation or self-assessment,
can involve examination, analysis, and evaluation in the
following areas:
Plans and procedures
Administrative control mechanisms
Planning basis and supporting analyses
Response tools, such as computer models, monitoring
equipment, communication systems
Resources and resource availability
Organizations and organizational interfaces
Training and training validation
Exercise program
Data sources include plans and procedures, other documents
[e.g., EPHAs; Memoranda of Understanding (MOUs)], databases
and records, limited scope tests/drills, and interviews.
The more thoroughly these data sources are examined and
analyzed, the more comprehensive is the evaluation.
An exercise evaluation, on the other hand, involves
observation, analysis and evaluation of the demonstrated
performance of integrated response capabilities during a
simulated emergency event. Observations of activities
included involve the ERO staff, utilization of facilities,
equipment, and procedures. The scope of the evaluation,
including the selection of the emergency management program
elements to be observed, is determined by specific
objectives developed for the particular exercise. Exercise
evaluation also addresses the overall conduct and control of
the exercise. This evaluation is based on exercise
documentation, including the scenario and objectives, and
the actual conduct of the exercise. Also included in the
evaluation are the conduct of ancillary activities, such as
controller and evaluator training, responder training, and
the exercise critique. The validity of the exercise as a
viable test of readiness should be a conclusion of this
evaluation.
The process for conducting internal and external evaluations
can differ substantially in areas such as scheduling,
preparation, and interactions with the evaluated
organization both during and after the evaluation. Since
the process used for both types of evaluations will be
determined by the evaluating organizations, this guide does
not address details related to organization-specific
logistics and coordination issues. However, some common
features of evaluations are discussed in the following
sections, namely, data/ information sources and the expected
output from evaluations.
4.5.1 Data Sources
Successful evaluations depend on the availability of diverse
sources of information. Choosing the appropriate sources
will vary depending on whether the team is conducting a
program or exercise evaluation. These sources can include
document and records reviews, interviews, and observation of
performance. [Some data sources for exercise evaluations
(e.g., hot wash, post-exercise critiques) are discussed in
Chapter 3.]
1.Document Reviews. Document reviews for program
evaluations include: analyses such as Hazards Surveys,
EPHAs, Safety Analysis Reports (SARs)/Safety Assessment
Documents(SADs)/Documented Safety Analyses (DSAs), and
Environmental Impact Statements (EISs); plans such as
emergency plans, building plans, spill prevention plans,
Resource Conservation and Recovery Act (RCRA) plans, and
training plans; procedures such as administrative
procedures for emergency management and Emergency Plan
Implementing Procedures (EPIPs); and agreements, such as
MOU and Memoranda of Agreement (MOA); and contracts.
Additionally, evaluators should be knowledgeable of
organizational structure, functional responsibilities,
and previous evaluations.
An exercise evaluation requires a more limited
facility/site or activity document review to provide the
team with sufficient familiarity with the facility/site
layout, activities, emergency management program plans
and procedures, response capabilities, and hazards. A
more extensive review effort is involved in an evaluation
of the planning documentation associated with the
exercise. The exercise package should contain sufficient
information for the conduct, control, and evaluation of
the exercise, including: exercise objectives, scope,
participants, simulations, time lines, injects, technical
data, safety/security provisions, controller
instructions, and evaluation criteria.
2.Records Reviews. Records reviews during program
evaluations could include the following:
Training records
Drill and exercise reports
Hazardous materials inventories
Corrective action tracking
Facility and equipment inventories
Systems tests such as notification, alerting,
monitoring and communications tests
Incident and event reports
Data and records review following exercise evaluations
can also provide supplementary supporting information
necessary to address exercise findings, so that an
implicit recommendation can be included in the write-up.
3.Interviews. Interviews during program evaluations should
be conducted after the initial document review, when the
program functions have been identified. Interviewees
should include the managers of the emergency management
staff, the emergency management staff itself, the ERO
(including back shift personnel), support organizations
such as training, industrial hygiene, health physics,
operations and maintenance, and offsite interface
organization representatives.
Evaluators should conduct interviews with a variety of
individuals with different backgrounds and at different
levels in the organization. Team members should talk to
operators, mechanics, and all levels of supervision as
necessary to get a thorough picture.
Interviews can elicit meaningful information concerning:
Personnel qualifications, training and drill
participation
Familiarity with procedures and equipment
ERO activation
Problems encountered during response
Adequacy of procedures and resources
Careful planning and conduct of the following steps in
advance of each interview will enhance their
effectiveness:
Delineate the topics to be discussed
Identify the types of questions to be asked
Determine what reference materials should be available
for the interview
Identify the best location to hold the interview; if
possible, perform the interview in the interviewee's
workplace so that the resources needed will be readily
available
Anticipate the interviewees perspective, possible
concerns, and expectations
Know the procedures and hazards that the interviewee
should know and understand
Determine how to establish a climate of trust and
collaboration and, hence, how to avoid an adversarial
encounter
Facilitate the interview process to ensure that
information needed is obtained and that the dialogue
continues after the interview, as required
Plan to follow through if information is promised for a
later time
Thank people for their time
The following techniques may facilitate the interview
process:
Setup the interview by introducing yourself, stating
the purpose and context of the interview, outlining the
topics to be covered, assuring confidentiality and
establishing your credibility
Ask open questions - why, when, how, who, where, and
what
Ask hypothetical questions
Observe and listen closely to the person being
interviewed
Show interest and respect and dont be critical
Restate the question and answer to ensure
understanding;
Inform interviewees that notes will be taken of topics
discussed
Verify and evaluate all information by requesting
demonstrations of procedures or equipment, soliciting
identical information from more that one person, and
checking records to verify claims
Include task performance testing of interviewees ERO
position with hypothetical scenarios; validate
hypothetical scenarios with knowledgeable trusted agent
and request confidentiality of subject matter until
evaluation is complete
Separate levels of supervision when interviewing; do
not have the interviewees supervisor present during
the interview
Do not make recommendations initially; determine the
facts first to ensure a clear understanding of the
problem
Ask for copies of documents shown or referenced by the
interviewee
Do not respond positively or negatively to answers
given, through either comments or facial expressions;
instead use clues to develop further questions
Hold your opinion during the course of the interview
and avoid arguing
For exercise evaluations, interviews are limited to
before or after the exercise, for logistic information or
follow-up questions. During exercises, team members
should refrain from interfacing with responders, since
interrupting or prompting the responders can prevent
their decisions or actions. Prior to exercises, team
members may interview personnel to gain further knowledge
and/or clarification of responder duties, procedures, and
equipment capabilities. After the exercise, responders
may need to be interviewed by evaluators to clarify their
response activities and observed performance.
4.Observations. When possible, observations should be made
during both program and exercise evaluations. Program
evaluations may include observations during the
following:
Individual skill/knowledge tests
Training sessions
Facility walk-downs
Scheduled drills
Exercise evaluations may include observations of the
following:
Responder briefings
Controller, evaluator, and/or specialized training
Application of response capabilities
Control of the exercise
Exercise hot-wash
Exercise critiques
To maximize the effectiveness of data collection from
observation of performance during an exercise, utilize
the following practices:
Review the applicable procedures to guide observations
of the planned performance required of the personnel.
Before the action begins, find a location where key
actions and communications can be seen and heard.
Record the time of each observation.
Observe whether exercise players have been pre-
positioned.
Note any impediments encountered by the players while
they are accomplishing their tasks.
Note significant deviations from the exercise time
line.
Note controller/evaluator effectiveness.
Observe and take note of key response activities,
including: ERO notification and mobilization; procedure
usage; facility and equipment availability and usage;
instrument calibrations; communications; decision-
making; notifications; record keeping; staffing of
functions; support requests; briefings; and key
scenario times
Additional opportunities for observation may include:
Observations of backshift operations to ascertain
staffing and resource availability, training and
responsibilities of backshift personnel, and general
capabilities when management is not generally
available.
Facility walk-downs to ascertain emergency response
facility characteristics (communications equipment,
power supplies, workspace, references, computer
systems, ingress-egress and habitability monitoring,
etc. Walk-downs of hazard facilities are also useful
to review and validate information on inventory,
storage, barriers in relation to EPHAs and EAL
procedures.
Demonstration drills may be requested in advance by
evaluators to observe team performance and validate
what actions are depicted in emergency response
procedures.
Scheduled training may be observed to review the actual
content of training and instructor capability.
Equipment demonstrations can provide evaluators
observations of equipment storage, security,
availability, operability, maintenance/ calibration,
operational procedures, personnel knowledge and skills.
Equipment observations may include: field monitoring
equipment readiness and use, communications systems,
hazardous release plume modeling and tracking systems,
notification systems, meteorological equipment,
decontamination equipment, medical emergency response
equipment.
5.Data Selection Techniques. A common problem associated
with evaluations is the normally limited time available
for collecting, reviewing, and evaluating information. A
number of data selection techniques can be employed to
perform a reasonably comprehensive evaluation within the
available time constraint. These selection techniques
include: sampling and horizontal-to-vertical review.
Sampling. The technique of data sampling is essential
to maximizing the evaluation effectiveness within the
time constraints and available resources. During a
program evaluation, team members cannot examine all
activities, operations, processes, documents, and
records for assigned functional areas. A selected
number of activities, processes, or operations should
be chosen to represent the whole. The selected
examples should be chosen at random to avoid
concentration on a particular group or time. Since
accuracy increases with sample size, multiple examples
for sampling should be examined, keeping within the
time allotted, resources, and reasonableness. Random
sampling may be used in any of the information
collection activities mentioned earlier, for example:
Training records
Procedures for review
ERO members to interview
Weekly communications checks
Exercise scenarios to review
Equipment maintenance records
Horizontal-to-Vertical Review. The horizontal-to-
vertical information review methodology provides a
means for investigating a specific emergency management
component or element in a logical and efficient manner,
without delving into details unnecessarily.
The technique applies primarily to program evaluations
and involves a step-by-step general review of materials
related to an area or activity to be evaluated. It may
begin with a general top-level review of an emergency
management element. If a potential problem is
discovered during the general review (horizontal), then
the evaluator can refocus the review effort on the
supporting details (vertical) in an attempt to arrive
at the underlying problem. The technique is a way to
cover major aspects of a program, and, on a selected
basis, examine supporting details to uncover potential
problems or to verify the status of an activity.
6.Field Notes. Field notes are a critical component of the
evaluation process that provides the mechanism for
documenting an observation or issue as soon as possible
after it is made or identified. Documentation should
contain sufficient detail to support later development of
findings and verification of issues or concerns. These
field observations should be recorded in an informal
notebook.
Names and titles of DOE and contractor personnel
interviewed, with times, dates and topics discussed
should be included in the notes, as well as the
observations made during a walk-down. Team member
entries should organize information collected, formulate
tentative findings, and reveal the nature of missing
information needed to resolve outstanding issues. These
serve as an excellent source of thoughts in the
preparation of the draft report. Under normal
conditions, field notes should not be released to
facility personnel being evaluated. These notes are for
the use of team members only, unless otherwise stipulated
by the team leader. Additionally, during exercises,
evaluators should keep a time line record, drill and
exercise observations, and critique information.
7.Limited Scope Tests and Evaluations. LSPTs and tabletop
drills are scenario-based discussions between evaluators
and interviewees that can be an effective method of
observing performance when there is no opportunity to
observe a drill or exercise. A pre-developed scenario is
used as the basis for discussion of the interviewees
role, responsibilities, and interfaces within the context
of the given emergency scenario. Several interviewees
can be interviewed and evaluated simultaneously in a
tabletop drill, while a LSPT generally involves one
emergency response position.
These observation methods are opportunities to observe
decision-making capabilities, use of procedures, and
interactions among ERO positions. However, because of
the complexity of the method, evaluators need to be very
well prepared, knowledgeable, and experienced.
Additionally, assistance will be needed from the
organization being evaluated in preparing scenarios and
providing a trusted agent to assist in setup and conduct
of the observations.
Preparation of a brief, simple and realistic scenario and
clear scenario objectives is necessary. The entire
session/interview should be executable in about 45 to 60
minutes. Objectives and scenario details must be
communicated clearly to avoid confusion. Clock time
should be observed for time sensitive activities, such as
event classification, notification, and protective action
decision-making. Responders should be allowed to talk
through their actions with minimal interference. Leading
or coaching should only be permitted at a level
commensurate with the procedures and personnel normally
present to assist responders.
The evaluator should maintain a timeline when conducting
these observations and document actions taken and
decisions made based on the scenario. Procedures and job
aids used for performance should be documented. Results
should be compared against the established objectives for
the session and strengths and weaknesses should be
documented. Time should be allowed for clarification and
constructive critique at the end.
4.5.2 Evaluation Results
Based on the analysis and evaluation of the information
collected from the data sources reviewed, an evaluation
should yield at least findings and corrective actions, as
required by the observed or projected performance. Other
judgments such as Improvement Items, Superior Performance,
and Noteworthy Practices are optional. An AAR should be
produced for all evaluations to provide supporting
documentation for the evaluation activity and the resulting
findings and program improvements.
Findings. An evaluation consists of a judgment of the
adequacy of demonstrated or projected performance in
specific functional areas and activities, developing
specific response or programmatic products, and utilizing
the appropriate equipment, facilities, and tools. This
performance is compared with a single evaluation criterion
or a selected subset of criteria to determine its adequacy.
Findings are then used to express these judgments resulting
from either program or exercise evaluations by identifying
the inadequacies in demonstrated or projected performance
and characterizing their significance. These finding
categories are Deficiency and Weakness, in order of
decreasing significance. Section 4.4 of DOE G 151.1-3
contains definitions of the two finding categories along
with guidance for their use in program and exercise
evaluations.
An essential aspect of evaluation results is a concise,
clear, and complete description of each particular finding.
This description must detail explicitly the situation,
circumstances, and special considerations and constraints
that are needed to characterize the finding and identify the
specific underlying fault. In addition, the finding
description should be sufficiently explicit so that a
recommended solution, if determined by the evaluator, is
clearly expressed and readily apparent in the finding text.
To ensure that the evaluation findings are accurate and
consistent with the observations and perceptions of multiple
evaluation teams, draft findings should be reviewed and
validated by all team members in an open discussion in order
to achieve consensus. Such a validation process should
resolve any identified discrepancies. This process is an
essential step in an evaluation process to be accomplished
prior to the preparation of the report.
Improvement Items. During an evaluation, an evaluator may
note situations where, while specific criteria are being met
and the performance objective for a particular program
element is being achieved, the performance of the evaluated
organization could be improved or made more efficient if
they were to adopt standard DOE or industry practices.
While the evaluator will have discussed the specifics with
representatives from the evaluated organization, these
recommendations are included in the report (in an appendix
is acceptable) in sufficient detail to allow interested
parties and subsequent evaluators to understand the
situation and justification. It is sometimes useful to
prepare Improvement Items in formats similar to that used
for findings, since management at the facility/site- or
activity-level may choose to track subsequent actions with
the Corrective Action Tracking Systems (CATS) used for
findings.
Superior Performance. Superior Performance denotes observed
site performance or documented practice that
demonstrates/implies excellence in a specific aspect of an
emergency management activity. They are not necessarily
recommended for other DOE/NNSA sites across the complex.
Examples of Superior Performance need not be developed for
every facility/site or activity and, therefore, the absence
of specific instances of Superior Performance does not
reflect shortcomings on their part.
Noteworthy Practices. When an evaluated organization has
demonstrated a superior and unique approach, technique,
product, tool, etc., this may be documented as a Noteworthy
Practice. Noteworthy Practices are actions worthy of being
emulated by other DOE/NNSA facilities/sites or activities.
They should be described in sufficient detail so that
interested parties and subsequent reviewers can understand
the justification for the designation and can modify the
practice to suit their particular situation.
Corrective Actions. Evaluations and the resulting findings
would be of no benefit to an emergency management program if
identified problems were not addressed through corrective
actions. The corrective action responding to each finding
is developed by the evaluated organization. To ensure that
the problems identified will be corrected in an effective
and timely manner, the evaluated organization should produce
the following:
Clear statement of the finding
Statement of the cause of the identified problem area
Details of the actions needed to eliminate the cause
Responsibility for corrective action
Schedule for completion of corrective action
4.6 Self-Assessments
A self-assessment can be viewed as an evaluation (program or
exercise) of an emergency management program performed
and/or sponsored by the organization itself or by some level
of the management of the organization responsible for
evaluating the emergency management program. Any level of
the DOE organizational structure can perform a self-
assessment, including personnel who are:
Involved in the emergency management program and directly
responsible for activities or program components that
they are assigned to evaluate
Involved in the emergency management program, but not
directly responsible for activities or program components
that they are assigned to evaluate
Within the sponsoring organization, but without a direct
relationship to the emergency management activities
(e.g., Quality Assurance Department)
With no direct connection or relationship to the program
or the sponsoring organization
Personnel performing self-assessments should be qualified
and trained in audits or evaluations, and, if possible,
should be at most indirectly associated with the specific
activities or emergency management program components they
evaluate. This is particularly important for an evaluation
of analyses (e.g., EPHA) related to the program bases and
analytical tools. While the author of such analyses can
certainly verify the required contents and end results, an
objective view by another analyst is the most efficient way
to evaluate their validity. The author may be too close to
the analysis to judge it objectively. This discussion is
not intended to preclude an author evaluating his own work,
but merely to point out possible limitations associated with
that approach. In areas other than technical analyses, the
evaluations conducted by directly involved personnel are
more likely to yield objective evaluation results consistent
with evaluations conducted by the other groups mentioned
above.
Self-assessments of emergency management programs should be
based on the standard performance evaluation criteria listed
in Appendix D of DOE G 151.1-3. Self-assessments may also
include compliance with external non-DOE codes and
regulations and, similarly, with internal organizational
requirements and commitments. The methodology associated
with self-assessments is very similar to that applied for
other forms of evaluations. The data- and information-
gathering and evaluation techniques will be substantially
the same.
Self-assessments can contribute significantly to enhancing a
program's effectiveness through:
Verifying program status on a periodic basis, for the
benefit of emergency management staff and management
Identifying program and performance weaknesses and
negative trends
Providing program-specific budgetary/staffing planning
basis
Preparing for an evaluation by an external sponsor
Secondary benefits of self-assessments, especially those
performed by members of the emergency management program
themselves, include:
Reinforcing confidence of staff/management in the ability
of the emergency management program to respond
Providing emergency management staff with the opportunity
to view the integrated program
Using a consistent set of performance evaluation criteria,
as given in Appendix D, and the same definitions for
expressing the significance of findings will simplify the
process of combining self-assessment results with the
results of other evaluations. Placing self-assessments and
external evaluation results on the same performance-based
scale can accomplish the following:
Validate self-assessment results and reinforce confidence
in the process;
Facilitate the identification of similar problems and
possible solutions from across the DOE complex; and
Track progress in improving and maintaining program
readiness using all available performance evaluation
data.
The internal program evaluation can be the primary focus,
but not the only part of a self-assessment program. It
provides the opportunity to examine the results of other
assessment activities throughout the year. Internal
evaluations should be planned and organized to include and
focus on those areas that the organization most critically
needs to test and evaluate each year. Emergency managers
should consider the following in determining the scope and
focus of self-assessments:
Results of evaluations from the previous year
Results of performance tests over the past year
(exercises, drills, etc.)
Results of response to actual events
The scope of evaluations from the previous years,
ensuring that areas that were not evaluated thoroughly
receive greater scrutiny
Progress in making corrective actions previously
identified
Weaknesses identified from other self-assessment
activities
Achievement of performance measures set for the
organizations emergency management program
Improvements that the organization is committed to making
As resources for external evaluation activities decrease,
more emphasis will be placed on the use of self-assessments
for assuring that the readiness of the emergency response
capabilities is maintained at a satisfactory level. A
reasonable and consistent approach for conducting these
internal evaluations is to follow the guidance suggested for
external evaluations using consistent evaluation criteria
and the same definitions for severity of findings. This
simplifies the process of combining the results of internal
program evaluations with exercise results, and combining
self-assessments with external evaluations.
Finally, Appendix E presents a discussion of a systematic
approach that views self-assessment as a continuous
monitoring and evaluation process. This approach provides
the emergency management program with a flow of data over
the course of a year by which program maintenance activities
(i.e., programmatic activities) can be monitored for
indications of potential performance lapses. This
continuous process maintains the concepts of evaluation and
improvement as a vital component of the emergency management
program.
4.7 Post-Emergency Evaluation
A Post-Emergency Evaluation is a critique of response to an
actual emergency event or condition at any DOE facility/site
or activity. In addition to being required by DOE O 151.1C,
it is required by DOE Order 5480.19 and 29 CFR 1910.120.
Emergency management specialists should accompany the team
assigned to critique the response to the event (e.g., a team
conducting an accident investigation required by
DOE O 225.1A; Type A or B Investigation). It is important
to emphasize that as a member of an accident investigation
team, the team leader has the responsibility to determine
the procedures and protocols involved in the conduct of the
investigation. Hence, as a member of the team, an emergency
management specialist must follow the direction of the team
leader.
This section is intended to give emergency management
personnel some preliminary guidance on a general approach to
the emergency management investigation related to a
response. The evaluation of the emergency response to an
actual event should include the following steps:
Step 1. Collect and Review Data
Collect and review shift logs
List personnel on shift
Review emergency communications in logbooks
Review ERO notification and activation times
Review sequence of response to emergency
Review press releases
Review recorded operating parameters of equipment
Step 2. Reconstruct sequence of events from data.
Step 3. Conduct interviews to validate/complete
event/response description.
Have two to three persons on the interview team
representing operations and the safety organization
Compare the interview sequence of events with logs to
verify the sequence and clarify discrepancies
Involve emergency response personnel as appropriate.
Include offsite response personnel involved with the
response.
Step 4. Several After-action, or follow-up, steps should be
accomplished as follows:
Draft a report on the reconstructed sequence of events.
Identify report items by source of information
(i.e., log, interview, work request, work permit, and
inspection team observations.) Have emergency response
personnel review the draft.
Conduct a tabletop response. Conduct a team tabletop
response to the event utilizing site procedures and
recording the actions that should have been performed
were the procedures followed. Compare against the
reconstructed sequence of events.
Analyze events, decisions, and response actions. Use the
evaluation criteria given in Appendix D to focus the
analyses of the response on the generic components of the
emergency management elements. The results of the post-
emergency analyses can be structured similarly and will
be compatible with exercise evaluation results.
Identify areas needing corrective action or improvement
(i.e., findings). Emergency response findings and
corrective actions should be identified. Implementing
the corrective actions is the responsibility of the
organization.
More detailed guidance for structuring an evaluation of an
actual response to an accident is available elsewhere. This
limited description is intended to support the assertion
that such a post-emergency evaluation is similar to an
exercise evaluation. Hence, the evaluation of exercises
and, likewise, the response during an actual emergency
should be based on the same evaluation criteria. However,
during an actual emergency, objectives selected for
evaluation will correspond to the response needs of the
specific emergency, while, for an exercise, objectives will
be selected to focus the exercise performance on specific
aspects of the program to be tested.
4.8 Performance Indicators
As emphasized previously, neither a program evaluation nor
an exercise evaluation provides a single, standalone measure
of emergency management program readiness. A reliable and
comprehensive estimate of the readiness of an emergency
management program must be based on an integrated assessment
of both quantitative programmatic data and subjective
evaluations of the various functions, activities, products,
analyses, tools, etc., that contribute to the
implementation, maintenance, improvement, and execution of
emergency response capabilities. For the estimate to be
meaningful, the contributions must be reasonably
comprehensive and reflect the emergency management program
within a somewhat narrow time frame.
Realistically, however, the contributions to such a
comprehensive assessment may take a number of years to
accumulate and, as a result, could reflect data and
evaluations from different stages of a program. This would
be especially true for large sites with extensive programs,
where substantial resources, planning effort, and time would
be involved in acquiring the necessary data, resulting in a
significant interval between valid estimates of readiness
for the full program. To avoid such potential
inconsistencies and delays in assessing and tracking program
readiness, an approach using specific programmatic data
and/or evaluation results in selected functional areas can
provide timely performance indicators to reflect discrete
aspects of program readiness over the broad scope of the
program.
Performance indicators for emergency management will consist
of a set of critical indicators or vital signs derived
from programmatic data and/or evaluation results to track
the readiness of the program. Similar to physiological
vital signs, such as temperature, blood pressure, heart
rate, glucose level, cholesterol levels, etc., used by
physicians to track patient health, critical performance
indicators can be developed to focus on specific aspects of
selected emergency management activities to measure the
health of the program. In the case of the physiological
vital signs, each by itself provides only a single
indication of a potential health threatening condition,
which leads the physician to order further tests and
procedures. Similarly, emergency management indicators only
tell a part of the story and further investigation is
required in order to determine if a problem actually exists
and, if it does, to provide further diagnosis.
The discussions that follow address two performance
indicators: performance metrics and performance measures. A
performance metric is a single parameter that reflects
performance in terms of its absolute value (e.g., percent
ERO personnel trained). A performance measure reflects
performance in terms of the value of a metric relative to a
pre-assigned goal (e.g., percent ERO personnel trained
relative to a goal of 90%). The performance metric is used
to follow or trend a value that characterizes a given
activity or function. A performance metric can be used when
a performance goal is unavailable or not meaningful for use
as the baseline for a performance measure. A practical
means for selecting an acceptable and achievable goal for
the performance measure is by tracking the values of a
performance metric over time, and selecting a candidate
value that approximates an acceptable goal. The goal can be
modified as more performance data is collected over time.
Performance indicators are usually discussed in terms of
deterministic processes that produce products or services
with identifiable characteristics that meet the needs of the
customer. Whether the product meets the specific
requirements is either directly apparent or becomes apparent
through deterministic testing. In contrast, the products
of emergency management are not necessarily objects that can
be directly observed and whose characteristics are readily
apparent and well defined. Actual tests (i.e., actual
emergencies, exercises) are not comprehensive, involve a
snapshot of performance in response to a single event
scenario, test only selected functions, and usually involve
only one cadre of the ERO. Hence, the ongoing activities
that maintain the program (i.e., programmatic activities)
must also be evaluated to provide assurances that the
program satisfies the comprehensive objectives associated
with the range of hazards on the facility/site or associated
with an activity and is maintained at a level of readiness
to protect workers, the public, and the environment.
The remainder of this section focuses on the types and
general characteristics of performance indicators and how
they relate to emergency management. This is followed by a
more detailed discussion of performance indicators specific
to emergency management programs.
4.8.1 General Characteristics of Performance Indicators
This section reviews general characteristics of performance
indicators as covered in literature focused on performance-
based management of organizations that deliver products or
services on a continuous basis [Cf. How to Measure
Performance. A Handbook of Techniques and Tools, prepared
by the TRADE Performance-Based Management Special Interest
Group (PBM-SIG), October 1995]. This fundamental
characteristic is in contrast to the standby nature of the
response elements of an emergency management program.
However, some general characteristics of performance
indicators shared by continuous and stand-by systems are
addressed in the following discussion.
Performance indicators reflect critical quantitative or
qualitative performance characteristics of products,
services, and processes. They represent a useful tool for
helping to understand, manage, and improve how organizations
(including, processes, programs, etc.) perform. In general,
performance indicators can reflect:
How well the organization is doing
If it is meeting its goals
If customers are satisfied
If processes are in statistical control
If and where improvements are necessary
A logical selection of performance indicators can provide an
organization with the information necessary to make
intelligent decisions related to achieving the
organizations mission and goals.
Most performance indicators can be grouped into one of six
general categories of performance characteristics.
EFFECTIVENESS: Process characteristic indicating the
degree to which the process output (work product)
conforms to requirements. [Are we doing the right
things?]
EFFICIENCY: Process characteristic indicating the degree
to which the process produces the required output at a
minimum resource cost. [Are we doing things right?]
TIMELINESS: Measures whether a unit of work was done
correctly and on time. Criteria must be established to
define what constitutes timeliness for a given unit of
work. Criteria are usually based on customer
requirements.
QUALITY: Degree to which a product or service meets
customer requirements and expectations.
PRODUCTIVITY: Value added by the process divided by the
value of the labor and capital consumed.
SAFETY: Measures the overall health of the organization
and the working environment of its employees.
Emergency management performance indicators will generally
reflect the first three performance characteristics:
Effectiveness, Efficiency, and Timeliness. Quality may
describe performance in specific functional areas of the
emergency management program, such as training, offsite
interfaces, or public information.
Performance indicators should reflect implementation
characteristics such as the following:
Use measurable data
Are understandable, practical
Allow uniform interpretation
Are normalized for bench marking
Apply broadly
Provide basis for continual self-assessment
Are compatible with existing sensors
Are an agreed-upon basis for decision-making; are
accepted and have owners
Are precise in interpreting the results; reflects
results, not activities used to produce results
Relate directly to a performance goal
It should be noted that several of the characteristics
listed above might not apply to performance indicators for
some organizations or programs because of the uniqueness of
their missions or the availability of data for constructing
performance indicators. Specifically, some performance
indicators for emergency management may not satisfy the
first and the last two characteristics, if they are
subjective evaluation results rather than measurable
parameters or they reflect activities rather than results of
activities, respectively. These deviations from the usual,
expected characteristics of performance indicators result
from the standby nature of emergency management programs
and the inability to accurately test the response without an
actual emergency. This forces the dependence on
programmatic data and evaluation results, rather than actual
emergency response, to estimate readiness to respond.
Performance indicators can also be characterized by what is
measured. A process can be divided into four steps: input,
activity, output, and outcome. The first three are
measurable in a continuous process, and the fourth is
sometimes measurable. This characterization leads to four
categories of measurement data:
INPUT-Based: what the process needs to perform
ACTIVITY-Based: what is done with the input
OUTPUT-Based: what the activity produces
OUTCOME-Based: what is the ultimate use/benefit/impact
of the output
Note that the data used in the performance indicator
represents a graded scale from basic INPUT to a process, to
the ACTIVITIES that use the INPUT, to the OUTPUT of the
ACTIVITIES, and, finally, to the intended OUTCOME of
delivering the OUTPUT to the customer. In other words, the
data used in indicators can vary from the most basic, the
INPUT, to the final result, the OUTCOME.
Emergency management performance indicators can be
characterized by all of the data types given above.
However, the last category is difficult to measure, since
the emergency management program performs or exhibits its
desired OUTCOME, protecting people, only in response to an
actual emergency. The infrequency of emergencies
necessarily limits the availability of OUTCOME indicator
results for estimating program readiness.
This characterization of performance indicators, based on
type of input data, implies that the more directly OUTCOME
can be measured, the more reliably the indicator reflects
program readiness. A performance indicator that uses
program INPUT data (e.g., budget or staff) is a weaker
indicator of program readiness than ACTIVITY
(e.g., training) or OUTPUT (i.e., trained ERO personnel)
performance indicators. INPUT-based indicators, although
clearly important, contain little direct, useful
information, by themselves, that will reflect how the
program will ultimately perform. The INPUT-based indicators
are the most basic/primitive. Hence, as the data used in
the indicators varies from INPUT, to ACTIVITY, and to OUTPUT
data, the more accurately the indicator approaches a measure
of readiness to achieve the desired OUTCOME of emergency
response.
Performance indicators can also reflect the evolution of an
organization or program. A different set of performance
indicators might reflect performance for each phase in its
development (e.g., concept development, design,
implementation, start-up, testing, steady-state maintenance,
etc.) In addition, performance indicators may also provide
indicators of program status or actual or predicted
performance in different time frames, within a given phase
of the organization or program, for example:
Trailing Indicators - Measure past performance
Current Indicators - Measure current status
Leading Indicators - Forecast future performance.
4.8.2 Emergency Management Performance Indicators
Performance indicators for an emergency management program
can be grouped into three categories: PROGRAMMATIC,
RESPONSE, and ORGANIZATIONAL. The first two categories
reflect the functions and activities associated with the
elements of an emergency management program. The third
includes organizational factors that may influence or
reflect the overall performance of the program as a whole.
PROGRAMMATIC indicators are associated with the programmatic
emergency management elements and associated functions and
activities listed in Section 4.4. These indicators will
reflect program status and ongoing preparedness
activities. Most of the PROGRAMMATIC indicators will be
quantitative parameters that will characterize the current
status of the program (e.g., plans, procedures, analyses
completed or reviewed, corrective actions implemented) and
accomplishments of preparedness activities (e.g., ERO
personnel trained, drills held to reinforce training,
individual skill tests performed, exercises conducted.)
RESPONSE indicators reflect the activities or functions
associated with the response emergency management elements.
In contrast to the quantitative PROGRAMMATIC indicators, the
majority of RESPONSE indicators are based on subjective
assessments of specific response activities or functions by
expert evaluators observing simulated performance during
exercises or making assessments of projected performance
through evaluations of plans and procedures. RESPONSE
indicators can be either a qualitative assessment or
evaluation of a single activity or function, or a
quantitative roll-up of individual qualitative assessments
of multiple activities or functions.
ORGANIZATIONAL indicators will reflect factors influencing
overall organizational behavior (e.g., management
commitment and participation, management decision making,
allocation of resources, organizational culture,
communication within the organization) that might influence
the performance of the various functions and activities
associated with the emergency management program. These
measures may be considered indicators of systemic
organizational influences that could affect the readiness
of an organization in various emergency management
functional areas.
Next, as noted above, performance indicators can be
characterized by What is measured. The following types of
indicators reflect the general spectrum of available
measurement data: INPUT-Based, ACTIVITY-Based, OUTPUT-Based,
and OUTCOME-Based. An example from an emergency management
program is given below:
TRAINING and DRILLS
INPUT-Based - Training requirements (Positions,
qualification criteria, personnel
assignments, course content, trainer
qualifications . . .)
ACTIVITY-Based - Conduct training sessions (Number of
sessions, hours, personnel, training aids,
drill requirements . . .)
OUTPUT-Based - Successfully trained for specific activity
(Written tests, individual performance tests,
or exercise results indicate success)
OUTCOME-Based - Personnel successfully execute the
activity for which they are trained
(Successful outcome based on performance
during an actual event.)
The spectrum of data, represented by INPUT-based through
OUTPUT-based, represents a graded scale covering data which
only remotely reflects the performance of the program
(i.e., INPUT) through data that directly measures the
success of the program (i.e., OUTCOME) in accomplishing its
mission.
Data can also be collected that reflects performance of a
program during the current time frame versus anticipated
performance in a future time frame. A DOE emergency
management program can be viewed as having at least two
phases:
1.Development, including planning and initial
implementation
2.Maintenance, review, and improvement (or steady-state)
Performance indicators selected to assess and track
readiness vary depending on the phase of the program. For
example, during the development phase, an indicator might
include completion of plans and procedures. In the
maintenance phase, the indicator might transition to
periodic review of plans and procedures. For mature
systems (i.e., in maintenance phase), the readiness
evaluation component based on program evaluations defaults
to quantitative programmatic data alone, since reviews of
the plans and procedures themselves will ultimately reach a
limit in effectiveness for identifying performance problems.
As a result, exercise evaluations will become the primary
means for tracking readiness during the maintenance phase of
an emergency management program.
Performance indicators may also reflect program status or
the actual or projected performance in different time
frames, within a given phase of the program. An example of
a trailing indicator might be the number of findings
associated with program and exercise evaluations for the
past year. A leading indicator that could predict future
performance is the percentage of required Corrective Actions
closed or resolved during the past year. A current
indicator might represent the percentage of required Hazards
Survey/EPHA completed at the end of a reporting period
associated with Phase I, for example.
Finally, performance indicators can be quantitative,
qualitative, or a combination of quantitative and
qualitative parameters. A quantitative indicator can be
represented by the absolute value of a numerical parameter
or its value relative to a goal. A qualitative indicator
derives from a subjective evaluation of a programmatic or
response activity or function, with the resulting indicator
having one of two answers, Pass/Fail or, equivalently,
Yes/No. An Ordered Scale can also be used with either a
quantitative parameter or a quantitative roll-up of
individual qualitative assessments. An Ordered Scale
provides a naturally ordered and labeled set of relative
grades by which to express a judgment (i.e., evaluation)
that becomes a performance indicator. Examples of Ordered
Scales include:
1.A, B, C, D, and F
2.EXCELLENT, VERY GOOD, GOOD, POOR, and INADEQUATE
3.GOOD, ADEQUATE, MARGINAL, and POOR
4.BEST INDUSTRY PRACTICE, EFFECTIVE PERFORMANCE,
IMPROVEMENT NEEDED, and SIGNIFICANT WEAKNESS
A combination of quantitative and qualitative
characteristics can be subjective and produce data in
categories that are naturally ordered. A combined indicator
can use subjective evaluation results, in terms of a roll-up
(i.e., weighted or unweighted addition) of numbers of
criteria satisfied (or failed) for a given function or
activity within a programmatic or response element; for
example, an evaluation judgment expressed in terms of an
Ordered Scale or a Pass/Fail criteria.
The source of most quantitative performance indicators will
be programmatic data and preparedness activity measures,
such as percentage of plans/procedures completed or
percentage of ERO trained, respectively. Note that the goal
for the first is 100%. However, the training goal may be a
site-specific percentage that can only be obtained through
experience in testing and evaluating the program. There are
also some quantitative indicators that could be indicative
of performance during the emergency response, but there
are not many. Task duration (i.e., promptness) is the
primary measure that may be derived from an exercise.
Others, such as the percentage of total ERO successfully
activated by the first call-out, might also reflect
performance using site plans and procedures as the standard.
Emergency management performance indicators resulting from
evaluations may not conform strictly to the definitions
found in the standard performance measurement literature.
These indicators may involve a subjective, qualitative
judgment of failure or adequacy or a quantity that reflects
failure or adequacy, such as, the number of formal findings
or evaluation criteria satisfied for a specific program
element. Qualitative performance indicators can also
reflect systemic failure across program elements, by
collecting findings or criteria to reflect performance
indicators such as decision-making, communication,
planning basis, performance, control, etc. Emergency
management fault modes for evaluation criteria were given in
Section 4.4.5.
The general development of performance indicators and how to
use these to measure progress or success are given in
DOE G 120.1-5, Guidelines for Performance Measurement.
Practical examples of performance indicators that are
specific to emergency planning and preparedness
(i.e., programmatic performance indicators), include:
Completing ERO training when it is due/scheduled
Maintaining a schedule for regular ERO equipment
surveillance and maintenance
Completing a given percentage of procedure reviews each
year
Meeting goals for drilling the organization with hazards
survey/assessment scenarios that have not been addressed
for some time
Contacting a target number of news media organizations
each year with information on the emergency response
program
Maintaining a schedule of contacts with mutual aid
organizations
Internal evaluation effectiveness number of findings
not self-identified
Examples of performance indicators that can be used by
facilities and sites to help monitor emergency response
(i.e., response performance indicators) capability from
exercises and drills:
Achievement of facility accountability within required
timeframe
Ability to release approved emergency information to
media in a timely manner
HAZMAT team response time
Correct event categorization/classification decision
making
Availability of prompt notification system
Performance indicators for emergency management can include
factors that reflect management commitment
(i.e., organizational performance indicators) to the
emergency management program. Examples include:
Management participation in ERO
Allocation of resources for the program
Participation in review meetings
CAP approvals
Communication within the emergency management
organization
Management involvement in program assessment activities
4.8.3 DOE/NNSA Framework and Performance Indicators
Performance indicators for emergency management provide the
means for each tier of the DOE/NNSA management system and
the Office of Emergency Operations to track areas of
critical importance to the mission of emergency management
throughout the DOE/NNSA complex. At each tier of the
DOE/NNSA system, indicators can be defined that measure
performance in establishing and maintaining emergency
management capabilities commensurate with their
responsibilities for planning, preparedness, readiness
assurance, and response.
Responsibilities of the levels of management in the DOE/NNSA
hierarchy that have not been addressed are the oversight
responsibilities with respect to facilities/sites or field
elements below them in the organizational structure.
Performance indicators can provide a mechanism for tracking
emergency management programs at multiple facilities/sites
within the responsibility of each DOE/NNSA Cognizant Field
Element or Headquarters organization. Most critical
performance indicators can be rolled-up or aggregated to
express a summary of performance for all applicable
facilities/sites and activities. For example, a performance
measure, such as percentage of ERO trained, has a goal for a
specific facility/site of XX%. At the next level up in the
DOE hierarchy, the performance indicator could be expressed
as percentage of facilities/sites that have met or exceeded
their specific ERO training goals. Similarly, performance
indicators that reflect performance of the DOE/NNSA field
elements can also be rolled-up for tracking at the
headquarters level. Roll-up techniques can also be applied
to qualitative PROGRAMMATIC, RESPONSE, and ORGANIZATIONAL
performance indicators.
APPENDIX C. Emergency Readiness Assurance Plans
This appendix provides an acceptable template of format and
content for Emergency Readiness Assurance Plans (ERAPs) for
reporting the status of Emergency Management Programs for
DOE/NNSA sites and activities. Unless otherwise indicated,
the following ERAP sections apply to both Operational
Emergency Base Program Facilities and Operational Emergency
Hazardous Material Programs. Example information used in
the tables to clarify expected entries is given in italics.
EMERGENCY READINESS ASSURANCE PLANS - FY 20XX
1.Program Description: Provide a brief summary statement
of the overall status (e.g., mature, evolving) and level
of readiness (commendable/acceptable/marginal) of the
emergency management program.
A.Based on the results of the Hazards Surveys/Emergency
Planning Hazards Assessments (EPHAs), briefly describe
the rationale for determining whether the site consists
of Base Program Facilities only or requires a Hazardous
Material Program.
Briefly, discuss the status of your Hazards Surveys.
If a survey has not been completed, provide information
on completing the surveys, as outlined in Table C-1.
Table C-1. Hazards Survey Status
Building/Fac Planned Actual EPHA Update
ility Completion Completion Require (when
Date Date d hazards
change or 3
years)
Bldg. XYZ 10/28/05 10/25/05 Yes 10/28/06
N-Area 5/31/06 6/12/06 No 5/31/09
Solid Waste 06/07/06 06/05/06 Yes 06/05/09
Disposal
Facility
Laboratory 1/31/06 2/12/06 Yes 1/31/07
9999
PQR Facility 09/30/06 Survey Unknown TBD
delayed due
to change in
facility
operation
Briefly, describe the generic types of Operational
Emergencies (OEs) (from your Hazards Survey) that could
affect your site, facility, or operation.
If Hazards Surveys indicate EPHAs are required, discuss
briefly the status of your EPHAs. Provide the status
of all EPHAs (complete/ incomplete) across the site
using Table C-2.
Table C-2. EPHA Status
Building/F Planned Completi Hazardous Update
acility Completion on Date Material (when
Date Program hazards
Required change or
3 years)
Bldg. XYZ 3/25/06 02/28/06 Yes 02/28/07
Laboratory 05/31/06 05/30/06 Yes 05/30/07
9999
Solid 09/015/06 09/12/06 No 09/12/09
Waste
Disposal
Facility
If a Hazards Material Program is required, identify the
dominant hazards on the site in terms of the most
severe consequences (i.e., General Emergency (GE), site
Area Emergency (SAE), or Alert; biological release OEs)
from potential OEs using the table format below. The
table should contain the majority of the most severe
potential hazardous material releases.
Table C-3. Dominant Potential OEs
Facility Emergency Radioactive/Chemical/Biolo
gical Material
Classification
GE SAE A
Building XYZ X X Chemical - Arsine
Radionuclide - Plutonium
Laboratory Biological Material (OE
9999 only)
B.In narrative format, provide the status of your
emergency plan and implementing procedures (date of
last review, and if appropriate, date of revision).
C.Exemptions: List any exemptions that have been
requested, and the associated rationale for exemption.
If none have been requested, provide the statement no
exemptions requested.
Table C-4. Exemptions
Exemption Reason Date of Approv Duration
Submissi al
on Date
Change time for Consistency 6/1/03 1/28/0 Review
notification to with 4 annually
state and local approved
agencies MOU
2.Program Application: This section of the ERAP describes
how provisions of the readiness assurance program,
outlined in the emergency plan, have been applied in the
past fiscal year and provides a forward look at what
changes in focus will be applied in the future to assure
continuous improvement. In narrative format, discuss:
A.What weaknesses in the program are being revealed
through self-assessment, evaluations, exercises, and
training and drills?
B.Are there differences between weaknesses found during
self-assessment activities and those found by external
oversight organizations? How are these differences
explained or reconciled?
C.How are weaknesses revealed in the past year to be
addressed? What progress has been made in implementing
corrective actions? Have corrective actions been
effective in resolving the original weakness?
Table C-5. Program Weaknesses
Evaluat Conducti Identified Facili Date Correctiv Correc
ion ng Weakness ty of e tive
Method Organiza specif Evalua Action(s) action
tion ic or tion Status
Site
Wide
Annual HS-63 Site EOC did Site 11/02/ Revise Comple
Exercis not notify 05 Job Aids, ted
e HQ Watch all
Office of Revise items
the event training 06/25/
and initial and 06
classificati
on. Retrain
EOC
members
Annual HS-63 None of the Facili 11/02/ Re-mark Comple
Exercis Bldg. XYZ, ty 05 rally ted
e personnel points, all
knew where items
their Revise 09/15/
evacuation facility 06
rally points job aid,
were
located; Conduct
accountabili refresher
ty took 60 training,
minutes.
Conduct
evacuatio
n
exercise
to
validate.
Self- Internal MOU(s)/MOA(s Site 01/28/ Update In
Assessm ) with 06 all progre
ent offsite MOU(s) ss,
agencies out and 90%
of date MOA(s) comple
ted
Self- Internal EPHA(s) does Site 01/28/ Revise In
Assessm not provide and 06 EPHAs to progre
ent full affect provide ss,
technical ed basis for 50%
basis for Facili source comple
source terms ties terms ted
No NA-40 EOC could Site 06/18/ Revise In
Notice not provide and 06 procedure progre
Exercis consequence affect ss;
e assessments ed Retrain 70%
to offsite Facili on-duty comple
agencies for ties EOC ted
two hours personnel
after event .
classificati
on. Conduct
drills
Conduct
evacuatio
n
exercise
to
validate.
Following the format in Table C-5, provide outstanding
evaluation/assessment findings (issues) identified by
external sources (i.e., HS-63). This applies to
findings reported in the Departmental Corrective Action
Tracking System (CATS) only:
Table C-6. Status of Findings/Corrective Actions (as reported in
CATS)
CAP Finding/Issue No. Due Status Projected
(identify number of Date Completion
corrective actions for each
issue in parentheses)
NTS-06/28/1999-I0001 (24) 07/30/0 On 07/30/05
5 schedule
D.What lessons are being drawn from operating experience
of other DOE facilities/sites, as well as from related
industries?
E.Where does the facility/site have resource or other
constraints that affect the process of continuous
improvement of the program? What is the effect of the
constraints?
3.Program Achievements: For the fiscal year just
completed, following the format in Table C-7, compare
actual achievements accomplished to projected goals,
milestones, and objectives. Progress in meeting assigned
program performance metrics should be reported here also.
Table C-7. Emergency Management Program Achievements: Goals,
Milestones, Objectives, and Status
Goal Milestones Objective Status
Conduct Exercise Plan Successful Exercise
Annual Approval 02/25/06 accomplishment of conducted
Exercise exercise 05/25/06;
Rehearsal Drill objectives. 2
04/25/06 deficienci
es and 3
Exercise 05/25/06 areas
needing
improvemen
t
identified
Conduct Bldg. XYZ: 02/28/06 All EPHAs revised All
annual or reviewed by facilities
review or Laboratory 9999: end of FY06 have
revision 05/30/06 completed
of all required
facility Solid Waste EPHA
EPHAs Disposal Facility: revisions/
09/12/06 reviews by
FY06.
Conduct Classes conducted All personnel Completed
Annual each quarter assigned to Site 09/15/06
Training and Facility
of ERO complete annual
training (initial
or refresher)
Conduct Develop initial set Develop Offsite
initial of objectives objectives for agencies
planning 06/25/06 FY07 Full concurred
for FY07 Participation on final
Full Conduct meeting with Exercise set of
Participa State Emergency objectives
tion Management and Public Get concurrence on
Exercise Health Dept. on on exercise from 09/27/06
exercise scope and State Emergency
objectives 07/15/06 Management and
Public Health
Conduct meeting with Dept.
County Emergency
Management and Public Get concurrence
Health Dept. on on exercise from
exercise scope and County Emergency
objectives 08/10/06 Management and
Public Health
Submit revised Dept.
objectives to State
and County agencies
for concurrence
09/22/06
4.Program Goals: For the new fiscal year, describe
projected goals, milestones, and objectives following the
format shown in Table C-8.
Table C-8. Emergency Management Program Projections:
Goals, Milestones, Objectives
Goal Milestones Objective
Conduct Exercise Plan Successful
Annual Approval 03/01/07 accomplishment of
FY07 Full exercise
Participa Rehearsal Drill objectives.
tion 04/25/07
Exercise
Exercise 05/25/07
Conduct PQR Facility: Complete Hazards
Hazards 4/20/07 Survey
Survey
Conduct Bldg. XYZ: 03/07/07 EPHAs revised or
an review reviewed as
of Laboratory 9999: needed by end of
process 05/30/07 FY07
changes
that Solid Waste
require a Disposal Facility:
review or 09/12/07
revision
of
facility
EPHAs
Conduct Classes conducted All personnel
Annual each quarter assigned to Site
Training and Facility
of ERO complete annual
training (initial
or refresher)
Conduct Develop initial set Develop
initial of objectives objectives for
planning 06/25/07 FY08 Annual
for FY Exercise
08 Annual Conduct meeting with
Exercise exercise participants Obtain
on exercise scope and concurrence on
objectives 07/15/07 exercise scope
and objectives
Submit revised from
objectives to participants.
participants for 10/03/2007
concurrence 09/22/07
Conduct Final
Planning
Conference.
10/15/07
5.Other: Briefly discuss concerns pertinent to the
emergency readiness assurance program. Specifically,
this section addresses issues unique to the facility/site
that should be brought to the attention of senior
management (i.e., concerns with state and local agencies,
funding, resources, etc.). Include suggestions or
methods to resolve these issues.
Resource requirements should include resources necessary
to administer and operate the emergency management
program. Be sure to include resources necessary for
conduct of self-assessments and emergency exercises,
resources needed to make corrective actions, improve on
weaknesses, and implement lessons learned. Changes in
regulatory and statutory requirements associated with
emergency management should also be considered when
determining required resources associated with program
changes.
A.Provide the total number of full-time/part-time
site/facility personnel required for the current and
next fiscal years in Table C-9 by Federal and
contractor staff. Describe the composition (i.e.,
types of personnel) included in the totals for each
category. In narrative format, provide justification
for changes in staffing levels.
Table C-9. Emergency Management Personnel -
Full Time Equivalents
Organization FY06 FY07
Federal 2.5 2
Contractors 15 15
Justification:
B.In Table C-10, summarize the Emergency Management
Program operational budget. Describe the general types
of costs in each category. In narrative format,
provide justification to support additional funding
requirements.
Table C-10. Emergency Management Operational Budget
Organization FY06 FY07
Federal $700K $1,005K
Contractors $2,200K $3,000K
Justification:
C.If equipment requirements are not included in the
Operational Budget, provide budget estimates for
equipment in Table C-11. This list includes
necessities such as EOC equipment, field monitoring
equipment, and Radiological Assistance Program (RAP)
Team equipment:
Table C-11. Equipment Requirements
Item FY06 FY07
Plotter Printer $50K
Replace TV Monitors $6K
in the EOC
Replace 2 $30K
radiological field
monitoring
instrument sets
Justification:
APPENDIX A.
APPENDIX D. Evaluation Criteria
D.1 Introduction
This appendix provides a standard set of generic performance-
based criteria to be used for the evaluation of the
planning, implementation, maintenance, and response of
emergency management programs across the DOE/NNSA complex.
These evaluation criteria were developed to provide a
comprehensive set of criteria applicable to the diversity of
DOE/NNSA facilities/sites and activities based on meeting
the performance goal(s) specified for each program element.
Performance goals (or standards) were developed from
DOE O 151.1C, Comprehensive Emergency Management System.
Differences in hazards will dictate how facilities/sites and
activities approach program planning, implementation,
maintenance, and response. Hence, evaluations of such
programs should be conducted in a manner commensurate with
hazards and missions. Criteria presented here are
intentionally generic and were developed to reflect the
actual or expected general performance of the emergency
management element and not the specific details of
plans/procedures, tools, organizational structure,
products, resources, training, etc.
Criteria should NOT be used during an exercise evaluation as
given. Program-specific expectations and characteristics
should be developed for each emergency management program
based on facility/site- or activity-specific hazards and
associated program capabilities (e.g., derived from existing
plans and procedures). From these attributes, generic
criteria can be restated in the context of the specific
program. This facilitates the evaluators task by bounding
the general intent and scope of the function or activity, as
expressed in the generic criteria, and focusing on the key
program-specific attributes incorporated in the revised
criteria. In contrast, during a program evaluation, generic
criteria for programmatic and response elements are used as
the standard, against which plans and procedures, and
preparedness activities are judged in the context of
facility/site- or activity-specific hazards, associated
program capabilities, and the commensurate with hazards
approach.
For example, a criterion that addresses sufficient
staffing of Emergency Response Organization (ERO) positions
is stated generally to cover a site with a 5 member ERO as
well as a site with 100 members. An actual quantitative
sufficient staffing requirement is program-specific. The
standard set of criteria can be interpreted in the context
of a particular program or activity, or a specific scenario
during an exercise, in a number of ways, including the
elimination of inapplicable criteria and the development of
program-specific criteria from the generic criteria given in
this chapter. Any modifications or additions to the set of
generic criteria contained herein should be documented and
well understood by all evaluating elements prior to conduct
of the evaluation.
Fifteen broad sets of criteria are grouped according to
Program Element in Sections D.2, D.3, and D.4 of this
appendix. Each program element section includes the
performance goal of the element and a set of evaluation
criteria, separated according to response or programmatic
subelements, as appropriate. The set of criteria are
labeled to identify applicable evaluation types:
P for program evaluation only,
E for exercise evaluation only,
P/E for both program and exercise evaluations; and
CE for evaluation of the conduct of an exercise.
Numbered criteria, for example, P2.1, P/E6.42, P/E12.2, and
P/E14.12, represent critical criteria necessary to ensure
that the performance goal for each element is satisfied.
Subcriteria labeled as a, b, c . . . represent performance
that supports the numbered criteria and emphasize a
distinguishable function, component, or activity that merits
special attention. Subcriteria do not necessarily represent
or define the total performance expectations of the
criterion; they help refine the interpretation of
performance failures and specific Findings. CE criteria
given in Section D.3.3 for the Exercise program element are
used for evaluating planning, control, conduct, and
evaluation of an exercise.
As indicated in DOE G 151.1-3, Section 4.4, several Response
elements have programmatic functions, in addition to their
primary response functions. These are given at the end of
the lists of response function criteria.
D.2 Technical Planning Basis
Performance Goal:
The Hazards Survey is an examination of the features and
characteristics of the facility/site or activity that
identifies the generic types of emergency events and
conditions and the potential impacts of such emergencies to
be addressed by the DOE Comprehensive Emergency Management
System. The Hazards Survey identifies key components of the
Operational Emergency Base Program that provide a foundation
of basic emergency management requirements and an integrated
framework for response to serious events involving health
and safety, the environment, safeguards, and security. For
facilities/sites and activities involved in producing,
processing, handling, storing, or transporting hazardous
materials that have the potential to pose a serious threat
to workers, the public, or the environment, the Hazards
Survey provides a hazards screening process for determining
whether further analysis of the hazardous materials in an
Emergency Planning Hazards Assessment (EPHA) is required.
An EPHA is performed for each facility/site or activity
involving at least one candidate hazardous material, as
identified through the hazardous material screening process
and indicated in the Hazards Survey. EPHAs involve the
application of rigorous hazards analysis techniques that
provide sufficient detail to assess a broad spectrum of
postulated Operational Emergency (OE) events or conditions
involving the potential onsite release of (or loss of
control over) hazardous materials and to analyze the
resulting consequences. Each EPHA reflects both the
magnitude and the diversity of the hazards and the
complexity of the processes and systems associated with the
hazards, and provides the technical planning basis for
determining the necessary plans/procedures, personnel,
resources, equipment, and analyses (e.g., determination of
an EPZ) for the Operational Emergency Hazardous Material
Program.
Evaluation Criteria:
Hazards Survey
P1.1 The Hazards Survey identifies the generic types of
serious emergency events or conditions to which the
specific facility/site or activity may be exposed
(e.g., fires; flood; tornadoes; earthquakes;
hazardous material releases; regulated pollutant or
oil spills; safeguards and security events; work
place accidents; malevolent acts; mass casualties;
wildland fires; nearby offsite non-DOE hazardous
material accidents).
P1.2 The Hazards Survey qualitatively identifies the
potential impacts of different generic types of
emergencies on health and safety, the environment,
and national security.
P1.3 The Hazards Survey identifies emergency management
requirements that constitute the Operational
Emergency Base Program:
a.DOE orders [other than DOE O 151.1C], other
Federal agency, state, and local emergency
planning and preparedness requirements associated
with different generic types of emergency events
or conditions and applicable to the facility/site
or activity; and
b.Existing plans, such as earthquake self-help plans
or mass casualty plans, detailing compliance with
Federal, State, or local standards, are
incorporated directly into the Operational
Emergency Base Program or are invoked by
reference.
P1.4 Facilities/sites and activities that require a
documented, quantitative EPHA are identified by a
hazardous material screening process and are
indicated in the Hazards Survey.
P1.5 Hazards Surveys are reviewed and updated every three
years to include changes in the hazards. If changes
result in an increase in hazards, the Hazards Survey
is updated immediately; otherwise, the Hazards
Survey can be updated at the next scheduled review.
Hazardous Material Screening Process
P1.6 A hazardous material screening process is developed
and applied to facilities/sites and activities
involved in producing, processing, handling,
storing, or transporting hazardous materials that
have the potential to pose a serious threat to
workers, the public, or the environment.
P1.7 The screening process identifies candidate hazardous
materials that, if released in an uncontrolled
manner, would immediately threaten or endanger those
who are in close proximity; have the potential for
dispersal beyond the immediate vicinity in
quantities that threaten onsite personnel or the
public; and have a potential rate of dispersal to
require a time-urgent response to implement
protective actions for workers or the public.
Protective Action Criteria (PACs) are used to
indicate when the consequences of a release of a
radioactive or chemical hazardous material threaten
or endanger health and safety.
P1.8 The hazardous material screening process identifies
all hazardous materials in a facility/site or
activity that require further analysis in an EPHA.
P1.9 All radioactive materials in a facility/site or
activity are subjected to a hazardous material
screening process.
P1.10 Radioactive materials excluded from further analysis
in an EPHA include:
a.Sealed radioactive sources that are engineered to
pass the special form testing specified by the
Department of Transportation (DOT) or the American
National Standards Institute (ANSI);
b.Materials in solid form for which there is no
plausible dispersal mechanism; materials stored in
DOT Type B shipping containers with overpack, if
the Certificates of Compliance are current and the
materials stored are authorized by the
Certificate; and
c.Materials used in exempt, commercially available
products.
P1.11 Radioactive hazardous materials that are analyzed in
an EPHA include the radioactive materials listed in
DOE-STD-1027-92 in quantities greater than the
Category 3 values given in Attachment 1, Table A.1,
of that Standard.
P1.12 All chemicals in a facility/site or activity with
known or suspected toxic properties are subjected to
a hazardous material screening process.
P1.13 Chemicals excluded from further analysis in an EPHA
include:
a.Materials used in the same form, quantity, and
concentration as a product packaged for
distribution and use by the general public;
b.Materials that have a Health Hazard rating of 0, 1
or 2 based on National Fire Protection Association
(NFPA) 704; and
c.Solid or liquid materials that because of their
physical form or other factors (e.g., plausible
dispersal mechanisms) do not present an airborne
exposure hazard.
P1.14 Quantities of chemical hazardous materials
considered to be easily and safely manipulated by
one person are determined in accordance with the
provisions of 29 CFR 1910.1450(b).
P1.15 Chemical hazardous materials in quantities greater
than a quantity that can be easily and safely
manipulated by one person that are analyzed in an
EPHA include:
a.Chemicals with an assigned Health Hazard rating of
3 or 4 based on NFPA 704; and
b.Chemicals without an assigned Health Hazard
rating.
P1.16 All biological hazardous materials in a
facility/site or activity are subjected to a
hazardous material screening process.
P1.17 At a minimum, Federally regulated biological Select
Agents and Toxins, identified in lists published in
Department of Health and Human Services (HHS)
regulations [42 CFR 73] and Department of
Agriculture (USDA) regulations [7 CFR 331 and 9 CFR
121], require further analysis in an EPHA. Toxins
listed in 42 CFR 73 and 9 CFR 121 must exceed the
minimum quantities specified to be Federally
regulated and require an EPHA.
P1.18 If biological agents and toxins require further
analysis in an EPHA, then a Hazardous Material
Program is established.
P1.19 The possibility that excluded materials could
initiate, through fires, explosions, or process
upsets, the release of other hazardous materials is
considered. In addition, excluded asphyxiates are
considered if they have the potential to affect
collocated workers because of the large quantity,
material characteristics, and favorable geography.
P1.20 If the screening process identifies at least one
hazardous material requiring further analysis, the
Hazards Survey must indicate that an EPHA is needed
for that facility/site or activity.
P1.21 The Office of Secure Transportation (OST) develops
an EPHA for OST shipments and establishes an
Operational Emergency Hazardous Material Program.
P1.22 An EPHA is developed for shipments that do not
satisfy governing DOT regulations and specifications
for commercial hazardous materials transport. No
EPHA is required if the shipment satisfies these
commercial transport regulations and specifications.
Emergency Planning Hazards Assessment (EPHA)
P1.23 The EPHA describes the site and facility or
activity, directly or by reference, including:
a.Site location, facility description, operations,
mission, processes, tenant activities, and
facility locations (including proximity to
adjacent facilities, site boundary, utility and
transportation networks);
b.Transportation activities, including types and
quantities of materials transported, containers,
routes, speeds, and controls exercised; and
c.Characteristics of the region beyond the site
boundary including summaries of demographics
(including special populations), administrative
boundaries, geographic features, and economic
enterprises (e.g., farms, factories).
P1.24 The EPHA contains a current, accurate compilation of
hazardous material inventories or maximum quantities
associated with a facility/site or activity based on
reliable and comprehensive methods of hazardous
material identification (e.g., walkthroughs,
shipping records, local chemical inventory systems).
P1.25 Analyzed hazardous materials are characterized in
the EPHA:
a.Storage location, process use, physical
properties, and health effect parameters;
b.Engineered controls, administrative controls,
storage segregation, safeguards and safety systems
for prevention and/or mitigation of releases are
identified; and
c.Actual barriers to release are identified, such
as, containers, buildings, berms, sumps, catch
basins, filters, and heating, ventilation and air
conditioning (HVAC) systems.
P1.26 A spectrum of potential emergency event/condition
scenarios are postulated and realistically analyzed
in the EPHA, including:
a.Applicable initiating events (e.g., fire,
explosion, natural phenomena, malevolent events,
accidents, external events);
b.Contributing events, accident mechanisms,
equipment or system failures, engineered safety
system and control failures, source terms,
material release chemistry and characteristics,
environmental transport and diffusion, exposure
considerations, and health effects;
c.Range of event probabilities and consequences,
from low probability, high consequence to high
probability, low consequence, including Beyond-
Design-Basis events;
d.Events exclusively affecting onsite personnel, as
well as those affecting the offsite public; and
e.Potential malevolent acts applicable to the
facility/site or activity based on Design Basis
Threat (DBT) guidance, if available.
P1.27 Emergency events or conditions are NOT excluded from
analysis in the EPHA based solely on calculated
occurrence probabilities or arbitrarily defined
delimiters (e.g., credible or incredible, likely or
unlikely).
P1.28 Indicators of emergency event/condition scenarios
that can be used for recognition purposes in
developing Operational Emergency (OE) categorization
criteria and Emergency Action Levels (EALs), as
appropriate, are identified and documented in the
EPHA.
P1.29 Estimates of the consequences of hazardous material
release scenarios (primarily radioactive and
chemical) are calculated and documented in the EPHA:
a.Receptor locations include facility and site
boundaries, collocated facilities, and offsite
locations, including special populations
(e.g., schools, hospitals, and prisons).
b.Calculations are performed for the purposes of
protective action determinations, response
decision-making, and special planning,
[e.g., Emergency Planning Zone (EPZ)
determinations].
c.Methods and models used for calculating
consequences are applicable to the releases
analyzed; assumptions used are valid and
documented.
P1.30 Biological agent release scenarios are analyzed to
obtain indicators for recognizing OE
events/conditions and for initial protective
actions. The analysis methodology is documented in
the EPHA.
P1.31 Classified material quantities and storage are
analyzed and documented in a classified annex to the
EPHA.
Maintenance of the EPHA
P1.32 The EPHA is reviewed and updated every three years.
P1.33 An accurate and timely method for tracking changes
in operations processes, or accident analyses that
involve hazardous materials (e.g., introduction of
new materials, new uses, significant changes in
inventories, modification of material environments)
is established and maintained for the facility/site
or activity.
P1.34 Management procedures are implemented to ensure that
emergency planners are notified of significant
changes in inventories, processes, or activities
that may affect the results of the EPHA
[e.g., active involvement of emergency management
personnel in the Integrated Safety Management System
(ISMS)].
P1.35 Sufficient transition time is allowed for emergency
management personnel to review the EPHA and modify
plans or procedures, as necessary, to account for
changes in the hazardous material situation.
P1.36 Changes made in the facility/activity or activity
safety analysis reports, probabilistic risk
assessments, vulnerability assessments, fire hazard
analyses, environmental impact statements, and other
documents that address hazards or potential
consequences are integrated with maintenance of the
EPHA.
P1.37 If changes result in an increase in hazardous
material inventories or release potential, the EPHA
is updated immediately; otherwise, the EPHA can be
updated at the next scheduled review.
P1.38 After a decontamination and decommission action is
completed, the Operational Emergency Hazardous
Material Program is adjusted to be commensurate with
the hazards that remain.
Emergency Planning Zone (EPZ)
P1.39 The size and shape of the EPZ is determined by the
spectrum of scenarios, the consequences of the
potential releases, health effect parameters, and
geo-political boundaries beyond the site boundary.
P1.40 The EPZ is the area within which protective actions
will most likely be taken to protect workers or the
public from the effects of the majority of airborne
hazardous material releases from the facility or
site.
P1.41 The EPZ defines an area within which protective
actions will provide for substantial reduction in
early lethality for all analyzed airborne hazardous
material releases.
P1.42 The EPZ is sufficiently large that the planning
efforts within the defined EPZ provide a substantial
basis for expansion of response activities beyond
the EPZ, if warranted by actual conditions.
P1.43 The maximum EPZ for any DOE or NNSA facility or site
does not exceed a nominal radius of 10 miles (16
kilometers).
P1.44 Biological hazardous material release scenarios are
not used in determining the size of the EPZ.
D.3 Programmatic Elements
D.3.1 Program Administration
Performance Goal:
Effective organizational management and administrative
control of the facility/site or activity emergency
management program is provided by establishing and
maintaining authorities and resources necessary to plan,
develop, implement, and maintain a viable, integrated, and
coordinated comprehensive emergency management program.
Evaluation Criteria:
Organizational Management and Administrative Control
P2.1 An individual is designated to administer the
facility/site or activity emergency management
program with responsibility and authority to ensure:
a.The development and maintenance of the Hazards
Surveys and EPHAs, emergency plans and procedures,
and related and supporting documentation;
b.The development of the annual Emergency Readiness
Assurance Plan (ERAP);
c.The development and conduct of the training and
exercise programs and the coordination of
readiness assurance (evaluation, assessment)
activities; and,
d.The coordination of emergency resources by
identifying resource needs and ensuring the
availability of adequate resources.
P2.2 The designated administrator has the authority and
resources, commensurate with assigned
responsibilities, and has access to top-level
management.
P2.3 For biosafety facilities, the designated Responsible
Official (RO) is responsible for implementing and
maintaining the emergency management program. This
designated administrator/official is responsible for
tasks that involve compliance with the requirements
for the Select Agent Rule(s) [i.e., HHS regulation
42 CFR 73 and USDA regulations 7 CFR 331 and
9 CFR 121] and with existing DOE/NNSA emergency
management policy as expressed in DOE O 151.1C.
P2.4 Administration of planning, preparedness, and
readiness assurance activities is established and
effectively maintained.
P2.5 Formal review and approval processes are established
and documented to ensure that the planning and
development of components of the emergency
management program (e.g., planning analyses, plans
and procedures, supporting documentation) receive
sufficient oversight by staff, management and DOE
elements to ensure consistency, correctness, and
completeness.
P2.6 Reasonable schedules (e.g., documentation
submittals, reviews, and approvals; preparedness and
readiness assurance activities) are established and
enforced to ensure that program planning,
preparedness, and readiness assurance activities are
initiated, completed, and repeated in a timely and
efficient manner.
P2.7 An emergency management document control system is
established that meets industry standards for
document review, approval, distribution, and change
control.
P2.8 An auditable administrative program for ensuring the
availability of vital records essential to the
continued functioning or reconstitution of an
organization during or after an emergency,
regardless of media, is established and reliably
maintained.
P2.9 If classified information or materials are being
used or generated, effective security procedures and
controls are implemented, and security reviews are
conducted.
P2.10 Financial resource requirements are identified and
budgeted.
P2.11 Facilities and equipment requirements are
identified, monitored, and acquired.
P2.12 Personnel requirements are identified and addressed.
Specific Program Responsibilities
P2.13 Emergency plans and procedures are developed,
verified, validated, reviewed periodically and
updated as necessary.
P2.14 Emergency management programs and emergency plans
are developed for facilities not requiring a
Hazardous Material Program that address the minimum
Base Program requirements.
P2.15 Emergency management programs and emergency plans
are developed for facilities requiring a Hazardous
Material Program that are seamlessly integrated with
Base Program requirements.
P2.16 Facility emergency management programs on a site are
consistent, and are integrated to ensure site-wide
consistency.
P2.17 A leased facility owned by DOE/NNSA effectively
integrates the activities of the leased facility
into the DOE/NNSA site-wide emergency management
program.
P2.18 Biosafety facility incident response plans are
integrated with the site-wide emergency management
program.
P2.19 Training, drills, exercises, and evaluation
activities are scheduled, conducted, monitored, and
documented.
P2.20 Development and approval of supporting documentation
[e.g., Memoranda of Understanding (MOUs), Memoranda
of Agreement (MOAs)] is accomplished; periodic
reviews and maintenance are scheduled and conducted.
P2.21 Emergency management documents are controlled,
available, and current.
P2.22 Correction of findings and incorporation of lessons-
learned are tracked, addressed, verified and
validated.
a.Methods are in place and implemented to remain
appraised of current events and lessons learned
and to utilize this information for continuous
improvement; and
b.A site-wide corrective action program is
implemented and effective in correcting problems
identified in the emergency management program.
P2.23 Specific emergency management Order requirements
related to administrative responsibilities and
emergency management activity (i.e., planning,
preparedness, readiness assurance)
parameters/constraints are monitored for compliance.
Document Requirements
P2.24 Current reviewed and approved Hazards Surveys and
EPHAs are available and provide technical planning
basis information for the development of the
Operational Base Program and Operational Hazardous
Material Program, commensurate with the hazards.
P2.25 Emergency plans and procedures:
a.An emergency plan documents the emergency
management program, including provisions for
response to an OE; Emergency Plan Implementing
Procedures (EPIPs) describe how the emergency plan
will be implemented;
b.Clearly state roles, responsibilities, and
requirements associated with program
administration, emergency response organizations,
individual positions, operations, and interfaces;
and
c.Describe the integration and coordination of the
emergency management program with the DOE ISMS.
P2.26 If a facility is generating classified information
or Unclassified Controlled Nuclear Information
(UCNI), all emergency management documents
(e.g., plans and procedures, supporting program
documentation, scenarios, and assessments) are
reviewed by a Derivative Classifier (DC) or UCNI
reviewing official.
P2.27 Documented arrangements with leased facilities
include:
a.Description of how each of the lessees emergency
management program elements are integrated into
the site-wide program; and
b.A requirement that the lessees hazardous material
inventories be reported to the site emergency
management program annually; and
c.A requirement that the lessee must report
significant changes to the facility or hazardous
material inventories prior to implementing the
changes.
D.3.2 Training and Drills
Performance Goal:
A comprehensive, coordinated, and documented program of
training and drills is an integral part of the emergency
management program to ensure that preparedness activities
for developing and maintaining program-specific emergency
response capabilities are accomplished.
Evaluation Criteria:
Training Program
P3.1 A comprehensive and systematic training program plan
for accomplishing emergency management training
goals includes: training objectives, target
audience, an outline and schedule of training,
resources and facilities, organizational
responsibilities, and training program
administration.
P3.2 The training program for all primary and alternate
personnel assigned to the facility- and site-level
ERO includes the following key provisions for
position-specific requirements:
a.Initial training and annual refresher training;
b.Refresher training when hazards or emergency
plan/implementing procedures change; and
c.Demonstrations of proficiency through testing and
drills.
P3.3 The emergency management training program provides a
current and structured view of program-specific
training requirements:
a.The training program is reviewed and updated
periodically, or as required, based on changes in
related emergency plans/procedures;
b.A detailed list of courses and drills provided by
the emergency management program is developed and
maintained; and
c.Matrices for the identification and implementation
of required training topics versus ERO positions
are developed and maintained.
P3.4 Administrative program records provide the source
for identifying qualified instructors, training
material approval authority, and qualification
signature authority.
P3.5 The program plan defines minimum program standards
for:
a.Training required for each position (i.e., certain
courses must be completed);
b.Proficiency (e.g., minimum grades on tests, how
prior experience is credited);
c.Performance (i.e., acceptable performance during
drills, exercises, or actual events); and
d.Retraining, and re-validation.
P3.6 The emergency management training program is
effectively integrated and coordinated with related
training programs provided by other organizations.
P3.7 Training courses are performance-based, customized
to program-specific ERO positions, contain learning
objectives, and have testing as a final validation
of satisfactory completion.
P3.8 Refresher training includes details of program
changes and lessons-learned from actual events,
exercises, DOE and industry operating experience,
and program evaluations.
P3.9 The training program requirements are in accordance
with the National Response Plan (NRP) and National
Incident Management System (NIMS).
Training Requirements - Onsite
P3.10 Initial training and periodic drills are provided to
all workers who may be required to take protective
actions (e.g., shelter-in-place; assembly,
evacuation). This training is required when they
are employed, when their expected protective actions
change, or when the emergency plan changes.
P3.11 Refresher training is provided annually to certified
operators and supervisors, and those workers who are
likely to witness a hazardous materials release and
who are required to notify proper authorities of the
release.
P3.12 Both initial training and annual refresher training
is provided for instruction and demonstration of
proficiency by all personnel (i.e., primary and
alternate) comprising the ERO for their assigned
position or function.
P3.13 Special team training is conducted for functional
groups, in particular those with technical and
management team assignments (e.g., consequence
assessment).
P3.14 To ensure that ERO decision makers are able to
perform their duties promptly and accurately:
a.Training emphasizes the need for prompt, accurate,
and practical judgments involving event
categorization and classification, protective
actions, and the urgency of notifications of OEs;
b.EAL training is conducted periodically to improve
the proficiency of ERO decision makers in timely
and conservative classification of OEs, including
decision-making when information is incomplete or
uncertain and for events and conditions that are
not covered explicitly by the EALs; and
c.ERO personnel authorized for initial
classification and protective action decision-
making validate their proficiency by participating
in performance tests that employ hypothetical
scenarios and available facility/site aids, such
as EALs.
Training Requirements - Offsite
P3.15 Offsite emergency response personnel and
organizations, including state, local, tribal, or
private hospitals, public health, medical, or
ambulance services, that are expected to support
onsite response efforts, are offered:
a.Training on facility- and site-specific emergency-
related information, conditions, and hazards; and
b.The opportunity to participate in training and
drills validating procedures for response
activities expected to involve integration of
onsite and offsite response resources.
Drills
P3.16 Drills provide supervised, hands-on training
and/or validation of classroom training for members
of the ERO.
P3.17 Drills provide opportunities to demonstrate
responder proficiency in infrequently performed
emergency management tasks.
P3.18 Scheduled drills include scenario driven events that
provide interface practice between the ERO and site
medical and security organizations.
P3.19 Drills are developed or modified based upon feedback
from actual events, exercise evaluations, and self-
assessments, or to validate new or revised
procedures and equipment modifications.
Training Documentation and Records
P3.20 Lesson plans, drill plans, training materials and
facilities, instructor and student manuals, and
training software are maintained, formally
documented, and included in an index or matrix.
P3.21 Training records are maintained for all personnel
assigned ERO positions, primary and alternate,
showing in-progress, final, and upcoming re-
qualification status.
P3.22 Drill and exercise participation and performance is
documented for each member of the ERO.
D.3.3 Exercises
Performance Goal:
A formal exercise program validates all elements of an
emergency management program over a 5-year period. The
exercise program validates facility- and site-level
emergency management program elements by initiating response
to simulated, realistic emergency events/conditions in a
manner that, as nearly as possible, replicates an integrated
emergency response to an actual event. Planning and
preparation use an effective, structured approach that
includes documentation of specific objectives, scope, time
lines, injects, controller instructions, and evaluation
criteria for realistic scenarios. Each exercise is
conducted, controlled, evaluated, and critiqued effectively
and reliably. Lessons-learned are developed, resulting in
corrective actions and improvements.
Evaluation Criteria:
Exercise Program
P4.1 A formal exercise program includes the validation of
elements of an emergency management program over a 5-
year period.
a.The exercise program includes a plan (e.g., a
matrix) for validating all the elements of each
program by incorporating specific objectives in
exercises over the 5-year period.
b.The exercise program also includes provisions for
incorporating objectives in each exercise that are
designed to validate revised plans/procedures,
implemented corrective actions, and program
improvements.
c.The exercise program includes provisions for
evaluating all exercises and establishes a
critique process, which includes gathering and
documenting observations of participants.
P4.2 The exercise program involves testing emergency
response capabilities by initiating response to
simulated, realistic emergency events/conditions in
exercises of varying scope over the 5-year period:
a.Facility Operations-Based Exercise - A facility or
group of facilities (i.e., with common facility-
level ERO positions) annually tests the
proficiency of personnel in facility-level ERO
positions in accomplishing facility-specific
emergency response duties and responsibilities.
b.Site Operations-Based Exercise - At least
annually, the site tests the integrated emergency
response capabilities of personnel in facility-
and site-level ERO positions, and includes both
facility- and site-level evaluation and critique.
For multi-facility sites, the basis for the
exercise is rotated among facilities or groups of
facilities.
c.Full Participation Operations-Based Exercise - A
site-level exercise is considered full
participation if offsite organizations
participate. Offsite response organizations are
invited to participate in a site-level exercise at
least once every 3 years.
P4.3 The schedule of exercises includes:
a.Periodic participation by appropriate DOE or NNSA
radiological response assets, if the facility/site
plans to use the assets in response to an
emergency.
b.Security scenario events to test the interfaces
between site security and the facility/site ERO.
P4.4 Exercises of each of the Departments radiological
emergency response assets are conducted at least
once every three years. These assets include the
Accident Response Group (ARG), Nuclear Emergency
Support Team (NEST), Federal Radiological Monitoring
and Assessment Center (FRMAC), Aerial Measuring
System (AMS), National Atmospheric Release Advisory
Center (NARAC), Radiation Emergency Assistance
Center/Training Site (REAC/TS), and Radiological
Assistance Program (RAP).
P4.5 At a minimum, building evacuation exercises are
conducted annually consistent with Federal
regulations [e.g., (41 CFR 102-74-360)], local
ordinances, or NFPA standards, to ensure that
employees are able to safely evacuate their work
area.
P4.6 Communications with DOE Headquarters (HQ), the
Cognizant Field Element, and offsite agencies are
tested at least annually or as often as needed to
ensure that communications systems are operational.
P4.7 Failed objectives of an exercise
(i.e., Deficiencies), as determined by a DOE or
NNSA organization responsible for evaluating the
exercise, are re-evaluated during a drill or through
a selected functional test within a fixed time
period following the exercise.
P4.8 Corrective actions and lessons-learned identified as
a result of facility- and site-level exercise
evaluation findings are addressed by the emergency
management program.
a.Completion of corrective actions for facility and
site exercises includes a verification and
validation process, which verifies that the
corrective action has been put in place and
validates that the corrective action has been
effective in resolving the original finding;
b.The verification and validation process is
independent of those who performed the corrective
action; and
c.Corrective actions involving revision of
procedures or training of personnel are completed
before the next exercise.
Exercise Planning
CE4.1 Exercise planning is effectively coordinated among
onsite and offsite organizations or groups regarding
their respective participation and exercise
objectives. Any limitations or simulations
regarding their participation are identified and
documented.
CE4.2 An exercise is fully documented by an Exercise Plan
(EXPLAN) that includes: specific exercise
objectives, scope, scenario, participants,
simulations, time lines, injects (i.e., messages),
technical data, safety and security provisions,
controller instructions, and evaluation criteria.
CE4.3 The EXPLAN is completed in sufficient time before
the conduct of the exercise to allow for review and
comments by DOE or NNSA line management and the DOE
Associate Administrator of Emergency Operations.
CE4.4 The EXPLAN contains sufficient information for
effective conduct, control and evaluation of the
exercise.
a.The roles, responsibilities, and interfaces among
exercise participants (i.e., players/responders,
controllers, evaluators, and observers) are
clearly addressed;
b.The provisions for exercise conduct and control
are clearly identified; and
c.The provisions for exercise evaluation are clearly
identified.
CE4.5 Specific exercise objectives provide the basis for
evaluating/validating the performance of response
capabilities by each participating organization.
CE4.6 The scenario is consistent with the set of exercise
objectives and explicitly supports an
evaluation/validation of each objective.
CE4.7 The exercise evaluation criteria used are
facility/site- or activity-specific, based on
existing plans and procedures, and correlate with
the exercise objectives.
CE4.8 The scenario reflects current facility/site- or
activity-specific hazards, correlates technically
with the EPHA, and is technically accurate in terms
of operations and radiological, chemical,
biological, and meteorological data.
CE4.9 The technical data that supports the scenario
(e.g., operational, radiological, chemical,
biological, medical, meteorological) is technically
accurate and clearly and unambiguously presented.
CE4.10 Simulations and limitations pertaining to
participants and exercise activities are clearly
identified and documented.
CE4.11 Injects/messages contain accurate, unambiguous, and
non-prompting information and technical data for the
players/responders and provide proper direction for
the exercise.
CE4.12 Provisions for safety, security, and public/media
interface are clearly identified and documented.
Exercise Preparation
CE4.13 Coordination among participants includes provisions
for exercise initiation, interruption, and
termination.
CE4.14 Controllers and Evaluators are provided generic and
exercise-specific training.
CE4.15 Controllers and Evaluators are provided with
training on the scenario package, and safety and
security/safeguards provisions.
CE4.16 Preparations, including participant briefings,
safety provisions, staging of simulation props,
positioning of controllers/evaluators, and
establishing of initial conditions, are completed
prior to exercise initiation.
CE4.17 Security of the exercise scenario is properly
managed; pre-staging of players and/or prior
knowledge of scenario material by players are
effectively prevented.
Exercise Conduct/Control
CE4.18 Controller organization(s) are adequately staffed
and positioned for effective exercise
conduct/control.
CE4.19 Controllers conduct/control the exercise in
accordance with the EXPLAN.
CE4.20 Controllers permit free play when free play would
not interfere with the scenario.
CE4.21 Controllers prevent interference and/or prompting by
non-responders.
CE4.22 The simulation of activities is sufficiently
realistic to provide confidence that the activity
could have been performed during a real emergency.
CE4.23 Players/responders perform their respective
functions, initially and throughout the exercise, in
a professional manner as if the situation were an
actual emergency.
Exercise Evaluation
CE4.24 The evaluator organization is sufficiently staffed
to evaluate the performance and key decision-making
of the responders in satisfying the exercise
objectives.
CE4.25 Evaluators display familiarity with responder
organizations, functions, procedures, and
anticipated responder decisions and response
activities.
CE4.26 Responders/players are evaluated with respect to
demonstrated proficiency of their respective
responsibilities and functions, communication and
coordination with other responders, familiarity
and use of applicable procedures and equipment,
and overall professional response.
CE4.27 Facilities and equipment are evaluated with respect
to adequacy of functions/operability.
CE4.28 Procedures are evaluated with respect to their use
by the responders, specifically, their adequacy of
content for the tasks performed.
CE4.29 Notifications and communications are evaluated
during every exercise.
Exercise Critique
CE4.30 Controllers conduct a post-exercise critique(s) to
gather and document observations and solicit
feedback from the players/responders.
CE4.31 A formal critique process is conducted by the
controller/evaluator organization to determine
whether the individual exercise objectives were
accomplished based on a synthesis of all the
observations and information/data gathered during
the conduct of the exercise.
Documentation
CE4.32 An After Action Report (AAR) documents the results
of the exercise critique and evaluation.
D.3.4 Readiness Assurance
Performance Goal:
The emergency management Readiness Assurance Program
provides a framework and associated mechanisms for assuring
that emergency plans, implementing procedures, and resources
are adequate by ensuring that they are sufficiently
maintained, exercised, and evaluated (including evaluations
and assessments) and that appropriate and timely
improvements are made in response to needs identified
through coordinated and comprehensive emergency planning,
resource allocation, training and drills, exercises, and
evaluations.
Evaluation Criteria:
General
P5.1 An effective formal and structured Readiness
Assurance Program is implemented consisting of
evaluation and improvement programs, and
documentation of the readiness of the emergency
management program based on emergency planning and
preparedness activities and the results of the
readiness assurance program [e.g., in ERAPs]
Evaluation Program
P5.2 An evaluation program assures that emergency plans,
implementing procedures, and resources are adequate
and sufficiently maintained, exercised, and
evaluated (including evaluations and assessments).
P5.3 Evaluations, including program evaluations and
exercise evaluations, are based on a consistent set
of performance-based evaluation criteria, issued by
the Associate Administrator, Office of Emergency
Operations [see DOE G 151.1-3, Chapter 4, Appendix
D].
P5.4 Self-evaluations:
a.A self-assessment of the emergency management
program is conducted annually by the facility/site
or activity.
b.Findings (i.e., weaknesses or deficiencies) are
identified in all program and exercise
evaluations.
c.Records are maintained of readiness assurance self-
evaluations (e.g., program or exercise self-
assessments) and any related findings.
P5.5 External evaluations:
a.Evaluation schedules are coordinated with all
involved organizations to minimize impacts and
maximize benefits. Evaluation schedules are
forwarded to the Associate Administrator, Office
of Emergency Operations to ensure maximum
coordination.
b.Personnel responsible for developing or
maintaining the emergency management program as
well as associated program documentation are made
available during periodic external evaluations.
c.Findings (i.e., weaknesses or deficiencies) are
identified in all external program and exercise
evaluations.
d.Evaluated findings from program and exercise
evaluations by organizations external to the
facility/site or activity are acknowledged within
30-working days of receipt of the final evaluation
report.
P5.6 Formal evaluation reports are prepared that document
evaluation results and specific findings.
P5.7 Performance indicators (including performance
measures and metrics) capture and track objective
data regarding the performance of emergency
management programs in key functional areas; the
results are shared with the Cognizant Field Element
and Associate Administrator, Office of Emergency
Operations.
P5.8 No-Notice Exercises (NNXs), conducted at the
discretion of the Associate Administrator, Office of
Emergency Operations, determine if the ERO
accomplishes selected objectives based on applicable
plans, procedures, and/or other established
requirements. Involvement is limited to providing
trusted agents and responding when the exercise is
conducted.
Improvement Program
P5.9 An improvement program provides assurances that
appropriate and timely improvements are made in the
emergency management program in response to needs
identified through coordinated emergency planning,
resource allocation, program assistance activities,
evaluations, training, drills, and exercises.
P5.10 Continuous improvement in the emergency management
program results from implementation of corrective
actions for findings (e.g., deficiencies,
weaknesses) in all types of evaluations, including
both self-assessments and external evaluations.
P5.11 Evaluated findings from program and exercise
evaluations by organizations external to the
facility/site or activity are acknowledged and
include corresponding corrective action plan.
P5.12 Corrective action plans are developed within 30-
working days of receipt of the final evaluation
report.
P5.13 Corrective actions are completed as soon as
possible. Corrective actions addressing revision of
procedures or training of personnel are completed
before the next annual self-assessment of the
program.
P5.14 Completion of corrective actions includes a
verification and validation process, independent
of those who performed the corrective action,
that verifies that the corrective action has
been put in place, and validates that the
corrective action has been effective in
resolving the original finding.
P5.15 Closure of findings from program and exercise
evaluations by organizations external to the
facility/site or activity is validated by the
evaluating organization.
P5.16 The improvement program prepares corrective action
plans, and establishes and maintains a tracking
system to monitor and verify correction of
findings from all program and exercise
evaluations, or from actual responses.
P5.17 The improvement program includes a system for
incorporating and tracking lessons learned from
training, drills, actual responses, and a site-wide
lessons learned program.
P5.18 An established improvement program ensures that
relevant lessons learned (i.e., complex-wide; other
non-DOE sources) are received at the facility/site
or activity, are reviewed for applicability, and
incorporated in the emergency management program as
appropriate.
P5.19 An effective and reliable improvement program is
ensured through sustained management commitment to
continuous improvement of the emergency management
program.
Emergency Readiness Assurance Plan (ERAP)
P5.20 The ERAP highlights program status, including,
significant changes in emergency management programs
(i.e., planning basis, organizations, exemptions)
and comparison of previous ERAP goals, milestones,
and objectives to achievements.
P5.21 The ERAP identifies what the program goals were for
the fiscal year that ended coincident with the due
date for the report and the degree to which these
goals were accomplished. The ERAP also identifies
the goals for the next fiscal year.
P5.22 The ERAP documents evaluation results and the status
(i.e., open/unresolved or closed) of associated
corrective actions. Evaluation results include
facility/site and activity self-assessments and
performance measures.
P5.23 The ERAP contains a sufficient level of accurate
information and analysis to provide management at
all levels with an adequate tool for gauging
emergency management program readiness.
P5.24 Accurate site (i.e., facilities consolidated into
one site document) ERAPs are developed and submitted
to the responsible DOE/NNSA Cognizant Field
Elements.
D.4 Response Elements
D.4.1 Emergency Response Organization (ERO)
Performance Goal:
An ERO, a structured organization with overall
responsibility for initial and ongoing emergency response
and mitigation, is established and maintained for each
facility/site and activity. The ERO establishes effective
control at the scene of an event/incident and integrates ERO
activities with those of local agencies and organizations
that provide onsite response services. An adequate number
of experienced and trained personnel, including designated
alternates, are available on demand for timely and effective
performance of ERO functions.
Evaluation Criteria [RESPONSE Functions]:
ERO Organizational Structure
P/E6.1 The organizational configuration of the ERO is based
on actual or potential emergency conditions.
P/E6.2 Management structure of the emergency response
facility provides for the collecting and
disseminating accurate data, setting priorities,
assigning work to functional groups, and keeping key
emergency response staff abreast of emergency
response status.
P/E6.3 An Emergency Director (ED) or equivalently titled
individual manages and controls all aspects of the
facility/site or activity overall response, and has
the authority to use necessary resources to mitigate
the emergency.
P/E6.4 The ED has the authority and responsibility to
perform the required functions, including initial
activation of onsite response assets, notification
of offsite authorities, and requests for offsite
assistance, in accordance with the National Response
Plan (NRP) and the National Incident Management
System (NIMS).
P/E6.5 The division of authority and responsibility between
the Incident Commander (IC) and the ED position is
clearly established and maintained.
P/E6.6 Control of operations, monitoring, and repair teams
is clearly vested in a single ERO position or
clearly defined between multiple ERO positions.
ERO Activation
P/E6.7 The ERO activation is based on actual or potential
emergency conditions.
P/E6.8 The on-shift operations staff performs initial ERO
response functions.
P/E6.9 The ERO is functionally staffed and activated in a
timely manner; key emergency response facilities are
operational within an hour after declaration of an
OE.
P/E6.10Staffing of ERO positions following the declaration
of an OE is orderly, controlled, and verifiable:
a.Personnel assigned to ERO positions gain access to
their response stations without impediment.
b.Non-ERO personnel are excluded from emergency
response work areas.
c.Individuals in key response positions/functions
are readily identifiable by other ERO staff
(e.g., through use of status board(s) or badging).
P/E6.11Procedures and/or checklists, which describe the
major activation and initial response activities of
key members of the ERO, are used.
P/E6.12The order of succession of management personnel
responsible for managing the emergency in the
absence of the primary designated ED is clearly
designated/ implemented.
P/E6.13Extended operations (i.e., shift arrangements to
cover 24-hour operations) are anticipated and
planned.
ERO Operations
P/E6.14The ED, in the lead role responsible for emergency
response, adequately and effectively performs
assigned functions utilizing sufficient and
practical knowledge of the effected facility/site or
activity and its operations, the emergency response
team and its mission, and the available tools and
resources necessary to affect an appropriate
response and mitigate the emergency.
P/E6.15Transfer of a command and control function to
another emergency facility, within an emergency
facility, or to a command external to the ERO or ICS
(e.g., another Federal agency, such as DOJ/FBI) is
completed in an orderly and formal manner, and ERO
personnel are informed of the transfer.
P/E6.16The fully staffed ERO establishes effective internal
and external interfaces with other agencies and
organizations; external interfaces may include:
local, state, tribal, and federal agencies, and non-
governmental groups such as concerned citizens and
the media.
P/E6.17An individual in the ERO is assigned liaison
responsibilities for coordinating with offsite
agencies to ensure that effective communications are
initiated and maintained during an emergency.
P/E6.18Members of the ERO:
a.Perform in their roles, functions, and interfaces
and in their use of emergency equipment,
facilities, and resources in a timely, effective
and efficient manner;
b.Clearly acknowledge and understand authorities and
responsibilities in functional areas; and
c.Identify and access available response resources
(e.g., personnel, equipment, consumables, and
replacement parts), and, as appropriate, take
account of resource limitations and specific
capabilities.
P/E6.19Based on current knowledge of the situation, the
responsible ERO operations and technical support
staff determine and implement a reasonable, well-
planned course of action within their sphere of
responsibility.
P/E6.20When priority actions are identified, tasking is
clearly made to emergency response staff, and
actions are followed through to completion.
P/E6.21Specialty groups (e.g., consequence assessment,
maintenance, operations, technical staff) supporting
the emergency response staff provide timely
information to the decision-making process.
P/E6.22Adequate data are obtained and analyzed to support
the operations staff in assessing and mitigating the
emergency events.
P/E6.23Information is accurately and efficiently
transmitted in an orderly and documented manner
throughout the chain of command and between/within
emergency facilities.
P/E6.24The use of acronyms, code words, convention and/or
technical terminology causes no misunderstandings
related to the response and associated data.
P/E6.25Periodic briefings are provided on the status of the
emergency and current significant response
priorities and activities.
P/E6.26Communications are maintained with and information
is provided regularly to the DOE Headquarters
Emergency Management Team (EMT).
P/E6.27The ERO management effectively coordinates State and
DOE site requests for use of DOE/NNSA assets.
P/E6.28An individual is assigned liaison responsibilities
with personnel representing DOE/NNSA assets
(e.g., NARAC, FRMAC, AMS, RAP, REAC/TS, ARG, and/or
NEST) involved in the response to coordinate
logistics, ensure that effective communications are
initiated and maintained, and ensure that data is
exchanged using consistent units of measure.
P/E6.29ERO personnel are provided with adequate briefings
concerning safety, operations, communications, and
hazards before being deployed.
P/E6.30ERO teams are debriefed upon return from assigned
missions and their accomplishments, failures,
exposures, and status information are recorded and
made available to other teams and emergency
facilities.
P/E6.31The responsible individual authorizes emergency
response personnel to receive exposures in excess of
site administrative limits (or other Federal
criteria) for carrying out lifesaving or other
emergency activities.
P/E6.32An individual trained to recognize, categorize, and
classify events and to conduct appropriate
notifications is available 24-hours a day, 7-days a
week. This individuals authority is unambiguous
and clearly communicated throughout the ERO.
Evaluation Criteria [Special RESPONSE Functions/Positions]:
Incident Command System (ICS)
P/E6.33.1 An IC is in charge at the event scene:
a. Control and coordination at the
event/incident scene is consistent with the
NRP and the NIMS/Incident Command System
(ICS), which integrates local agencies and
organizations that provide onsite response
services.
b. The ICS is identified in the emergency plan
and memoranda of understanding/agreement with
local response organizations.
P/E6.33.2 The ICS is organized in the five major
functional areas of NIMS/ICS: Command, Operations,
Planning, Logistics, and Finances and
Administration.
P/E6.33.3 The incident is assessed and priorities are
established with life saving, safety, and incident
stabilization receiving top priority.
P/E6.33.4 Incident command strategic goals and tactical
objectives are clear and a flexible action plan is
implemented.
P/E6.33.5 Incident command evolves from providing oral
direction to the development of a written Incident
Action Plan (IAP).
P/E6.33.6 The incident command staff continually
assesses the situation, develops a mitigation
strategy, and requests additional assets as needed.
P/E6.33.7 Incident command coordinates internal and
external response assets in an effective manner.
P/E6.33.8 An ICS command post is strategically located
in a safe area, where command and control may take
place safely and effectively.
P/E6.33.9 Command post and staging area(s) habitability
is periodically assessed and moved, as necessary,
for safety purposes.
P/E6.33.10 Incident command staff ensures that response
personnel take necessary precautions for personal
safety and contamination control, as follows:
a. Incident command staff establishes a staging
area where arriving asset personnel are briefed;
communications are checked; special equipment is
issued; and the assets are deployed upon request.
b.Asset personnel being released are debriefed;
personnel are accounted for; personnel and
equipment are surveyed for contamination;
decontaminated as necessary; and issued equipment
is returned.
Hazardous Material Survey, Sampling, and Sample Analysis
Teams
P/E6.34.1 Teams implement survey and sampling procedures
in a timely manner:
a.Field teams are provided with adequate monitoring
equipment and Personnel Protective Equipment (PPE)
to accomplish field monitoring and plume tracking
within and beyond the EPZ; and
b.Teams correctly use protective equipment, such as
protective clothing and respirators, filter masks,
and dosimetry.
P/E6.34.2 Equipment required for emergency response is
adequate, accessible, functional, and calibrated.
P/E6.34.3 Teams make effective use of maps or general
arrangement drawings showing pre-determined and
potential monitoring points.
P/E6.34.4 Teams are briefed on facility and
meteorological conditions, and exposure control
procedures before deployment and when changes occur.
P/E6.34.5 Teams maintain effective communications to
transmit accurate and timely readings and results to
their team coordinator.
P/E6.34.6 Field teams are well directed and effectively
controlled by emergency response management, who:
a.Provide directions to survey specific areas;
b.Provide directions to minimize hazardous material
exposure by exiting high airborne and whole body
dose areas (i.e., for radiological materials), or
high concentration areas (i.e., for toxic non-
radiological materials), when not actively engaged
in sample and survey activities; and
c.Set exposure limits for survey and tracking teams,
and collect and record survey results.
P/E6.34.7 Teams utilize proper survey equipment and log
results accurately.
P/E6.34.8 Teams collect samples, bag and mark them, and
log results accurately and efficiently.
P/E6.34.9 Samples are received, properly packaged, and
labeled with information such as sample time and
date, sample location, volumetric data, sample
media, and sample or survey collection person's
name.
P/E6.34.10 Analysis procedures and equipment are used to
support processing of samples received, either
properly analyzing the samples in the field or
transporting them to a laboratory.
P/E6.34.11 Analysis results are promptly and accurately
communicated to other emergency response
organizations.
Security Staff
P/E6.35.1 Security procedures of protective forces for
carrying out their responsibilities during response
to OEs are promptly, safely, efficiently, and
effectively implemented.
P/E6.35.2 An ICS, in accordance with NIMS/ICS
requirements, is implemented for security
emergencies.
P/E6.35.3 Response of protective force personnel and
equipment is characterized by effective command and
control.
P/E6.35.4 Access and egress control is quickly and
properly maintained for the facility/site or
activity, facility/site areas, impacted areas
(i.e., safe perimeters), and emergency response
facilities.
P/E6.35.5 Security practices facilitate timely movement
and access of facility/site operating and response
personnel (including offsite personnel) to required
areas during the emergency situations.
P/E6.35.6 Under emergency conditions, material
accountability and protection for Special Nuclear
Material (SNM) and other critical DOE assets are
handled in a timely and effective manner.
P/E6.35.7 Common protocol for local law enforcement
backup of the onsite security force is used
(e.g., use of deadly force, weapons employment,
tactics, code words, radio frequencies, etc.).
P/E6.35.8 A mutual understanding of authorities and
responsibilities, response plans, utilization of
command and control facilities, and terminology
enables site security to effectively coordinate and
correlate response activities with other components
of the ERO.
Fire and Rescue
P/E6.36.1 Fire/rescue personnel and equipment are
assembled and deployed to the scene of the emergency
in a safe and timely manner.
P/E6.36.2 Fire/rescue personnel take necessary
precautions for contamination, exposure, heat, and
personal safety.
P/E6.36.3 Search and rescue operations are carried out in
an efficient manner, coordinating their efforts with
medical, industrial hygiene, and health physics
personnel.
P/E6.36.4 Injured personnel are properly extricated,
immobilized, and moved during search and rescue
operations.
P/E6.36.5 When responding onsite, both onsite and offsite
fire personnel are outfitted with the appropriate
specialized equipment and supplies specific to the
onsite hazards.
Repair and Maintenance
P/E6.37.1 Facility and field repair and maintenance
activities are carried out in a timely and efficient
manner:
P/E6.37.2 Proper tools are available for repair and
maintenance activities and the procurement of
replacement parts is expedited.
P/E6.37.3 Emergency work order procedures are used and
emergency tagging (e.g., lockout/tagout or
clearance) is implemented.
P/E6.37.4 Repair and maintenance activities include
personnel protection and monitoring as well as
coordination with support groups, such as health
physics and chemistry personnel.
Evaluation Criteria [PROGRAMMATIC Functions]:
ERO Staffing
P6.38 ALL personnel who may be needed to perform duties,
beyond those specified by 29 CFR 1910.120 for the
first responder awareness level, during a response
to any of a broad range of emergencies defined in
the Hazards Survey or EPHA are members of the ERO.
P6.39 Fully trained personnel are assigned to facility-
and site-level ERO positions to ensure adequate
staffing for emergency response.
P6.40 All personnel assigned to facility- and site-level
ERO positions demonstrate their proficiency in their
assigned positions through periodic participation in
an exercise, an evaluated drill, or an actual
response. All primary and alternate personnel
accomplish this participation on a rotating basis.
P6.41 An adequate number of experienced and trained
personnel for initial and ongoing response,
including designated alternates, are assigned to
each functional area.
ERO Maintenance
P6.42 To ensure that personnel are available on demand for
timely and effective performance of ERO functions,
the ongoing, standby staffing of ERO emergency
facility positions and response teams is effectively
accomplished by:
a.Using a technique, such as duty-cycle or static
roster, to ensure that qualified personnel are
available on-demand and properly assigned.
b.Ensuring that sufficient trained personnel for
initial and ongoing response, including designated
alternates, are candidates for call-up in each
functional area.
c.Periodically reviewing ERO rosters for accuracy
(e.g., current qualifications, correct phone
number, correct response time etc.).
d.Periodically reviewing and updating ERO personnel
qualifications.
P6.43 Communication systems used to activate both on-shift
and off-shift emergency response personnel are
periodically tested to ensure their adequacy and
reliability.
D.4.2 Offsite Response Interfaces
Performance Goal:
Effective interfaces are established and maintained to
ensure that emergency response activities are integrated and
coordinated with the Federal, Tribal, State, and local
agencies and organizations responsible for emergency
response and protection of the workers, public, and
environment, in accordance with the NRP and NIMS.
Evaluation Criteria [RESPONSE Functions]:
Activation
P/E7.1 Interfaces with Federal, Tribal, State, and/or local
authorities responsible for protection of the public
and the environment are identified and established.
P/E7.2 Support is requested, as required, from Federal,
Tribal, state, and/or local response
agencies/organizations responsible for augmenting
site resources in response to an onsite emergency
event.
P/E7.3 Offsite authorities are informed of the availability
of assistance from DOE or NNSA national assets
(i.e., RAP, FRMAC, NARAC, AMS, and REAC/TS) and
subsequent requests for support activate the
applicable assets.
Communication and Information Exchange
P/E7.4 Methods of communication (e.g., telephone circuits
and/or radio channels) and communication protocols
with the offsite agencies/organizations are in
place, identified, and operable.
P/E7.5 Communication capabilities allow effective
communication with offsite officials, the Cognizant
Field Element, and the DOE HQ EMT.
P/E7.6 Offsite officials are briefed upon activation of
their respective facilities.
P/E7.7 Offsite agencies/organizations, responsible for
emergency response and for the protection of
workers, the public, and the environment, are
provided initial and ongoing information sufficient
to perform their respective functions.
P/E7.8 Timely, clear, accurate, and effective information
exchange occurs between the ERO and offsite
personnel.
P/E7.9 Mutual understanding of acronyms, code words,
conventions, and/or technical terminology
(e.g., units) provides effective information
exchange.
P/E7.10Incoming offsite agency inquiries/concerns are
directed to the appropriate personnel for
resolution.
Coordination and Integration
P/E7.11A mutual understanding of capabilities, especially
the command and control system, supports an
integrated and effective response.
P/E7.12An effective working relationship exists between the
offsite officials and their ERO counterparts.
P/E7.13Coordination and integration with offsite response
agencies and organizations follow established, pre-
arranged and documented plans and protocols,
including, responsibilities and authorities,
coordination of response, notifications, facility
activations, communications, Emergency Operations
Center (EOC) interfaces, public information
activities, and logistic protocols (e.g., working
space and site access)
P/E7.14Provisions are in place and implemented with State,
Tribal, and local agencies and organizations for
coordinating the release of information about the
emergency to the public.
P/E7.15There is a mutual understanding of response measures
to be implemented by the facility/site in
anticipation of the involvement of local and State
public health agencies or agricultural authorities
following an actual or potential release of a
biological hazardous material.
Evaluation Criteria [PROGRAMMATIC Functions]:
Maintaining Interfaces
P7.16 An individual (s) with the appropriate authority,
knowledge, and training is responsible for
establishing and maintaining ongoing and effective
interfaces with offsite political, technical,
security (e.g., local law enforcement), public
health, and emergency services officials.
P7.17 Agreements to provide mutual assistance to or to
receive assistance from offsite organizations
(e.g., hospitals, fire departments) are documented
in a formal memorandum of agreement or memorandum of
understanding, which are accessible in the emergency
plan and maintained current through periodic
reviews.
P7.18 Offsite response agencies and organizations are
provided with specific information and/or offered
training on the nature and characteristics of the
biological agents and/or toxins present at the
DOE/NNSA Biosafety facility.
P7.19 Effective coordination with offsite response
agencies and organizations is accomplished and
maintained through routinely scheduled meetings.
P7.20 Through formal agreements, DOE supports offsite
agencies under the good neighbor policy, in areas
of emergency assistance including: fire, medical,
and hazardous material releases (including, field
monitoring resources)
P7.21 Routine coordination and interfaces through
training, drills, and good neighbor support ensure
that offsite services (e.g., fire and medical, law
enforcement), as indicated in the documented
agreements, will be integrated with onsite
resources.
P7.22 Planned response functions to be provided by offsite
organizations are periodically tested and verified.
P7.23 Offsite response organizations are invited to
participate in a site-level exercise at least every
3 years.
P7.24 Organizations which may be needed in a supporting
role and/or needed for long-term support have been
identified and pre-designated offsite points-of-
contact, including organization, names, and
telephones numbers are documented, maintained, and
available to the response organization.
D.4.3 Emergency Facilities and Equipment
Performance Goal:
Facilities and equipment adequate to support emergency
response are available, operable, and maintained. At a
minimum, facilities/sites include an adequate and viable
command center. Equipment includes, but is not limited to,
PPE, detectors, and decontamination equipment.
Evaluation Criteria [RESPONSE Functions]:
General
P/E8.1 Facilities and equipment adequate to support
emergency response are available, operable and
maintained.
Facilities
P/E8.2 A facility is available for use as a command center
by the ED, the EMT, and other members of the ERO
during an emergency response.
P/E8.3 Characteristics of the dedicated command center, and
other auxiliary facilities, are adequate to reliably
support the designated functions and assignments.
P/E8.4 As required, facilities are available to accommodate
classified discussions at the appropriate clearance
levels.
P/E8.5 Provisions are established for use of an alternative
location if the primary command center is not
available.
P/E8.6 Facility systems and installed equipment
(e.g., HVAC, sanitation, lighting, radiation
monitors, computer systems, communications, and
visual displays) are adequate to support facility
functions and level of staffing.
P/E8.7 Emergency response facilities use backup or
alternate power supplies in the event of loss of
power.
P/E8.8 As necessary, conversion of facilities to response
facilities for the emergency is accomplished in a
timely and efficient manner.
P/E8.9 Command center access control is adequate and
results in the efficient and timely identification
of assigned staff.
Equipment
P/E8.10The capability to notify employees of an emergency
to facilitate the safe evacuation of employees from
the work place, immediate work area, or both is
available.
P/E8.11Provisions are established to ensure operational
compatibility between facility response capabilities
and DOE or NNSA assets.
P/E8.12Adequate PPE, and other emergency equipment and
supplies are readily available and operable to meet
the needs determined by the results of the EPHA.
P/E8.13Actual function(s) and operating characteristics of
specific equipment adequately support the intended
function(s) during emergency response.
P/E8.14Secure communication equipment is available to
support classified discussions and transmittal of
classified documents/reports.
P/E8.15Equipment needed during the emergency response
functioned as expected and intended (or was repaired
or obtained in a timely manner), including: current
reference materials (e.g., maps, facility drawings);
decisional aids (including computers); area and
process monitors; public address system; PPE;
portable monitoring instruments and personnel
monitoring devices; siren and alarm systems;
decontamination equipment; communication equipment.
Evaluation Criteria [PROGRAMMATIC Functions]:
Facilities
P8.14 Designated response facilities, especially multi-use
facilities, are adequately maintained to ensure
timely activation and availability to support an
emergency response.
Equipment
P8.15 Inventories of all emergency equipment and supplies
are maintained with the equipment location
identified.
P8.16 Periodic inspections, operational checks,
calibration, preventive maintenance and testing of
equipment and supplies are carried out as required
in accordance with manufacturers instructions or
industry standards.
P8.17 Communication systems with DOE HQ, Cognizant Field
Elements, and offsite organizations are periodically
tested.
P8.18 Communication systems used to activate both on-shift
and off-shift emergency response personnel are
tested and maintained regularly.
D.4.4 Categorization and Classification
Performance Goal:
Major unplanned or abnormal events or conditions that:
involve or affect DOE/NNSA facilities/site and activities by
causing or having the potential to cause serious health and
safety or environmental impacts; require resources from
outside the immediate/affected area or local event scene to
supplement the initial response; and, require time-urgent
notifications to initiate response activities at locations
beyond the event scene, are recognized promptly,
categorized, and declared as OEs.
In general, to be considered an OE, an event or condition
involving the uncontrolled release of a hazardous material
will: immediately threaten or endanger personnel who are in
close proximity of the event; have the potential for
dispersal beyond the immediate vicinity of the release in
quantities that threaten the health and safety of onsite
personnel or the public in collocated facilities,
activities, and/or offsite; and have a potential rate of
dispersal sufficient to require a time-urgent response to
implement protective actions for workers and the public. In
addition to being categorized as OEs, events involving the
actual or potential airborne release of (or loss of control
over) hazardous materials from an onsite facility or
activity also require prompt and accurate classification as
an Alert, Site Area Emergency, or General Emergency, based
on health effects parameters measured or estimated at
specific receptor locations (e.g., facility and site
boundaries) and compared with PACs. Predetermined
conservative onsite protective actions and offsite
protective action recommendations are associated with the
classification of these OEs.
Evaluation Criteria [RESPONSE Functions]:
Process
P/E9.1 Authority and responsibility for categorizing an
event/condition, and if necessary, determining the
emergency classification, is clearly defined,
recognized, and understood by ERO personnel.
P/E9.2 The designated (authorized) individual with the
responsibility for categorization and classification
makes the determination(s).
P/E9.3 Recognition/categorization/classification process of
OEs is effectively integrated with existing
operations, management, emergency response,
reporting activities, and the security
classification scheme.
Categorization
P/E9.4 Categorization of abnormal events/conditions as OEs
is accomplished accurately using facility/site- or
activity-specific criteria.
P/E9.5 An OE event is categorized as promptly as possible,
but no later than 15-minutes after event
recognition/identification/discovery.
P/E9.6 The set of facility/site- or activity-specific
criteria is readily accessible to the responsible
decision maker.
P/E9.7 Criteria for categorizing OEs are clear,
straightforward, usable, and unambiguous to the
decision maker, and stated in terms of readily
available indications or observable conditions.
P/E9.8 If the event or condition is categorized as an OE
involving an airborne release of (or loss of control
over) hazardous materials (i.e., from a
facility/site), the decision maker recognizes the
requirement to promptly classify the event. This
does not apply to biological hazardous materials.
P/E9.9 A tool (i.e., an EAL-like tool) for recognizing and
categorizing biological OEs, based on recognition
factors identified in the EPHA, is part of the
DOE/NNSA emergency management program for Biosafety
facilities. Default initial protective actions are
associated with each biological OE.
P/E9.10An abnormal event/condition, categorized as an OE,
is only downgraded (e.g., to Significance Level 1-4)
if the original categorization was incorrect. A
properly categorized OE remains in effect until the
emergency response is terminated.
Classification
P/E9.11Classification of an OE involving the actual or
potential airborne release of (or loss of control
over) hazardous material is accomplished promptly
and accurately using a current set of a
facility/site-specific EALs. [Classification does
not apply to biological hazardous materials.]
P/E9.12Facility/site-specific EALs are applicable to the
spectrum of potential OEs identified by the EPHA.
P/E9.13Appropriate facility/site-specific EALs are readily
accessible to the responsible decision maker.
P/E9.14Classification of an OE involving the actual or
potential airborne release of hazardous material as
Alert, Site Area Emergency, or General Emergency is
based on the distance at which estimated
consequences exceed the applicable health effect
threshold [i.e., PAC for the specific hazardous
material released].
P/E9.15EALs for classifying OEs are clear, straight
forward, usable, and unambiguous to the decision
maker.
P/E9.16EALs for classifying OEs provide for early
recognition, are reliable, redundant, and internally
consistent, and are comprehensive and anticipatory
of potential/future consequences.
a.EALs are stated in terms of readily available
indications or observable conditions.
b.Facility/site-specific EALs are developed and
approved for the spectrum of OEs resulting in the
actual or potential airborne release of (or loss
of control over) hazardous material OEs, as
analyzed in the EPHA.
c.Facility/site EALs provide for classifying events
on the basis of measured or predicted hazardous
material consequences at specific receptor
locations (i.e., facility and site boundaries).
P/E9.17If a suspected release of (or loss of control over)
hazardous material fails to meet or exceed an EAL,
then a common sense, conservative assessment of the
indications or observable conditions leads to an
initial default estimate of the classification of
the emergency event/condition using the
discretionary EAL (i.e., a discretionary EAL is
included in the EAL set to compensate for possible
incompleteness and to ensure that a decision can be
made rapidly based on the current understanding of
the situation).
P/E9.18Associated with a specific OE event EAL, the
decision maker obtains default (i.e., pre-
determined), conservative Protective Actions (PAs),
for immediate implementation onsite, and Protective
Action Recommendations (PARs), for immediate
recommendation offsite.
P/E9.19The current classification is modified
(i.e., upgraded) based on continuous monitoring for
event degradation or a reassessment that indicates
that the event is more severe than originally
perceived.
a.An OE is reclassified at a lower classification if
the original classification decision was in error
(e.g., the decision maker used the wrong EAL or
received incorrect information).
b.A properly classified OE remains in effect until
the emergency response is terminated.
P/E9.20Site-wide, non-facility-specific EALs are used to
classify events such as: terrorist threats, major
natural phenomena, external events that can affect
site operations, etc.
Evaluation Criteria [PROGRAMMATIC Functions]:
P9.21 OE categorization criteria and EALs are reviewed and
tested regularly against a range of initiating
conditions and emergency event/condition scenarios
to validate the indicated emergency
categorization/classification.
D.4.5 Notifications and Communications
Performance Goal:
Initial emergency notifications are made promptly,
accurately and effectively to workers and emergency response
personnel/organizations, appropriate DOE/NNSA elements, and
other Federal, Tribal, State, and local organizations and
authorities. Accurate and timely follow-up notifications
are made when conditions change, when the emergency
classification (as an Alert, Site Area Emergency, General
Emergency) is upgraded, or when the emergency is terminated.
Continuous, effective, and accurate communication among
response components and/or organizations is reliably
maintained throughout an OE.
Evaluation Criteria:
Notifications
P/E10.1For OEs, prompt initial emergency notifications are
accurately and efficiently made to workers and
emergency response personnel/organizations,
including DOE or NNSA Cognizant Field Element EOCs
and the HQ Operations Center; other Federal, Tribal,
State, and local response organizations; as well as
all other appropriate organizations and authorities.
P/E10.2Tribal, State, and local officials, the Cognizant
Field Element EOC, and the HQ Operations Center are
notified within 15 minutes of classification of an
OE as an Alert, Site Area Emergency, or General
Emergency; all other organizations are notified
within 30 minutes.
P/E10.3The Cognizant Field Element EOC and the Headquarters
Operations Center are notified within 30 minutes of
the declaration of an OE that does not require
classification; local, State, and Tribal, and all
other organizations are notified within 30 minutes
or as established in mutual agreements.
P/E10.4Points of contact for emergency notifications are
accurate and readily available to response
personnel.
P/E10.5Emergency notifications to the HQ Operations Center
consist of a phone call providing as much
information as is known at the time. The same
information is also provided by e-mail or a fax
either immediately prior to or following the phone
call. Information for initial notification includes
as much as possible of the following:
(a) An OE has been declared and, if appropriate,
the classification of the emergency;
(b) Description of the emergency;
(c) Date and time the emergency was discovered;
(d) Damage and casualties;
(e) Whether the emergency has stopped other
facility/site operations or program activities;
(f) Protective actions taken and/or recommended;
(g) Notifications made;
(h) Weather conditions at the scene of the
emergency;
(i) Level of any media interest at the scene of
the emergency or at the facility/site; and
(j) Contact information for the DOE or NNSA on-
scene point of contact.
P/E10.6A rapid notification and recall system is used to
make initial and follow-up notifications to primary
and alternate response staff. The system provides
for authentication and feedback indicating
unsuccessful contact.
P/E10.7Follow-up notifications use pre-arranged and
standardized content and format that supports the
inclusion of critical information concerning: the
nature of the event, description and status; key
times; classification and release status (as
required); meteorology; protective actions; affected
facility; and, notification authority.
P/E10.8Follow-up notifications are made when conditions
change or when the emergency classification is
upgraded or the emergency is terminated.
P/E10.9The ED or designee personally approves release of
notification information.
P/E10.10 Emergency status reports (also referred to as
situation reports or SITREPs) are forwarded to the
next-higher EMT on a continuing basis throughout the
OE.
Communications
P/E10.11 A formally established communication chain for
reporting and notification within the facility, site-
wide, and to offsite organizations is properly
followed.
P/E10.12 Installed public address and siren systems
adequately accomplish the notifications of workers
and onsite or neighboring public.
a.Installed building and area alarms or public
address systems alert facility personnel to
emergency conditions.
b.Systems are in place for notification of onsite
workers and public present onsite but outside the
immediate vicinity of the affected facility.
c.Where agreements with offsite agencies dictate,
systems alert the public outside the site
boundary.
P/E10.13 Communications systems are in place to support
management and tracking of evacuation of facility
personnel, personnel accountability and assembly.
P/E10.14 Installed voice communications systems
adequately accomplish notification and information
exchange processes.
a.Reliable equipment exists for communications with
emergency organizations and response personnel.
b.Dedicated primary and backup voice communications
links are provided between key emergency response
facilities and sufficient non-dedicated voice
communication links are provided to access offsite
organizations.
c.Mobile and commercial phone lines are available.
P/E10.15 Continuous, effective, and accurate
communications among response components and/or
organizations (e.g., event scene responders,
emergency managers, response facilities, and workers
who have taken protective actions) is reliably
established and maintained throughout an OE.
Documentation/Reports
P/E10.16 Notifications and key communications are
properly documented and displayed in emergency
response facilities.
P/E10.17 A formal system is in place to record,
sequence, validate, and track the flow and
chronology of emergency information.
P/E10.18 Logs are maintained and other record-keeping
methods utilized to support post-event analysis,
report production, and a legally defensible
chronology of notification and communications
activities.
P/E10.19 All reports and releases are reviewed for
classified or Unclassified Controlled Nuclear
Information (UCNI) prior to being provided to
personnel without security clearances, entered into
unclassified databases, or transmitted using non-
secure communications equipment.
P/E10.20 Following termination of the emergency
response, and in conjunction with the Final
Occurrence Report, each activated EMT develops and
submits a final report on the emergency response to
the ED for submission to the Associate
Administrator, Office of Emergency Operations.
D.4.6 Consequence Assessment
Performance Goal:
Estimates of onsite and offsite consequences of actual or
potential releases of hazardous materials are computed and
assessed correctly and in a timely manner throughout the
emergency. Consequence assessments are: integrated with
event classification and protective action decision-making;
incorporated with facility and field indications and
measurements; and coordinated with offsite agencies.
Evaluation Criteria [RESPONSE Functions]:
Process
P/E11.1A Timely Initial Assessment (TIA) of the actual or
potential consequences of an emergency is performed
effectively and efficiently, shortly after initial
classification, using any available real-time event
and meteorological data to provide an event-specific
estimate of consequences.
P/E11.2Timely in-depth assessments of event consequences
are made continuously throughout an emergency.
a.Consequence estimates performed by hand and/or
from computer calculations are accomplished in a
timely and efficient manner throughout the
emergency to adequately assess the actual or
potential onsite and offsite consequences.
b.Assessments are updated when there are actual and
projected changes in facility status, release
conditions, or meteorology, or when there are data
from field monitoring teams.
c.Different models, assumptions, and input data are
used, as available, to add to the understanding of
the event and its consequences.
d.Indicators (e.g., system pressures, flow rates,
radiation levels, release rates, etc.), necessary
to continually assess the consequences of the
emergency events/conditions, are identified and
monitored.
P/E11.3Consequence assessment process is integrated with
processes for categorizing an event as an emergency,
determining the appropriate emergency class,
protective action decision-making, and locating and
recovering materials.
P/E11.4Provisions are made for requesting support from the
DOE radiological emergency response assets
(e.g., AMS or NARAC) to assist in accident and
consequence assessments as well as to estimate the
integrated impact of a hazardous materials release
to onsite and offsite populations.
P/E11.5Facilities have access to NARAC or have procedures
in place to activate or request NARAC capabilities:
a.If a facility has the potential for an OE
classified as a General Emergency, connectivity to
NARAC capabilities is established and procedures
are in place to use the NARAC capability
effectively as part of near real-time consequence
assessment activities for the mode (primary,
backup, corroborating) selected by the facility.
b.If a facility has the potential for an OE
classified as a Site Area Emergency, procedures
are in place to activate or request NARAC
capabilities and to use those capabilities as part
of near real-time consequence assessment
activities.
P/E11.6For facilities with access to NARAC, or that have
procedures in place to activate or request NARAC
capabilities, meteorological data and information on
source terms for actual or potential releases of
hazardous materials to the atmosphere are available
or can be made available to NARAC in a timely manner
to facilitate near real-time computations.
P/E11.7Natural phenomena (e.g., tornados, floods, severe
wind, ice, or snow), which may result in or
exacerbate an emergency condition at the facility,
operation, and/or activity, are monitored.
P/E11.8A formal document control system is implemented
during an emergency to record, sequence, validate,
and track the flow and chronology of information.
P/E11.9A primary function of the consequence assessment
process for releases of biological agents, either
observed or unobserved, involves the confirmation
that a release to the environment from a biosafety
facility has occurred.
Consequence Calculations
P/E11.10 Tools used in consequence assessment, such as
system hardware and software for meteorological
monitoring and dose modeling, etc., are available,
reliable, calibrated, and consistent with DOE and
industry standards.
P/E11.11 The type of hazard and source term for the
release of a hazardous material is successfully
determined either with available and reliable
facility system parameters and effluent monitors or
with data that is not normally monitored and
measured.
a.Data for source term estimates is available from
reliable sources (e.g., stack or process flow
rates, concentrations, tank volumes, and
containment or process building leak rates).
b.Methodology for determining the type of hazard and
source term is compatible with
instrumentation/monitor values (e.g., engineering
units, range, and conversion factors).
c.Instruments used for detection of chemical
releases to the atmosphere have sufficient range
to accurately determine the concentration of the
released chemical(s) in air versus the chemical
Protective Action Criterion (PAC).
d.Indicators that are not continually monitored
(e.g., chemical analyses of fluids, contamination
levels, etc.,) are sampled to identify the
particular indicators to be continually monitored
to assess the consequences of potential events, in
addition to occurring events, by identifying
trends, relationships, etc., that would indicate
degrading conditions.
P/E11.12 Adequate meteorological information is obtained
for use in transport and dispersion calculations to
project the consequences of the hazardous material
release to the environment, onsite and offsite.
P/E11.13 Onsite and offsite receptors of interest are
identified quickly and are readily available to
emergency managers (e.g., receptor locations at the
facility and site boundaries, to or beyond the EPZ
boundary, and populations with special needs).
P/E11.14 Consequence estimates for actual or potential
releases of hazardous materials:
a.Are made in a timely manner, efficiently, and
accurately (i.e., consistent with the accuracy of
the input data);
b.Account for releases from ground level and
elevated release points, or monitored and
unmonitored pathways; make use of post-accident
analysis results and field monitoring team data,
as appropriate;
c.Include calculations of radioactive dose or toxic
chemical exposure for the external, inhalation,
and ingestion pathways, as appropriate;
d.Are provided for receptor locations at the
facility and site boundaries, to or beyond the EPZ
boundary, and for populations with special needs;
and
e.Use the appropriate facility-specific PAC, which
is identified and readily available to consequence
assessment teams for estimating health effects at
a specified distance from the event.
Field Measurements
P/E11.15 Field teams receive initial, conservative
estimates of projected consequences in a timely
manner prior to being dispatched for sampling,
monitoring, and plume tracking activities.
P/E11.16 Field sampling and monitoring activities are
used to verify, update, and refine the source term
and projected consequences through coordination with
those responsible for consequence estimates.
P/E11.17 Field teams (i.e., radiological and non-
radiological field teams) successfully accomplish
field monitoring and plume tracking within and
beyond the EPZ, and, similarly, verify the absence
of consequences in specific areas.
P/E11.18 As available, data from environmental
monitoring programs is used to support consequence
assessment, including data from installed air
monitors, area radiation monitors, and in-plant
surveys.
Coordination
P/E11.19 Effective coordination is established with
Federal, Tribal, State, and local organizations to
estimate the impact of the release on the public and
the environment, locate and track hazardous
materials released, and locate and recover
materials, especially those with national security
implications.
P/E11.20 Field monitoring and data collection by
facility and site teams, State and local teams, and
Federal teams is coordinated to facilitate exchanges
and correlation of information.
P/E11.21 Assessments and analyses are clearly
communicated to offsite emergency management
decision makers.
a.Engineering units used in facility/site
consequence assessment are understood and
compatible with the units used by offsite
emergency response authorities.
b.Differences in modeling methods are well
understood by onsite and offsite emergency
response personnel.
Evaluation Criteria [PROGRAMMATIC Functions]:
P11.22 A formal Quality Assurance Program is implemented
and maintained for control of the tools used in
consequence assessment, such as the meteorological
monitoring system hardware and software and dose
modeling hardware and software.
D.4.7 Protective Actions and Reentry
Performance Goal:
Protective actions are promptly and effectively implemented
or recommended for implementation, as needed, to minimize
the consequences of emergencies and to protect the health
and safety of workers and the public. Protective actions
are implemented individually or in combination to reduce
exposures to a wide range of hazardous materials.
Protective actions must be reassessed throughout an
emergency and modified as conditions change. Reentry
activities must be planned, coordinated, and accomplished
properly and safely.
Evaluation Criteria:
Protective Action Decision-Making
P/E12.1All emergency response activities, including search
and rescue, incident mitigation activities, field
monitoring, and reentry, are planned and controlled
with a focus on health and safety of emergency
responders within pre-planned protective action
exposure guidelines.
P/E12.2Applicable PACs are used in protective action
(e.g., sheltering, evacuation) decision-making for
the actual or potential release of hazardous
materials to the environment.
a.For radioactive materials, Protective Action
Guides (PAGs), promulgated by the Environmental
Protection Agency (EPA), are used.
b.Listed in order of preference, PACs used for toxic
chemicals are: Acute Exposure Guideline Levels
(AEGLs), promulgated by the EPA; Emergency
Response Planning Guidelines (ERPGs), published by
the American Industrial Hygiene Association; and
Temporary Emergency Exposure Limits (TEELs),
developed by DOE.
c.For hazardous biological materials, PACs are
considered exceeded and immediate protective
actions are required for any actual or potential
release of agents or toxins outside of secondary
containment barriers. Long-term PACs are
specified by State or local public health
officials.
P/E12.3Protective actions reflect a conservative assessment
of the level of health effect and extent of
potentially affected/impacted area and populations.
P/E12.4The notification and implementation of onsite PAs
and notification of offsite PARs are made in a
timely, efficient, and unambiguous manner confirmed
and monitored by the ERO.
P/E12.5Initial onsite PAs and offsite PARs are linked to
facility-specific OE event classification criteria
[i.e., EALs]
P/E12.6Initial onsite PAs and offsite PARs are linked to
facility-specific biological OE event recognition
and categorization criteria.
P/E12.7Protective actions are implemented individually or
in combination to reduce exposures to a wide range
of hazardous materials.
P/E12.8Modifications to initial protective actions are
developed and implemented based on updated and
refined data generated from the continuous
consequence assessment process.
P/E12.9Decision makers consider other possible protective
actions for onsite and offsite populations, such as
thyroid blocking agent, chemical neutralizing
agents, water and food intervention levels,
transportation route access controls, and impromptu
respiratory protection.
P/E12.10 Onsite PA decision-making is coordinated with
site organizations such as security and safety.
a.Security and law enforcement measures implemented
during a physical attack that impact worker and
responder access and egress (e.g., lockdown) are
coordinated with emergency management and site
security.
b.The identification of necessary PPE is coordinated
with emergency management and safety
professionals, including industrial safety,
industrial hygiene, health physics, and fire
protection engineering.
Onsite - Protective Actions (PAs)
P/E12.11 Plans are followed for the timely evacuation
and/or sheltering of onsite personnel, along with
provisions to account for employees after emergency
evacuation has been completed.
P/E12.12 Emergency evacuations for site personnel:
a.Evacuation route selection and logistical details
are implemented promptly and efficiently;
b.Multiple evacuation egress routes provide options
based upon release type and wind direction;
c.Evacuation routes avoid hazards, are familiar to
site personnel, and are coordinated with offsite
authorities;
d.The reception/relocation center is sufficient to
accommodate the expected number of personnel; and
e.Adequate personnel are assigned to control
evacuees and are kept aware of changes in onsite
protective action modifications.
P/E12.13 Accountability in emergency evacuations for
site personnel:
a.Trained and assigned individuals assume and carry
out responsibilities for building or facility
accountability in the event of personnel
evacuation;
b.Initial accounting for all evacuated personnel is
completed in a timely manner to support initial
search and rescue activities; and
c.Accountability is continued to support ongoing
search and rescue activities following an
emergency evacuation.
P/E12.14 Provisions are implemented to protect workers,
covered by 29 CFR 1910.120, involved in response and
cleanup. This includes measures to ensure that
security, fire, medical, and other response
personnel are protected from exposure to hazards
during the course of their movements while
supporting response.
P/E12.15 Habitability of onsite facilities, including
emergency facilities, is periodically determined
using dosimetry and survey instruments, and
relocation/evacuation measures are taken, if
necessary.
P/E12.16 Actions that may be taken to increase the
effectiveness of protective actions (i.e., HVAC
shutdown during sheltering) are implemented in a
timely and efficient manner.
P/E12.17 Access to and egress from actual or potentially
contaminated areas, or the site, is monitored and
controlled.
a.People, vehicles, and equipment are effectively
monitored before leaving contaminated areas and
the site, if possible, and also upon arrival at
designated decontamination, relocation, or
assembly areas.
b.Sufficient staffing and equipment are available to
activate designated monitoring locations.
P/E12.18 Emergency facilities, equipment, personnel, and
implemented methods and criteria provide effective
decontamination of personnel and equipment for
various levels and types of contamination
(e.g., skin contamination).
Offsite - Protective Action Recommendations (PARs)
P/E12.19 Timely PARs, such as sheltering, evacuation,
relocation, and food control, are made to
appropriate Tribal, State, or local authorities.
P/E12.20 Candidate PARs are coordinated with offsite
authorities and well-defined geographic areas for
sheltering and evacuation, special needs areas or
special populations, and evacuation routes are
readily available.
P/E12.21 Ingestion pathway PARs are formulated, when
appropriate, and communicated to offsite
authorities.
Reentry Activities
P/E12.22 Reentry and approval of extended dose or
exposure limits is within the authority and
responsibility of the ED.
P/E 12.23 Facility personnel estimate exposure to
hazardous materials to protect workers and the
public during reentry and recovery activities.
P/E12.24 Reentry activities are performed safely and
efficiently, with specific team composition
(e.g., minimum of one medically trained member) and
equipment that accomplishes the mission.
P/E12.25 Reentry planning addresses the following:
conduct of operations during reentry; range of
hazardous materials which may be encountered; hazard
control procedures; type and nature of potential
safety failures; guidelines for prioritization of
reentry activities; team selection, personnel
safety, job planning, communications during reentry;
record keeping; and provisions for backup to every
reentry.
P/E12.26 Reentry planning includes contingency planning
to ensure the safety of reentry personnel, such as
planning for the rescue of reentry teams. The
reentry plan must include a hazards/safety briefing,
consistent with Federal, Tribal, State, and local
laws and regulations, for all individuals involved
in reentry.
P/E12.27 Exposure criteria are established and available
for each type of reentry activity, including search
and rescue, and repair. 10 CFR 835, Subpart N,
limits are observed for radiological events, such as
lifesaving, protection of health and property, and
recovery of deceased. Volunteers are used for high-
risk situations.
P/E12.28 Responders involved in reentry receive pre-
reentry hazards/safety briefings prior to emergency
response activities and post-reentry briefings
consistent with Federal, Tribal, State, and local
laws and regulations.
Record Keeping
P/E12.29 Records of personnel exposures to hazardous
materials (radioactive, chemical, and biological)
are effectively controlled, monitored, and
maintained.
P/E12.30 Names of individuals surveyed, the extent of
any contamination found, the instruments used and
the methods employed, and results of any
decontamination efforts are recorded.
P/E12.31 Contaminated individuals are scheduled for
follow-up actions (e.g., subsequent whole body
counts and/or bio-assays).
D.4.8 Emergency Medical Support
Performance Goal:
Medical support for contaminated or injured personnel is
planned and promptly and effectively implemented.
Arrangements with offsite medical facilities to transport,
accept, and treat contaminated, injured personnel are
documented.
Evaluation Criteria [RESPONSE Functions]:
General
P/E13.1Provisions for response to emergency medical
situations and medical treatment of injured
personnel are implemented.
P/E13.2Medical treatment is provided for mass casualty
situations (Cf. DOE O 440.1A).
P/E13.3Onsite personnel who respond to a medical emergency
show proficiency in first aid or emergency medical
treatment comparable with that of any offsite team
employed and similarly equipped.
P/E13.4Employee medical records and treatment history are
readily available and accessed as needed.
P/E13.5Onsite and offsite coordination:
a.Treatment protocols are coordinated among onsite
and offsite mutual aid response units.
b.Offsite and onsite medical support services and
capabilities are effectively integrated.
c.Standing orders/protocols ensure that patients are
transported to the receiving facility best
equipped to provide the appropriate level of care
for the patients condition.
d.Onsite and offsite medical communications systems
are compatible and effective.
P/E13.6Ambulance crews initiate communications with
receiving medical facilities while en route.
P/E13.7Procedures are in place in biosafety facilities that
allow rapid and effective communications among
public health officials, emergency rooms, law
enforcement, and emergency management officials
about unusual biological events.
P/E13.8During an event involving the release of hazardous
biological material, medical personnel assume the
role of primary responders. Medical personnel
assist in release detection/confirmation,
consequence assessment, and development of
protective actions.
P/E13.9Security clearance issues do not impede medical
treatment or transport of injured personnel.
Contaminated Injured
P/E13.10 Medical support for contaminated or injured
personnel is promptly and effectively implemented.
P/E13.11 Appropriate recognition and emphasis is focused
on medical treatment versus radioactive or chemical
contamination for contaminated/injured personnel;
proper and effective decisions are made.
P/E13.12 Onsite and offsite medical facilities are
outfitted and staffed to utilize specialized
equipment and supplies specific to onsite hazards.
P/E13.13 Immediate, effective onsite first aid and
emergency medical treatment is provided for injured
workers, including those with hazardous material
contamination:
a.Onsite radiation protection, industrial hygiene
personnel, and infectious disease specialists are
properly equipped to assist medical and Emergency
Medical Service (EMS) staff in performing patient
survey, decontamination, contamination and
exposure control, urine and fecal analysis, and in-
vivo counting methods.
b.Proper contamination control procedures are
implemented in handling injured and contaminated
personnel; and
c.Decontamination facilities are available and
adequately equipped.
P/E13.14 Personnel, vehicles, facilities, and equipment
are adequate for treating and transporting injured,
contaminated, or exposed individuals in a safe and
effective manner.
a.Onsite and offsite medical and emergency medical
technician personnel use required equipment for
assessing patient conditions, including PPE and
medical service protective clothing;
b.Exposure and contamination information is sent
with victims, and expert technical support is
provided to the receiving hospital(s); and
c.The site takes responsibility for removal of
contaminated material in offsite medical
facilities or vehicles.
P/E13.15 Additional medical assistance and treatment
procedures, and associated points of contacts, are
accessed, as necessary, including: search and rescue
resources, REAC/TS assistance, Public Health Service
coordination, long-term longitudinal health testing,
chelation, handling contaminated remains, and other
sophisticated medical procedures.
Evaluation Criteria [PROGRAMMATIC Functions]:
P13.16 Arrangements with offsite medical facilities to
transport, accept, and treat contaminated, injured
personnel are established, documented, and
periodically reviewed.
P13.17 The sharing of patient information between onsite
and offsite health care providers during
emergencies, consistent with the requirements of
Health Insurance Portability and Accountability Act
of 1996 (42 USC 300), is coordinated in advance.
P13.18 Onsite and offsite medical personnel are offered
information and training on facility-specific
hazardous materials and offered opportunities for
participation in drills and exercises.
P13.19 Biosafety surveillance plans for detecting unusual
medical events are established onsite and specific
responsibilities for surveillance and reporting are
identified. The veterinary profession is involved
in surveillance activities, as appropriate. Key
indicators and medical surveillance baselines for
agent/toxin are effectively implemented.
P13.20 An information system is installed at biosafety
facilities for patient monitoring, management, and
tracking.
P13.21 Key indicators and medical surveillance baselines
for facility-specific agents/toxins are provided to
offsite medical surveillance programs for detecting
unusual medical events that may have resulted from a
release at a DOE/NNSA biosafety facility.
D.4.9 Emergency Public Information
Performance Goal:
Emergency Public Information (EPI) provides accurate,
candid, and timely information to workers, news media, and
the public during an emergency to establish facts and avoid
speculation. EPI efforts are coordinated with DOE and NNSA
(if appropriate); Tribal, State, and local governments; and
Federal emergency response organizations, as appropriate.
Workers and the public are informed of emergency plans and
planned protective actions before emergencies.
Evaluation Criteria [RESPONSE Functions]:
General
P/E14.1Information distributed by EPI to workers, site
personnel, and the public during an OE is:
a.Accurate, candid, and understandable;
b.Current and timely;
c.Provided to ensure the health and safety of
workers and the public;
d.Provided to establish facts, and avoid rumors and
speculation;
e.Responsive to public concern and information
needs; and
f.Consistent with the requirements of the Freedom of
Information Act and the Privacy Act.
P/E14.2The Cognizant Field Element public affairs director
or his designee, responsible for EPI review and
dissemination, approves initial news releases or
public statements.
P/E14.3Following initial news releases and public
statements, updates are coordinated with the
DOE/NNSA Cognizant Field Element public affairs
director and the HQ ED.
Functions/Staffing
P/E14.4Functions and staff of the EPI organization:
a.Functions of the EPI during an OE response include
information collection, coordination, production,
dissemination, and monitoring and analysis of
media coverage, public concerns, and information
needs.
b.Functions and staffing are consistent with the
nature, severity, duration, and public and media
perception of the event or condition.
c.Trained spokespersons provide support in media
interface.
d.A news writer and other trained personnel provide
support in media services, public inquiry, media
inquiry, management and administrative services,
and media monitoring.
P/E14.5Rumors and misinformation are detected, controlled,
and corrected; accurate information disclaiming
rumors and correcting misinformation is incorporated
in media briefings and press releases as necessary.
P/E14.6Communications with the media and public are timely
and responsive to public concerns.
a.Information released to the public through the
news media regarding the emergency is accurate and
relevant.
b.An initial press statement is released as soon as
possible, but within one hour of event
categorization.
c.Frequency and content of news conferences are
consistent with information needs of the public
and media.
d.Press briefings are held with regular frequency
and whenever new or breaking information is
available concerning emergency conditions,
protective actions, or response.
e.Technical briefers are utilized and are
knowledgeable and effective in communicating with
the news media.
P/E14.7EPI staff is proactive in obtaining emergency
information from the facility command center or EOC.
P/E14.8Medical personnel associated with the biosafety
program are involved in the development of materials
to be used in news releases to ensure that
characterization of the hazard is conveyed
accurately.
P/E14.9Public announcements in areas involving classified
information or unclassified controlled information:
a.The appropriate official (e.g., DC) reviews news
releases or announcements before release to the
public to ensure that no information is provided
that may present a security risk.
b.Sufficient publicly releasable information is
provided to adequately explain the emergency
response and protective actions required for the
health and safety of workers and the public.
Joint Information Center (JIC)
P/E14.10 The JIC is established, directed, and
coordinated by a senior DOE or NNSA Cognizant Field
Office public affairs manager or alternate.
P/E14.11 The designated JIC location:
a.Is available, equipped, maintained and controlled
to accommodate members of the news media, DOE,
contractor, and offsite agency representatives,
and to facilitate the preparation and coordination
of emergency information release to the public
through the news media.
b.Provides adequate space, equipment, communications
lines, security provisions, and information
resources to accommodate personnel (both media and
staff) and to accomplish required functions.
P/E14.12 JIC functions and staffing:
a.The JIC is adequately staffed with personnel
trained to serve as spokesperson and news writer.
b.Persons with technical expertise related to the
emergency and with spokesperson training are
assigned to the JIC.
c.Personnel are assigned to the JIC to provide
support in media services, public inquiry, media
inquiry, JIC management and administrative
activities, and media monitoring.
d.The JIC supports response to public inquiries in a
timely manner.
e.The JIC has provisions in place to detect,
correct, and control rumors and misinformation.
P/E14.13 An alternate JIC is available in the event that
the primary JIC becomes uninhabitable.
P/E14.14 JIC access control is adequate and there is a
means to readily identify media representatives and
staff.
P/E14.15 Prepared relevant information concerning
affected facilities, emergency plans, hazards, and
logistics is provided to news media in the JIC.
P/E14.16 Appropriate visual aids are available and
utilized for briefing news media regarding events,
impacted areas, consequences and protective actions.
Offsite Coordination
P/E14.17 The management team and outside agency
representatives effectively, openly, and readily
share and coordinate information.
P/E14.18 An EPI communications system is established
among DOE/NNSA HQ, the Cognizant Field Element, and
on scene locations.
P/E14.19 Public information functions and efforts during
the emergency are coordinated with DOE HQ, other
Federal agencies, and Tribal, State, and local
government organizations and are a part of Federal
emergency response plans, as appropriate.
P/E14.20 Information (written and verbal) that is to be
released to the news media is coordinated with DOE,
and other Federal, state, tribal and local response
organizations, as appropriate.
P/E14.21 The DOE/NNSA Cognizant Field Office public
affairs director and HQ ED are notified of all
DOE/NNSA emergency public information actions.
These notifications are made as soon as practicable.
P/E14.22 A public information officer is assigned to a
facility/site or activity emergency response team
deployed offsite to provide mutual aid to a
significant response.
Evaluation Criteria [PROGRAMMATIC Functions]:
P14.23 An EPI Plan, which can cover more than one facility
on a site, provides the following:
a.Identification of personnel, resources,
facilities, and coordination procedures necessary
to provide emergency public information;
b.Training and exercises for personnel who will
interact with the media;
c.A methodology for informing workers and the public
of DOE/NNSA emergency plans and protective
actions, before and during emergencies;
d.Coordination of public information efforts with
local, State, and Tribal governments, and Federal
emergency response plans, as appropriate.
P14.24 The EPI program has provisions for establishing a
media center to operate as the single source of
information during an OE. [A media center is a
designated location where the DOE/NNSA Cognizant
Field Element and contractor personnel can conduct
the necessary briefings and press conferences
regarding an OE at the facility.]
P14.25 For Hazardous Material Program facilities/sites, the
EPI program has provisions to establish a Joint
Information Center (JIC), where multiple
jurisdictions gather, process, and disseminate
public information during an OE.
P14.26 The EPI program is integrated with facility/site
emergency management program plans and procedures.
P14.27 Prior to emergencies, workers and site personnel are
informed of emergency response plans, response
capabilities, and planned protective actions.
P14.28 Continuing education is provided to the area news
media for the purpose of acquainting media with the
facility, management personnel, facility/site
hazards, emergency plans, and points of contact.
P14.29 In coordination with Tribal, State and local
governments, information is disseminated
periodically to the public regarding facility
hazards, how they will be alerted and notified of an
emergency, what their actions should be in the event
of an emergency, and points of contact for
additional information.
P14.30 Internal and external organizational relationships
for EPI are documented and maintained in the public
information program.
P14.31 A list of 24-hour media points-of-contact is
available and maintained current.
D.4.10 Termination and Recovery
Performance Goal:
An OE is terminated only after a predetermined set of
criteria has been met and termination has been coordinated
with offsite agencies. Recovery from a terminated OE
includes: communication and coordination with Tribal, State,
and local government and other Federal agencies; planning,
management, and organization of the associated recovery
activities; and ensuring the health and safety of the
workers and public.
Evaluation Criteria:
Termination
P/E15.1The decision to terminate emergency response for an
OE is made by the site ERO and is coordinated with
all principle participating response organizations
(i.e., local, State, Tribal, DOE HQ, other
participating Federal agencies). Internal and
external communications that are associated with
termination are performed.
P/E15.2The decision to terminate an OE not requiring
classification formally announces or acknowledges
that the situation is stabilized and that the
response activity is ending or has been
substantially scaled back. Termination criteria are
observables associated with the event/condition.
P/E15.3The decision to terminate an OE requiring
classification is based on the need for the ERO to
remain fully active to monitor and manage the
situation and is a declaration that a decision has
been reached that the full ERO is no longer needed
and the ERO may now begin to reduce its support.
The termination criteria represent decision criteria
to be satisfied.
P/E15.4An approved, predetermined set of criteria for
terminating an OE requiring classification [e.g., an
airborne release of (or loss of control over)
hazardous material] is met. Selected termination
criteria may include the following:
Recovery plan is developed
Recovery staff is identified
Event scene/facility is in stable condition
Event scene/facility is isolated and can be
preserved
Resources are available to begin recovery
activities
All releases of hazardous materials are ended or
below level of regulator concern
Accountability of all personnel is complete
Contaminated areas are identified, isolated and
secured
All injured and contaminated personnel have been
treated and transported
Notification of next-of-kin of victims
Protective actions have been adjusted according to
extended conditions
Recovery manager and staff have been fully briefed
by the Emergency Director
Notifications are made to DOE, other Federal,
Tribal, State, and local response organizations.
P/E15.5An approved, predetermined set of criteria for
terminating an OE not requiring classification are
met. Selected general termination criteria that
apply may include the following:
Recovery plan is developed
Recovery staff is identified
Event scene/facility is in stable condition
Event scene/facility is isolated and can be
preserved
Resources are available to begin recovery
activities
Next-of-kin of victims have been notified
Recovery manager and staff have been fully briefed
by the ED
Notifications are made to DOE/NNSA, other Federal,
Tribal, State, and local response organizations.
P/E15.6Additional OE-specific criteria for emergencies not
requiring classification are met.
P/E15.7Termination criteria for a hazardous biological
material release OE are similar to criteria for an
OE that requires classification, such as the release
of toxic or radioactive materials. The decision to
terminate a biological OE is based on the perceived
need for the ERO to remain fully active to monitor
and manage the situation. Termination is a
declaration that a decision has been reached that
the full ERO is no longer needed and the ERO may now
begin to reduce its support.
P/E 15.8 Facility personnel estimate exposure to
hazardous materials to protect workers and the
public during reentry and recovery activities.
Recovery
P/E15.9Prior to terminating the emergency response, the
site ERO establishes the recovery organization and
determines the resources needed to begin recovery
operations.
P/E15.10 The beginning of the recovery phase is marked
by the termination decision and subsequent
notifications that an event no longer constitutes an
OE.
P/E15.11 The recovery plan to return the affected
facility/area to normal operations following the
termination of the OE is developed by the recovery
organization, and depends on (i.e., is commensurate
with) the severity and nature of the emergency event
or condition.
P/E15.12 The plan and recovery organization address the
following areas, as needed:
Dissemination of information to Federal, Tribal,
State, and local organizations regarding the
emergency and possible relaxation of protective
actions;
Notifications associated with termination;
Accident assessment and investigation;
Recovery planning and scheduling;
Establishment of a recovery organization;
Repair and restoration;
Planning for cleanup and decontamination;
Waste management;
Regulatory (e.g., environmental) compliance;
Security;
Crime scene investigation;
Communication and notifications;
Development and approval of recovery procedures;
Repair or replace emergency equipment, replenish
consumables;
Health and safety (e.g., medical follow-up
planning);
Reporting requirements; and
Criteria for the resumption of normal operations.
P/E15.13 Accident assessment and investigation are
performed, consistent with event severity, including
root cause analysis, accident reporting, event
documentation collection, assessment of facility
condition, and assessment of contamination effects,
if relevant.
P/E15.14 Root cause(s) of the emergency are investigated
and corrective action(s) to prevent recurrence are
developed according to Departmental requirements.
P/E15.15 Recovery activities are coordinated with
Federal, Tribal, State, local and other agencies,
and are in compliance with their requirements.
APPENDIX E. Systematic Approach for Performing Self-
Assessments of Emergency Management Programs
The purpose of this appendix is to provide guidance for the
development and conduct of systematic self-assessments of
emergency management programs. DOE O 151.1C requires
DOE/NNSA facility/site managers to establish and maintain an
internal assessment program, including developing and
conducting self-assessments, to provide assurances of the
readiness of emergency response capabilities. The emergency
management program administrator at each facility/site is
responsible for developing self-assessment programs and for
implementing them according to a predetermined schedule.
The Cognizant DOE/NNSA field element manager is required to
review contractor self-assessment programs annually to
ensure compliance with DOE/NNSA directives and policies.
Typically, emergency management self-assessment has been
considered a single internal assessment or self-evaluation
event. On the other hand, a systematic self-assessment
approach is a continuous assessment process and is based on
the understanding that emergency management includes a set
of routine functions and activities that must be maintained
on an ongoing basis for readiness assurance. Systematic
self-assessment provides a holistic assessment of these
functions and activities to help maintain and sustain
emergency management programs.
This appendix describes an approach for ongoing evaluation
of contractor emergency management programs. Evaluation is
executed through a scheduled self-assessment program,
consisting of individual self-assessment activities that are
accomplished throughout the year. These assessment
activities should focus on a full range of emergency
management functions and areas. Results of these activities
provide information that management can use in assessing
overall program readiness and effectiveness. Assessment
activities carried out over the annual assessment period
support program managers in maintaining a state of program
readiness. Performance data from these activities
contribute to trending and analyses that focus management
attention and resources on areas of greatest need.
Some self-assessment activities may occur once a year, while
others will be assessed more frequently to ensure that
timely corrective actions contribute to program improvement.
For example, one type of self-assessment activity is
emergency response facility tours and equipment tests.
These are designed to verify the state of readiness of the
Emergency Response (ER) facilities. It may be appropriate
to conduct this self-assessment activity at some sites
through monthly visual inspections and more frequently at
other sites. Other assessment activities will occur on a
random basis, for example, the conduct of a no-notice
exercise or a management tour of a facility. Additionally,
when emergency response teams are activated to respond to an
actual emergency the effectiveness of that response should
be self-assessed and the results factored into the annual
program assessment.
The systematic self-assessment approach suggests an annual
management review of the results of the self-assessment
activities, including emergency management performance
trends and data concerning performance problems and their
causes. Section E.3 of this appendix provides a suggested
approach for an annual Management Review of the emergency
management program.
E.1 Overview of an Emergency Management Self-Assessment Model
for DOE/NNSA Sites
Following are generic examples that illustrate a systematic
approach to emergency management program self-assessment.
Self-assessment activities should be conducted at a minimum
annually but staggered so that assessment is ongoing and
trending of performance data can be spread over the annual
assessment period. Section E.1.1 lists potential self-
assessment activities and an approach for documenting their
conduct. Section E.1.2 illustrates an example schedule of
such activities.
E.1.1 Annual Self-Assessment Activity Summary
The following table is an example summary of self-assessment
activities scheduled for the year:
Self-Assessment Activity Scheduled Date Completed Name
of Assessor
Hazards Survey (HS)/
Emergency Planning
Hazards Assessment
(EPHA) Review __________ _______________
__________________
HS/EPHA Facilities Tour __________ _______________
__________________
Emergency Response
Organization (ERO)
Qualification review __________ _______________
__________________
ERO Quarterly Drill __________ _______________
__________________
ER Plans/Procedures Review __________ _______________
__________________
ER Facilities Tour and
Equipment Test __________ _______________
__________________
Communications Test __________ _______________
__________________
ER Drill Program Review __________ _______________
__________________
Corrective Action
Timeliness Review __________ _______________
__________________
Training Documents Review __________ _______________
__________________
Offsite Interface Review __________ _______________
__________________
Training Effectiveness
Review __________ _______________
__________________
E.1.2 Emergency Management Program Management Schedule
By staggering the conduct of self-assessment activities
throughout the year, emergency management program managers
can use the self-assessment activities to evaluate system
health throughout the year. To accomplish this effectively,
self-assessment activities can be included in a Program
Management schedule. A program management schedule is a
simple management tool that supports the manager in
planning, monitoring and maintaining the program. Managers
should establish and document in advance a calendar of
assessment activities that are to be carried that year. The
program manager should ensure that assessments are carried
out, regularly review the results, and track trends to
determine program progress toward meeting performance goals.
The following is a simplified example of a schedule focusing
on assessment activities throughout the year.
Sample Program Management Schedule
Scheduled Conducted
January:
HS/EPHA Facility Tour ________ ________
ERO Drill Program Review ________ ________
Communications Test ________ ________
February:
ER Facilities Tour and
Equipment Operability Test(s) ________ ________
ERO Training Effectiveness Review ________
________
(Conducted by training group)
March:
Annual Internal Program Evaluation ________ ________
HS/EPHA Facility Tour ________ ________
ERO Quarterly Drill ________ ________
HS/EPHA Review ________ ________
April:
No-Notice Exercise ________ ________
ER Facilities Tour and
Equipment Operability Test(s) ________ ________
Communications Test ________ ________
Corrective action timeliness review ________
________
May:
Management Review of Program ________ ________
HS/EPHA Facility Tour ________ ________
ER Drill program Review ________ ________
June:
ER Facilities Tour and
Equipment Operability Test(s) ________ ________
ERO Qualification Review ________ ________
Offsite Interface Review ________ ________
ERO Quarterly Drill ________ ________
July:
HS/EPHA Facility Tour ________ ________
ERO Training Effectiveness Review ________
________
Training Documents review ________
________
August:
ER Facilities Tour and
Equipment Operability Test(s) ________ ________
Communications Test ________ ________
September:
HS/EPHA Facility Tour ________ ________
Annual Exercise ________ ________
ERO Quarterly Drill ________ ________
October:
ER Facilities Tour and
Equipment Operability Test(s) ________ ________
Communications Test ________ ________
November:
HS/EPHA Facility Tour ________ ________
Corrective action timeliness review ________
________
December:
ER Facilities Tour and
Equipment Operability Test(s) ________ ________
ER Plans/Procedures review ________
________
ERO Quarterly Drill ________ ________
E.2 Example Self-Assessment Activities
The remaining sections of this appendix describe some (not
all) of the self-assessment activities that are listed
above. The conduct of other assessment activities, such as
ERO drills, communications tests, training documents review
and corrective action timeliness review, are either
described in other chapters of the DOE G 151.1-series,
Emergency Management Guide (EMG), or should be self-evident
to program managers. The following selected examples are
intended to convey representative content for the self-
assessment activities indicated.
E.2.1 Hazards Survey and Hazards Assessment Facilities
Tour
A periodic assessment of the accuracy of facility HS and
EPHA is a critical self-assessment function. Qualified
personnel should be assigned to conduct reviews, including
walk-down of facilities to ensure that changes to facilities
are noted, evaluated, and incorporated to correct HS and
EPHA documents. Action items should be entered into the
organizations corrective action program and tracked to
completion to ensure the currency of these documents.
Results should also be included as part of the self-
assessment activity. The following table contains a sample
form to use for a facility tour that focuses on the current
validity of the EPHA and Sample Assessment Form for a
HS/EPHA Facilities Tour
Individual conducting the tour:
______________________________________________
Date and time of the tour:
__________________________________________________
Facilities and areas toured: (list areas toured)
____________________________________________________________
____________________________________________________________
____________________________________________________________
____________________________________________________________
________________________________________________
Checklist of observation points prior to and during the
tour:
__HS and EPHA assessment documents (if applicable) for
assigned facility
was reviewed before tour.
__Hazardous materials inventories documents were reviewed
prior to tour
__Inventories of hazardous materials observed during tour
__Hazardous material storage conditions
__Hazardous material use conditions
__Energy sources
__Facility boundary
__External hazards
__Transportation hazards
__Engineered controls, safeguards/safety systems
__Administrative controls
__System barriers
__System controls and protection features
Tour conclusions
__The facility was toured and conditions and controls remain
as stated in the HS/EPHA
__The facility tour was satisfactory; however the assessor
noted the following minor
discrepancies from the HS/EPHA.
__The facility tour was not satisfactory. Significant
discrepancies were noted from the
HS/EPHA. (Notify the Emergency Management
Department Manager immediately)
Tour Findings
List findings below and be as descriptive as possible.
E.2.2 Emergency Response (ER) Drill Program Review
The emergency management drill program should be included in
the annual exercise program (for evaluated drills) or
training and drills program (for training drills) and
conducted in accordance with facility procedures. Results
of evaluated drills should be captured in critiques that
identify personnel, procedure and equipment deficiencies as
well as recommendations for improvement and lessons learned.
Action items generated by the critiques should be entered
into the organizations corrective action program and
tracked to completion. Results should be included as part
of the self-assessment activity. The drill program should
also be assessed in the annual internal evaluation.
Sample form for ER Drill Program Review
Individual conducting Drill Program Review:
Date of Review:
Drill records reviewed:
Drill Title:
Date of drill:
Records reviewed:
__Drill conducted as scheduled
__Scenario is used
__Drill objectives are clearly documented
__Critique documents drill results and lessons learned
__Personnel and equipment problems are documented in
corrective action system
__Corrective and preventive actions are tracked to
completion
__Drill participants are documented
E.2.3 Offsite Interface Review
Program managers should ensure that communications that
interface with offsite organizations, especially those that
support the DOE or NNSA sites emergency response, are
received, documented and responded to. Further guidance for
program implementation is provided within the EMG. Internal
review of the offsite interface program should be performed
as part of the annual internal program evaluation using
evaluation criteria presented in this guide. This should
include review of the effectiveness of any training that the
site provides for offsite organization personnel. (See
Training Effectiveness Review below)
To ensure that communications and interface issues are
effective throughout the year, program managers may choose
to have these external interface activities reviewed more
frequently and trend results as part of self-assessment.
E.2.4 ERO Qualification Review
Program managers should ensure that a review of ERO
qualifications be conducted periodically to ensure that
response personnel are current with regard to training,
qualification, re-qualification and emergency drill/exercise
participation. ERO training requirements and corresponding
personnel training records are used to conduct this review.
The results should be included as part of the self-
assessment record.
E.2.5 Training Effectiveness Review
Periodically, qualified individuals should conduct a
comprehensive evaluation of individual training programs to
identify program strengths and weaknesses. This should
include:
Evaluation of instruction setting, materials, and
instructor performance
Feedback from trainee performance during training
Feedback from former trainees and their supervisors
Change actions are monitored and evaluated for their
applicability to development or modifications of initial
and continuing training programs, and change is
incorporated in a timely manner
Improvements and changes to initial and continuing
training are systematically initiated, evaluated, tracked
and incorporated to correct training deficiencies and
performance problems.
Changes to job scope are evaluated to determine need for
development or modification of initial and continuing
training programs
Subcontracted training is evaluated for its contribution
to meeting job performance requirements and to ensure
that its quality is consistent with the facility training
standards.
The results should be included as part of the self-
assessment activity
E.2.6 Emergency Response (ER) Facilities Tour and
Equipment Operability Test
Assessment and trending of emergency response facilities and
equipment performance relies upon the output of drill and
exercise critiques, operability tests, and facility tours to
identify potential issues to the corrective action program
database. Program managers should review data from these
sources annually to determine if trends in performance can
be ascertained. Documentation of the results regarding
resultant findings or trends should be documented as part of
the self-assessment activity record.
Following is an example of assessment activity related to
facilities and equipment, an ER Facilities Tour reference
checklist.
Sample Form
Emergency Response Facilities Tour and Equipment Operability
Test
This form can be used as a reference guide while conducting
a tour of one or more emergency response facilities.
Significant findings or action items should be entered into
the organizations corrective action program and tracked to
completion to ensure facility readiness. Results should
also be included as part of the self-assessment activity.
Individual conducting the tour:
__________________________________________
Date and time of the tour:
______________________________________________
Facilities and areas toured: (list areas toured)
Weekly tour Monthly tour
Emergency operations center Joint
information center
Operational support center Remote
shutdown station
Control room Alternate EOC
Checklist of observation points during the tour:
__Ingress/egress points of facility clear of obstructions
__Facility is secured
__Cleanliness of facility
__Normal and emergency lighting OK
__Status boards are clean and ready for use
__Facility furnishings intact
__Procedure binders are in place at appropriate positions
__Visual aids/displays are in place
__Facility equipment fully in place and appears operational
(note: this is not a substitute
for communications tests)
__Clocks/chronological displays are in place and reading
properly
__Computer displays appear to be functioning properly (note:
this is not a substitute for
equipment test programs)
__Communications systems appear to be functioning properly
(note: this is not intended
to be a substitute for communications testing programs.
__Administrative controls
__System barriers
__System controls and protection features
Tour conclusion
__The facility was toured and conditions and controls
support activation
__The facility tour was satisfactory & activation is
supported, however the assessor noted
the following minor discrepancies.
__The facility tour was not satisfactory. Significant
discrepancies were noted. (Notify
the Emergency Management Department Manager immediately)
Tour Findings
List findings below and be as descriptive as possible:
E.3 Annual Management Review
Management should review the emergency management program at
least annually to ensure its continuing suitability,
adequacy, and effectiveness. The review should include
assessment of opportunities for improvement and the need for
changes to the emergency management program. The management
review can examine the following types of information:
__Results of internal and external audits, evaluations, and
assessments
__Results of emergency exercises performed
__Results of investigation of emergency response to actual
events
__Feedback from interested parties, including offsite
agencies, local public, site and field office management,
other site departments
__Results of emergency management self-assessment components
__Status of corrective and preventive actions for emergency
management
__Follow-up actions from pervious management reviews
__Results of reviews of HSs and EPHAs
__Results of performance trending associated with emergency
management
__Results of training effectiveness review
__Results of offsite interface review
__Changes at site that could affect the emergency management
program
__Recommendations for improvement
Output from the management review should include any
decisions and actions related to:
__Improvement of the effectiveness of the program and its
processes
__Changes to the program or its processes
__Resource needs
Records from the management review should be maintained,
including the persons participating in the review.