Protocol Questions

1.01   Application and Customization of 'Off-the-Shelf Programs'
Does the operator’s plan identify covered tasks and does it specify task-specific reevaluation intervals for individuals performing covered tasks? [Enforceable]

1.02   Contractor Qualification
Does the operator employ contractor organizations to provide individuals to perform covered tasks? If so, what are the methods used to qualify these individuals and how does the operator ensure that contractor individuals are qualified in accordance with the operator’s OQ program plan?

• Verify that the operator’s written program includes provisions that require all contractor and subcontractor individuals be evaluated and qualified prior to performing covered tasks, unless the covered task is performed by a non-qualified individual under the direction and observation of a qualified individual. [Enforceable]

1.03   Management of Other Entities Performing Covered Tasks
Has the operator’s OQ program included provisions that require individuals from any other entity performing covered task(s) on behalf of the operator (e.g., through mutual assistance agreements) be evaluated and qualified prior to task performance?

• Verify that other entities that perform covered task(s) on behalf of the operator are addressed under the operator’s OQ program and that individuals from such other entities performing covered tasks on behalf of the operator are evaluated and qualified consistent with the operator’s program requirements. [Enforceable]

1.04   Training Requirements (Initial Qualification, Remedial if Initial Failure, and Reevaluation)
Does the operator’s OQ program plan contain policy and criteria for the use of training in initial qualification of individuals performing covered tasks, and are criteria in existence for retraining and reevaluation of individuals if qualifications are questioned? [Enforceable]

1.05   Written Qualification Program
Did the operator meet the OQ Rule requirements for establishing a written operator qualification program and completing qualification of individuals performing covered tasks?

• Verify that the operator’s written qualification program was established by April 27, 2001. [Enforceable]
• Verify that the written qualification program identified all covered tasks for the operator’s operations and maintenance functions being conducted as of October 28, 2002. [Enforceable]
• Verify that the written qualification program established an evaluation method(s) to be used in the initial qualification of individuals performing covered tasks as of October 28, 2002. [Enforceable]
• Verify that all individuals performing covered tasks as of October 28, 2002, and not otherwise directed or observed by a qualified individual were qualified in accordance with the operator’s written qualification program. [Enforceable]

2.01   Development of Covered Task List
How did the operator develop its covered task list?

• Verify that the operator applied the four-part test to determine whether 49 CFR Part 192 or 49 CFR Part 195 O&M activities applicable to the operator are covered tasks. [Enforceable]
• Verify that the operator has identified and documented all applicable covered tasks. [Enforceable]

2.02   Evaluation Method(s) (Demonstration of Knowledge, Skill and Ability) and Relationship to Covered Tasks
Has the operator established and documented the evaluation method(s) appropriate to each covered task?

• Verify what evaluation method(s) has been established and documented for each covered task. [Enforceable]
• Verify that the operator’s evaluation program ensures that individuals can perform assigned covered tasks. [Enforceable]
• Verify that the evaluation method is not limited to observation of on-the-job performance, except with respect to tasks for which OPS has determined that such observation is the best method of examining or testing qualifications. The results of any such observations shall be documented in writing. [Enforceable]

2.03   Planning for Mergers and Acquisitions (Due Diligence re: Acquiring Qualified Individuals)
Does the operator have a process for managing qualifications of individuals performing covered tasks during program integration following a merger or acquisition (applicable only to operators engaged in merger and acquisition activities)?

• Verify that the OQ program describes the process for ensuring OQ qualifications, evaluations, assignment and performance of covered tasks during the merger with or acquisition of other entities. [Enforceable]

3.01   Development and Documentation of Areas of Qualification for Individuals Performing Covered Tasks
Does the operator’s program document the evaluation and qualifications of individuals performing covered tasks, and can the qualification of individuals performing covered tasks be verified at the job site?

• Verify that the operator’s qualification program has documented the evaluation of individuals performing covered tasks. [Enforceable]
• Verify that the operator’s qualification program has documented the qualifications of individuals performing covered tasks. [Enforceable]

3.02   Covered Task Performed by Non-Qualified Individual
Has the operator established provisions to allow non-qualified individuals to perform covered tasks while being directed and observed by a qualified individual, and are there restrictions and limitations placed on such activities?

• Verify that the operator’s program includes provisions for the performance of a covered task by a non-qualified individual under the direction and observation by a qualified individual. [Enforceable]

4.01   Role of and Approach to 'Work Performance History Review'
Does the operator use work performance history review as the sole method of qualification for individuals performing covered tasks prior to October 26, 1999, and does the operator’s program specify that work performance history review will not be used as the sole method of evaluation for qualification after October 28, 2002?

• Verify that after October 28, 2002, work performance history is not used as a sole evaluation method. [Enforceable]
• Verify that individuals beginning work on covered tasks after October 26, 1999 have not been qualified using work performance history review as the sole method of evaluation. [Enforceable]

4.02   Evaluation of Individual’s Capability to Recognize and React to AOCs
Are all qualified individuals able to recognize and react to AOCs? Has the operator evaluated and qualified individuals for their capability to recognize and react to AOCs? Are the AOCs identified those that the individual may reasonably anticipate and appropriately react to during the performance of the covered task? Has the operator established provisions for communicating AOCs for the purpose of qualifying individuals?

• Verify that individuals performing covered tasks have been qualified in recognizing and reacting to AOCs they may encounter in performing such tasks. [Enforceable]

5.01   Personnel Performance Monitoring
Does the operator’s program include provisions to evaluate an individual if the operator has reason to believe the individual is no longer qualified to perform a covered task based on:
  - Covered task performance by an individual contributed to an incident or accident.
  - Other factors affecting the performance of covered tasks.

• Verify that the operator’s program ensures evaluation of individuals whose performance of a covered task may have contributed to an incident or accident. [Enforceable]
• Verify that the operator has established provisions for determining whether an individual is no longer qualified to perform a covered task, and requires reevaluation [Enforceable]

5.02   Reevaluation Interval and Methodology for Determining the Interval
Has the operator established and justified requirements for reevaluation of individuals performing covered tasks?

• Verify that the operator has established intervals for reevaluating individuals performing covered tasks. [Enforceable]

6.01   Program Performance and Improvement
Does the operator have provisions to evaluate performance of its OQ program and implement improvements to enhance the effectiveness of its program? [Non-Enforceable]

7.01   Qualification 'Trail' (i.e.,covered task; individual performing; evaluation method(s); continuing performance evaluation; reevaluation interval; reevaluation records)
Does the operator maintain records in accordance with the requirements of 49 CFR 192, subpart N, and 49 CFR 195, subpart G, for all individuals performing covered tasks, including contractor individuals?

• Verify that qualification records for all individuals performing covered tasks include the information identified in the regulations. [Enforceable]
• Verify that the operator’s program ensures the retention of records of prior qualification and records of individuals no longer performing covered tasks for at least five years. [Enforceable]
• Verify that the operator’s program ensures the availability of qualification records of individuals (employees and contractors) currently performing covered tasks, or who have previously performed covered tasks. [Enforceable]

8.01   Management of Changes (to Procedures, Tools, Standards, etc.)
Does the operator’s OQ program identify how changes to procedures, tools standards and other elements used by individuals in performing covered tasks are communicated to the individuals, including contractor individuals, and how these changes are implemented in the evaluation method(s)?

• Verify that the operator’s program identifies changes that affect covered tasks and how those changes are communicated, when appropriate, to affected individuals. [Enforceable]
• Verify that the operator’s program identifies and incorporates changes that affect covered tasks. [Enforceable]
• Verify that the operator’s program includes provisions for the communication of changes (e.g., who, what, when, where, why) in the qualification program to the affected individuals. [Enforceable]
• Verify that the operator incorporates changes into initial and subsequent evaluations. [Enforceable]
• Verify that contractors supplying individuals to perform covered tasks for the operator are notified of changes that affect task performance and thereby the qualification of these individuals. [Enforceable]

8.02 Notification of Significant Program Changes
Does the operator have a process for identifying significant OQ written program changes and notifying the appropriate regulatory agency of these changes once the program has been reviewed?

• Verify that the operator’s written program contains provisions to notify OPS or the appropriate regulatory agency of significant modifications to a program that has been reviewed for compliance. [Enforceable]

9.01 Are field/job supervisors aware of their responsibilities as defined under the operator's OQ program?

9.02 Are the observed covered task(s) performed in accordance with appropriate operator-approved procedures, and are the procedures present at the job site?

9.03 Are the individuals performing the observed covered task(s) adhering to the operator-approved procedures as written?

9.04 Are the proper tools, techniques and processes detailed in the operator-approved procedures employed in the performance of the observed covered task(s)?

9.05 Are the qualifications of all individuals involved in performing the covered task(s) verified at the job site? Is this verification process performed as detailed in the operator's OQ program plan? Is this verification process applied to employees and contractors alike?

9.06 Are the qualified individuals performing the observed covered task(s) knowledgeable of how to recognize the applicable AOCs and what their expected reaction to the AOCs should be?

9.07 Are individuals not qualified to perform a covered task performing a covered task? If so, are the non-qualified individuals being directly observed by a qualified individual in accordance with the requirements of the regulation?

9.08 How are field/job supervisors informed of changes that affect the performance of covered tasks under their responsibility?

9.09 In cases where the field office is part of a subsidiary or separate district, is implementation of OQ program requirements consistent with other districts and the overall organization's OQ program?

9.10 How is performance of the covered task(s) reviewed/inspected in the field by internal auditors or third party inspectors?

9.11 What problems have been experienced in the field in implementing the operator's OQ program? If problems have been experienced, how have they been communicated back to the individual responsible for the OQ program?

9.12 How are Control Center operations coordinated with remote operations that are conducted with other operations personnel? Are these 'other operations personnel' qualified to perform the covered tasks being performed?