National Conference on Interstate Milk Shipments (NCIMS)

Evaluation of the NCIMS HACCP Pilot Program Phase II Expansion

Table of Contents

• NCIMS HACCP Committee Membership
• Participating Plants
• Executive Summary
• Report of the Evaluation Team
  • Introduction
  • Definitions
  • Implementation of the HACCP System
  • Prerequisite Programs
  • Hazard Analysis
  • HACCP Plan
  • Corrective Actions
  • Verification and Validation
  • Records
  • Regulatory Sample Testing
  • Pasteurization and Aseptic Processing Equipment Testing and Sealing
  • Training and Standardization
  • HACCP Audits, Listing Procedures, and Follow-up Actions
  • FDA State Program Evaluations & Follow-up Actions to be Taken
  • Acknowledgements
• Appendix I. Industry, State & FDA Questionnaires - Responses and Summaries
• Appendix II. Listening Session Summary and Response
• Appendix III. Summary of Regulatory (Inspection & Sample) Information
• Appendix IV. Positive Observations and Comments from Plant Visits

NCIMS HACCP Committee Membership

• Claudia Coles - Chairperson, State of Washington
• Paul Hoge - State of Pennsylvania
• John Beers - State of Virginia
• Susan Esser - State of Michigan
• Richard Graham - State of Louisiana
• Dave Robbins - Dean Foods
• Rob Byrne - NMPF
• Chris Newcomer - New-Tech Consulting, Inc*
• Bill Sveum - Kraft
• Paul Dersam - Upstate Milk Coop.
• John Rushing - N.C. State University
• Allen Sayler - IDFA
• Rebecca Piston - Garelick*
• Steve Sims* - FDA MST CFSAN
• Randy Arbaugh* - FDA DFSR ORA
• Steve Pierson* - FDA RMS ORA

*Technical Advisors

Plants Participating in Phase I & II of the NCIMS HACCP Pilot

Connecticut Yofarm*	Florida Publix
Maine Garelick (Deans) Oakhurst*	Michigan Michigan Milk (Phase I only)
New Jersey Tropical Cheese Parmalat*	Pennsylvania Meadowbrook* Dutch Valley*
Utah Gossner Dannon**	Vermont St. Alban's
Washington Safeway Wilcox*
* Phase II Plants only

EXECUTIVE SUMMARY

Background:

The 1997 National Conference on Interstate Milk Shipments (NCIMS) appointed a committee to address how a voluntary HACCP System should be implemented, evaluated, monitored and enforced under the National Conference on Interstate Milk Shipments (NCIMS) as an alternative to the traditional Inspection/Rating/Check Rating System. This program will:

• Utilize current National Advisory Committee on Microbiological Criteria for Foods (NACMCF) HACCP principles consistent with current FDA HACCP recommendations.
• Continue to assure at least the same level of milk safety provided by the traditional Inspection/Rating/Check Rating System.
• Continue to provide uniformity and reciprocity under the HACCP alternative to the traditional Inspection/ Rating/Check Rating System.

The 1999 NCIMS Conference authorized the committee to conduct a voluntary pilot study to evaluate the proposed plan. The study was conducted in six plants from different states and FDA regions. The plants were chosen to represent a range of plant sizes and product mixes. Sites were selected from volunteer plants with state regulatory approval.

Phase I Evaluation Team

An Evaluation Team, composed of members from state regulatory, industry, FDA and academia, visited all six pilot sites during the summer and fall of 2000. The team presented questionnaires to plant management and personnel, state regulators, listing officers, Grade A milk program directors, and Regional Milk Specialists prior to onsite visits, then conducted follow-up interviews onsite when possible, to evaluate the development, implementation, and maintenance of the pilot program.

On-site visits included an extensive review of the HACCP plant's records as well as a comprehensive evaluation of the plant's facilities, equipment, operation and implementation of the HACCP system. State's regulatory system was evaluated. These were used to evaluate the pilot first-hand at the participating plant location and at the regulatory agency offices. The result was a report to the NCIMS HACCP Committee.

The HACCP Committee made changes to the HACCP system as a result of information gathered from the pilot. The voting delegates at the 2001 NCIMS Conference voted to continue the pilot until the 2003 Conference and to authorize the inclusion of additional plants in the study.

Phase II Evaluation Team

The Phase II Evaluation Team activities were patterned after Phase I and included the use of questionnaires to industry and regulatory personnel and on-site visits at the five continuing Phase I and the three new Phase II plants listed at the time of the Evaluation Team visit. These visits included seven states. On-site activities included review of plant facilities, extensive reviews of HACCP records, plant employee and regulatory personnel interviews and evaluation of state regulatory systems, including records.

Responses to questionnaires for industry, state regulators, and FDA Regional Milk Specialists showed that these groups generally considered the level of food safety under the pilot to be at least equivalent to that provided under the traditional system. Under the conditions of the pilot, the Evaluation Team found the level of food safety under the pilot to be at least equivalent to that provided under the traditional system based on information obtained from survey responses, a comparison of inspections made under the traditional system with state HACCP audit reports, regulatory sample reports, and review of regulatory enforcement action records

The Evaluation Team found areas in the pilot document needing further clarification and modification, including the need for a standardization program for HACCP listing officers and improved training of all participants. This report contains the Evaluation Team's recommendations for these clarifications and modifications, which the Evaluation Team believes, if implemented, will result in the capability to uniformly apply a voluntary alternative HACCP program in a manner equivalent to the traditional NCIMS program and that will form an adequate basis for reciprocity between states that will continue to assure at least the same level of milk safety provided by the traditional Inspection/Rating/Check Rating System.

Technical Resource Team

In order to provide uniform interpretation of the pilot, and to answer technical questions arising from the pilot, a Technical Resource Team consisting of NCIMS HACCP Committee members from state regulatory, industry and FDA was formed. The team functioned effectively throughout the pilot and communicated its decisions utilizing the FDA web site. The Evaluation Team recommends that this Technical Resource Team continue at least through the implementation phase of a voluntary alternative HACCP program.

Summary of Findings

Resources were found to be an issue for both industry and regulatory agencies. The authority to provide these resources pre-supposes industry and state regulatory upper management commitment.

Industry resources that were needed included technical expertise, training, and supervisory oversight. Time was necessary for outside training of key technical personnel and in-house training of team members, supervisors and operators. Further time commitments were required to develop, implement and maintain the program. Documentation and verification of the applied program required the time of a coordinator. Due to the demands of the pilot schedule, program development and in-plant training was rushed in Phase I, but the time frame for implementation was not prescribed in Phase II.

It was necessary for the regulatory agency to commit the personnel resources normally required for the plant inspection effort, plus additional time for training of regulators and listing officers in the basics of HACCP and auditing. Increased communications with the plant and extra visits were necessary in the development and implementation phases. The team found that time spent on the audit will take longer than the traditional inspection time, at least until the auditor becomes comfortable and familiar with the plant's system.

The training that was provided for the pilot program was adequate for the start-up of the pilot, however additional training will be necessary to implement the proposed program. The pilot study pointed out areas of need for increased training emphasis. There was confusion among the participants about the relative difference between the prerequisite program, associated SSOPs and the CCPs. In Phase II, the HACCP committee retitled "SSOPs" as "mandatory Prerequisite Programs". Participants requested more "handson" training and more guidance in the technical aspects of hazards identification, control and verification. In response to this concern, the Evaluation Team has recommended the development of a Hazards and Controls Guide^*, and made specific recommendations to the HACCP Committee Training Team to provide enhanced training in auditing techniques.

The majority of recommendations from the evaluation team deal with the issues of training, auditing and oversight. Some regulators perceived the HACCP pilot system to be more subjective and less prescriptive than the traditional NCIMS program. This issue was acknowledged as a problem under Phase I. Modifications under Phase II helped minimize this problem by developing audit training, critical listing elements and further clarification of the audit form and HACCP documents. This report contains recommendations that we believe, will adequately address this issue.

Industry responses to questionnaires reflect the positive impact of implementing HACCP in their facility, noting the improvements in plant sanitation, corrective action programs, and reductions in withheld finished product. Industry comments indicated that operators were more cautious and thorough due to more training. Implementation of the HACCP pilot program resulted in a significant improvement in allergen management.

The Evaluation Team observed that roles and coordination are somewhat different under HACCP and may change as the program matures and more experiences are gained. This is necessary to effectively implement HACCP program after the pilot. The team found that the relationships between industry, state regulators and FDA Regional Milk Specialists in most cases, reported to be positive.

Conclusions

The evaluation team recommends that the NCIMS HACCP Committee should modify the pilot document incorporating the team's interim findings and recommendations. The committee should produce a proposal that recommends the NCIMS implement HACCP as a voluntary alternative to the traditional NCIMS program. This document represents the final report to be presented to the NCIMS HACCP Committee.

REPORT OF THE EVALUATION TEAM:

Introduction

The Evaluation Team was charged with compiling data and observations from many sources, carefully considering these data in light of what was found during plant and state regulatory visits, and producing findings for the evaluation. The following report is compiled from information collected by the evaluation team with the assistance of industry and regulatory personnel. The evaluation team made onsite visits to all the participating listed HACCP pilot plants and their state regulatory agencies. Personnel interviews, regulatory records and questionnaires from these participants and from FDA, were also used. Specific observations were made from plant and state regulatory visits and these were also recorded for compilation of the Evaluation Report.

To prepare this report, the team evaluated the observations to produce findings that had widespread relevance to implementation and maintenance of the pilot program. In some instances, an observation made in only one or two plants was considered to be of such general interest or importance that it was incorporated into the findings. Also note that some individual observations did not rise to the level of general concern or were determined not to be relevant to the study at hand and, therefore, do not appear in findings that result in recommendations.

Recommendations primarily take the form of changes or clarifications to the pilot program. In some cases, the recommendations may address ancillary programs to the official documents, such as training issues and model guidance for the evaluation of hazards and production of the complete HACCP system.

A word of caution to the reader is appropriate. These general findings and recommendations represent the consensus of the evaluation team's deliberations. Much of the raw data and observations collected by the Evaluation Team are presented in the attached documents to substantiate the general findings of the Evaluation Team. These initial observations are very site and date specific. Many of these are individual team member's observations and may not represent the judgment of all who participated in the evaluation. Individual's observations were sometimes captured for further discussion and later evaluation by the complete evaluation team.

As a final word to those who may review this report, the recommendations are those of the NCIMS Evaluation Team. They were drafted for the purpose of presentation to the full NCIMS HACCP Committee. The NCIMS HACCP Committee membership is broad-based and is directly charged with the responsibility for making changes to the NCIMS HACCP program.

The sections of the report correspond to those sections found in the NCIMS Phase II Pilot document.

Definitions

Findings

The definitions used in the Phase II program come substantially from the 21CFR120, the juice HACCP regulations, and are mainly from the NACMCF recommendations.

1. "Control point" is an archaic term whose concepts have been commonly replaced by the terms "SSOPs," and in our case "PPs."
2. There was some confusion in using the terms "potential hazard", "control point" and "control measures."
3. Plants exhibited some confusion between the terms "adulteration" and "cross contamination" since no definitions were provided.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. It is important to understand that hazards are not to be controlled by prerequisite programs. Prerequisite programs can reduce severity or likelihood of occurrence of a potential hazard, but not control it.
2. Note to the definitions section - include, fix or remove "potential hazard", "control point", "control", "prerequisite programs", "control measures".
3. NCIMS Committee must do a better job of clarifying differences between Adulteration and Cross Contamination. Many items in the Cross Contamination prerequisite are really Adulteration.

Implementation of the Hazard Analysis Critical Control Point (HACCP) System

Findings

Implementation was found to be the most time-consuming and difficult part of the HACCP pilot. It uses considerable resources from both the plant and the regulatory agency.

1. Most states in the Phase I pilot felt pressure to get plants listed according to schedule so the pilot could proceed. In Phase II, it was made clear that there was no pressure to get the plants listed if they were not ready. The question of when the plants were ready was usually answered in terms of operating under the fully implemented plan for a specified period of time (usually mentioned as 60 to 90 days) and having records to document monitoring.
2. The Evaluation Team found that the success of implementation depended on management decision, declaration of intent and provision of resources--technical, supervisory, and clerical. It was important that the program had a champion, but sole dependence on a single person such as the QC director made implementation and maintenance difficult in all but the most centralized management systems.
3. Plants and states often remarked that they did not have enough guidance and would appreciate a hazard guide and examples of acceptable PPs, HACCP plan and related documents.
4. Conversion from the Phase I to the Phase II requirements caused challenges for plants. In some cases, plants did not give updating of the HACCP program priority. Because of this there were inconsistencies in documentation and practice.
5. Some plants had difficulty meshing their corporate QC plan with the requirements of the HACCP pilot, resulting in confusing documentation and procedures. This usually resulted in complicated programs that were difficult to verify and audit.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by th e complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. Problem with HACCP team not being very functional. Dependent on single person. Meetings are not frequent or fully attended. HACCP Coordinator seems to be in traditional role of quality control that does not involve other HACCP team members.
2. The Evaluation Team found a very modern, well-equipped and well maintained plant, with an enthusiastic, committed workforce, dedicated management and a mature HACCP system. The plant has experienced a stable workforce during the last year that has contributed to their ability to maintain and upgrade their HACCP program. It has also contributed to a better-trained staff.
3. The HACCP Committee should establish one set of reporting forms and charts to improve uniformity between HACCP plants and state enforcement efforts.
4. The continuous updating of the written HACCP program has resulted in some inconsistencies in the HACCP document:
   1. a) Use of names for prerequisite programs in the table of contents, hazard analysis and in specific prerequisites.
   2. b) The prerequisite for equipment sanitary design requires 3-A construction but woven material belt, rough welds and exposed bolts in cottage cheese equipment does not meet this requirement.
   3. c) Entering of information on the Hazard Analysis form (i.e. eggnog, blending step, air blows used to move product throughout the plant, use of a metal detector for plastic lidded cottage cheese cups but not for foil-lined cottage cheese cups, fortified skim milk) including identifying specific prerequisites for reducing the likelihood of a hazard.
   4. d) Hazard analysis entries for staph. toxin development did not support likelihood of a hazard.
   5. e) Documentation
   6. f) Verification
5. The Evaluation Team continues to find that the resources needed in order to develop and maintain the HACCP program are significant. Once the HACCP plan design has been completed, fully implemented and functioning for a reasonable amount of time, the team believes that resource requirements will be reduced.

Prerequisite Programs

Findings

Prerequisite programs design and implementation were probably the result of more observations than any other category. The following general comments can be made based on the observations:

1. Plants were inconsistent in the breadth of their PP's. The Phase I pilot recommended PP's in several categories. For the Phase II pilot, the emphasis was placed on the 8 SSOPs required by the juice HACCP regulations along with those PPs which were used to justify that an identified potential hazard was not likely to occur. Some plants did not address general-purpose PP's such Building, Premises, Construction and Maintenance or Training.
2. Without clear guidance from the pilot, plants had some difficulty understanding which procedures to include in prerequisite programs.
3. Several plant personnel noted that models of appropriate PP's would be useful in deciding what a typical PP should look like and what should be included in each PP.
4. Plants were inconsistent in the depth of their PP's. They varied considerably in level of detail. Some may still have too much material not related to sanitation and safety, making extra work for the plant and the auditors. Prerequisite programs should identify the goals, the personnel responsible, and those procedures designed for accomplishing the goals, along with documents used for records of monitoring and correction. Frequency of monitoring was an issue in some situations.
5. Implementation of PP's was not consistent between plants. Some followed the programs carefully while some used the programs more as guidelines. There were numerous instances where PP's were too general or referred to a general section of the PMO or a corporate policy. Several plants had PP's that were inconsistent with their practices, or in some cases procedures had been changed without changing the PP.
6. PP documents were sometimes difficult to access for the auditor because of inconsistent terminology between the HA and the PP. A list of associated documents would have been helpful.
7. The committee noted that there was no method of tracking repeat occurrences of noncompliance in the prerequisite program that could be used for verification.
8. There seemed to be a need to improve verification of documents as well as verification of PP's as a whole. Verification should identify inconsistencies and other shortcomings.
9. Pasteurization equipment and operation requirements were scattered among several different PP's in some plants, making the pasteurization CCP difficult to verify.
10. Those items which deal with proper installation and design of equipment and construction items such as plumbing and cross connections often have not found a home in PP's. Some plants have handled these things using a plant audit checklist.
11. In the case of corrections required by most of the prerequisite programs that were product-related, most were accomplished.
12. Corrections to general sanitation related prerequisites such as pest control and monthly audit checklists were not well tracked, and often, repeated noncompliance was noted.
13. In those places where corrections could not be accomplished immediately there often was no mechanism in place, such as a corrections log and timetable to encourage follow- up.
14. The Phase II HACCP pilot document does not specifically mention prerequisites identified in the hazard analysis that reduce the likelihood of occurrence of a potential hazard as having to be written, monitored, verified and documented similar to the eight mandatory prerequisites.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. Transport vehicle inspection program referenced in prerequisite program but assigned individual not checking transports (example, 4 transports at loading dock with 3 dirty, but checklist filled out showing all were clean).
2. Overall, the PP are some of the best we have seen, in particular PP #2 on Cleaning and Sanitizing of equipment such as specifying visual inspection monitoring, chemical strength monitoring, frequency of cleaning, broken down by type of equipment, very detailed.
3. Instructions for operation of COP tank in the prerequisite program states "heat up to 150 degrees"; however the steam controls are not working very well and there is no thermometer. The temperature was found to be 120 degrees. How does the plant monitor temperature, length of wash, equipment washed, etc.?
4. Plant lists quarterly micro. analysis in PP to control microbial hazards in caps, jugs and cartons, but they do not do this.
5. Plant does not appear to have incorporated the overall concept of HACCP into their operational culture.
6. Documents were inconsistent between corporate SSOPs and the written PP's such as, safety of water PP for analysis noted in different documents as annually, twice a year, or quarterly. Need to have consistency.
7. Some documents are not specific enough to monitor and verify the program such as the requirement for safety of water. PP says that it must be analyzed to confirm potability but does not indicate what it should be analyzed for and at what frequency, etc.
8. Recirculated water is noted in the PP to be treated and monitored. Information needed on treated with what, how often, and what are limits.
9. Condition and cleanliness of equipment states that environmental testing for pathogens is to be done on a routine basis. What does this mean? Provide more detail.
10. Prevention of cross contamination prerequisite does not address pressure differential calibration for the two HTST pasteurizers as per pasteurization model.
11. Protection from Adulteration - to find all the things to be covered under this PP, you have to go to three separate ones, "Building interior", "Design and Installation", "Sanitation".
12. The mandated eight required prerequisites are difficult to identify within the complete PP. This results in difficulty in following because they are contained in other general PPs. Lack of a required PP on cross connection are difficult to find (condition and cleanliness of food contact surfaces is in three different places, one called "Food contact surfaces", "Sanitation" and "Design and Installation".
13. Recommend to NCIMS committee that the 8 required PPs should be written separately from the rest of the PPs.
14. Transportation and Storage PP does not address that source of milk must originate from a listed source.
15. Potential cross connection between raw and pasteurized lines in valve cluster in blending area. Need physical break
16. The prerequisite for Transportation and Storage mention temperature control for ingredients but gave no criteria for temperature control of finished product
17. Prerequisite for Equipment did not cover cross connections between raw and pasteurized (one was noted in the plant). This PP included calibration of equipment but the Evaluation Team noted that cheese dressing tank thermometer was inaccurate.
18. The prerequisite on Premise states boiler chemicals meet EPA/USDA approval but did not cite CFR even though culinary steam is used for direct injection into cottage cheese.
19. The prerequisite on personnel and employee health did not address high risk disease situations, i.e. visiting daycare centers, attendance at church dinners, contact with individuals with food borne infectious disease such as TB, etc.
20. There is a need to designate or define what are "toxic" materials to differentiate them from non-toxic materials in the Transportation and Storage prerequisite.
21. Quarterly microbiological analysis records on non-food products were not available and appear not to exist as per the Transportation and Storage prerequisite. The terms, "When possible" or "Where necessary" is used frequently in the prerequisite program description. This is a weak, non-committal statement that does not provide consistent control of a prerequisite and should be eliminated.
22. The write-up of the prerequisites tends to be very general and does lead to uncertainty regarding the coverage and details of prerequisite management.
23. The hazard analysis on Equipment addresses all equipment, but equipment maintenance for separators and homogenizers not mentioned.
24. Finished product storage and finished product are monitored for temperature daily according to the prerequisite but records only address product and not storage area.
25. Flow diagrams are not required by the pilot program document to be present, accurate, and verifiable.
26. Preliminary steps are not mentioned in the document.
27. The floor in the pasteurizer area is in poor repair: Note the "Premises-plant construction PP" has been withdrawn.
28. Pathogen growth at the Finished Product Storage and Transportation steps is identified as a hazard for a number of products by the Transportation and Storage prerequisite but control measures do not specifically address.
29. The use of a corporate Food Safety Audit form performed every 2 periods (every 28 days) is too infrequent for some activities to use this document as the sole record to support control under a prerequisite program. Tailor form so more specific to plant activities and necessary monitoring frequency.
30. The PP program does not address all issues identified in the HA and also does not identify specific records. Appears the PP was driven more by need for consistency with the monthly Corporate Food Safety audit form than on developing the mandatory prerequisites as detailed under the NCIMS HACCP pilot program.
31. The prerequisite on Equipment under Food Contact Surfaces addresses rust on the outside of equipment. What is the relationship to food safety?
32. All equipment is maintained to insure that there is not chemical or physical hazard that will affect food safety. Unsure what was intended to cover cross connections, metal fragments, protection of openings, etc.
33. Prerequisites do not reference these records.
34. The plant's prerequisite program needs to control all conditions that contribute to overall product safety. Some examples of deficiencies are:
    1. a) Air under pressure is not addressed even though plant was monitoring it and had supporting records.
    2. b) Silo tank wash frequency is not addressed with specificity, i.e. frequency was referenced as "daily" or "as used" but actually practice is a maximum of every 72 hours.
    3. c) Details regarding manual cleaning of equipment are not completely described in the Cleaning and Sanitizing prerequisite even though the plant does monitor and document this activity.
    4. d) The cross connection prerequisite focused on regenerator pressures only and did not address piping connections. The Team found a cross connection at batch tank #1.
    5. e) The water prerequisite for boiler water required current boiler water letters, but letters on file are very dated (10 years old).
    6. f) The prerequisite for Protection from Contamination does not address re-work (included only a temperature reference).
    7. g) Supplier's letters of guarantee are outdated (up to 10 years old). Additionally, certificates of analysis and supplier guarantees need to be verified.
    8. h) The prerequisite on adulteration did not address allergens.
    9. i) The prerequisite on incoming ingredients required certificates of analysis but there were no specifications or details indicated.
    10. j) Find a way to include GMPs
35. Monitoring is very good on the PP. Some gaps particularly in premises, surroundings, flies.
36. PP general comment: Prefer the PP would use name or reference number to aid in finding the document.
37. A document list would be helpful.
38. In a plant this old recommend that an additional PP for construction, housekeeping, facilities, and premises be included.
39. Monthly walk through is still being used, but no longer using a checklist. If there are standards given for the criteria, it is not mentioned
40. An outside contractor is handling the pest control. Plant Management reviews report, does follow up, and approves the report.
41. It is not clear what the reference to the PMO meant in terms of the PP requirements for the plant
42. PP's for premises-- grounds, plant construction, recalls and training, were withdrawn by the plant. We were unsure why
43. What is the FDA illness form mentioned on the rating-Food code.
44. Use of the PMO references---what do they bind the plant to?
45. COP records are well documented and log used to track concentrations of CIP solution.
46. Log for changing air filters is best ever seen because they check every air filter every day.
47. Note: PP #6 talks about reconciliation of vitamin usage daily by weight. Most current reconciliation results in 108% and 112% which is excellent.
48. Prerequisites are silent on facility or premise. In prerequisite #3 & 5 were silent on piping cross connections. Also silent on piping cross connections for cleaning solutions.
49. Prerequisite on cross contamination is silent on raw to pasteurized contamination except through regenerator of the pasteurizer.
50. The storage of single service containers - needs to be stored in a clean dry area as per the PP but actually stored in CIP room that does have water and is not dry room.
51. The prerequisite on water safety required boiler water additives to meet 21 CFR 310 but the only documentation provided to address the PP requirement was USDA accepted, not according to 21 CFR. Approval letter should reference 21CFR instead of USDA compliance
52. Stainless 10 gallon cans found in processing room with milk residue in them. Training is that employees are to hand wash cans or replace with clean one but appears this is not being done. Also not in any prerequisite and no hazard analysis
53. Heat treated cream silo chart uses chart with temperatures progressing from high to low toward the center of the chart. This makes it difficult to determine time.
54. The heat treated cream load-out is done with the tanker manhole lid partially raised with no overhead protection or filtering.
55. The designation of the corrections for a PP is titled a "Deviation Log" when in reality it is a "Nonconformity Log".
56. Eliminate use of nonspecific terms in the PP such as "should".
57. Construction of the powder horn is formed into table top and there is potential for drainage off table top into powder horn.
58. Storage of toxic compounds is a problem because stored in areas adjacent to packaging materials. Toxic chemicals found outside of toxic storage area.
59. Very good system for cleaning transports prior to begin loading; however, we recommend that the cooler be protected from the aerosolized over-blow and debris.
60. Housekeeping on premise, outside has inadequate drainage. Also a problem throughout facility with maintenance. Need adequate drainage of outside drains
61. PP program appears to be in very good shape. Very good content including identification of records, frequency of records, corrections, examples of records.
62. Flies appear everywhere including inside the processing area. Maybe partly the result of negative air pressure in the processing room.
63. 10 gallon dump cans all over and very common. Question of whether temperature of product is maintained until stored in the cooler.
64. PP 3 contains info. on maintaining diff. pressure as needed but does not address integrity of seals for diff. pressure controller.
65. PP #5 Protection from Adulteration is where all allergens are addressed. Good job on allergens, but for some reason the diff. pressure info repeated from PP#3.
66. PP#7 Employee Health conditions, does not cover high-risk activities. Employee handbook only talks about lesions on hands, and employees as to wear gloves but does not address food borne contagious illnesses.
67. PP#8 Exclusion of Pests does a good job on chemicals, traps and bait stations but should have references on construction to preclude entrance of pests. Reference problem is with flies.
68. The brief written description of each prerequisite should contain the description, numerical values or other details that define control (i.e. ozone level and UV light intensity for plant water prerequisite). This information can then be used by the plant and regulatory agency to determine control and compliance.
69. Corrections to operational problems in the Prerequisite Program are generally effective and timely, but tracking and documentation of repeat occurrences in a central record is not available. Comment: This may have an impact on the adequacy of the oversight by the regulatory agency also.
70. Supplier control prerequisite may be needed to better document safety, storage, inventory control and allergen issues.
71. Pest control done by outside contractor. The Pest Control plan was comprehensive and adequate, but there were no records available to verify that procedures in the plan had been performed.
72. COP records monitor cleaner/sanitizer strength and temperature but did not indicated what parts were cleaned and were not initialed by the operators. (Note: not carried forward by Evaluation Team since not required by pilot or plant's written HACCP program.)
73. The HACCP Committee needs to assign a group or individual to put together a model prerequisite guide that covers at a minimum the eight mandatory prerequisites and includes a model form or checklist to specify the minimum amount of information to be included by the plant in the "brief written description".
74. More extensive consideration of allergens for all products would strengthen HACCP program.
75. Manual records system for raw silo temperatures that log temperatures every 15 minutes not frequent enough to identify potential temperature problems and computerized temperature recording system is in advanced stages of failure. This needs attention through increased manual logging frequency or replacement of the computer system.
76. Appendix N testing records for incoming condensed milk should use the same system as established for incoming raw milk tankers to reduce confusion.
77. Prerequisite on pest control was not being effectively documented.
78. The prerequisite for equipment design stating all new equipment will be reviewed and inspected by the State Regulatory Agency is not consistent with the plant's intent to have only significant, complex, unique or large pieces of equipment reviewed and inspected by the state prior to being made operational.
79. High piping between booster pump and raw regenerator in HTST #4 prevents natural drainback during unexpected shutdowns of system.
80. Contamination with pathogens is also apparently considered to be a potential hazard.
81. Cleaning PP is listed. The Transportation and Storage PP indicates tank cleaning frequency but procedures are outlined under Food Contact Surfaces. Again, internal consistency throughout the HACCP System is important. When dealing with a PP there should be documentation, a brief written description or checklist, that outlines procedures for implementation, monitoring and correction. This serves as a basis for auditing.
82. Storage and Pest Control PPs are not consistently named with the actual PPs. Storage and Pest Control PPs probably are contained in portions of the 8 mandatory PPs and are to be in place even in the absence of the requirements of the HA.
83. The titles for Cleaning PP and Adulteration PP do not match up with program titles. There should be internal consistency which allows the plant to audit the plan routinely. Without this internal consistency from HA through descriptions of the programs, monitoring, corrections and documentation, the PP cannot be audited.
84. PP for air blows are used throughout plant to move past. product. Not considered in HA.
85. PP for equipment sanitary design requires 3-A or equivalent construction. CC drainer has woven material belt, rough welds and exposed bolts in product contact area. This is why one of CC line has metal detector.
86. If plastic lidded containers that go through cheese drainer need to be screened with metal detector to prevent hazard, why do foil lined containers not need same screening?
87. Records for ozone level and UV light intensity or functionality need to be maintained for the plant process water.
88. Problem with corrective follow-up on problems identified in monthly internal audits (30 repeated problems).
89. Monthly internal audit forms identify problems but same problems are noted month after month.

Hazard Analysis

Findings

HACCP is based on the determinations made in the HA. It must be complete and identify all Potential hazards. Following are the major observations.

1. Completeness of the hazard analysis was often an issue. The individual steps of the hazard analysis is dependent on a complete and accurate flow diagram listing all steps, processes and inputs as well as a complete list of ingredients. The current document does not require a written flow diagram.
2. Common steps between products were sometimes inconsistent between flow charts.
3. Hazard analysis is dependent on the availability of technical expertise with good technical information. Most plants requested some assistance in identifying and assessing hazards.
4. The hazard analysis form most often used is the one that was presented in training. It was inconsistently filled out in column 3 for the determination of reasonable likelihood of occurrence.
5. Listeria in finished products was often not identified as a potential hazard on the hazard analysis form. This organism probably is associated with the most food safety concerns of any in pasteurized product.
6. Plants often were inconsistent in how they dealt with an identified potential hazard such as allergens, vitamin intoxication, and therapeutic drug residues.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. Product returned in company owned distribution trucks from stores that stayed on truck is reworked, if it meets certain criteria in the PP. This needs a hazard analysis
2. Flow diagrams are equipment specific, not product specific
3. There needs to be additional research to obtain true scientific information regarding the potential generation of Staph. aureus toxin production and development to support or modify the current four hour rule of thumb. This issue is being faced by almost all pilot plants in their product hazard analysis.
4. The hazard analysis for many products includes a hazard for animal drug residues in the raw milk receiving step. A "yes" in column 3 (potential food safety hazard is significant) means that its likely to occur even in light of properly applied PPs and will require control by a CCP. This concept in the use of the hazard analysis form used by the HACCP Committee in its Baltimore training does not appear to be understood by the HACCP plants. The NCIMS HACCP Committee needs to either provide better guidance on the use of the hazard analysis form or scrap the form and use the IDFA form or another one that is more intuitive to the plants filling them out.
5. Cottage cheese vats not protected during set and open to airborne contamination, such as from aerosols created by water hoses. Room opens directly into processing room and traffic from processing and packaging areas can contribute to contamination. Hazard analysis should address airborne contamination as potential hazard
6. The hazard analysis for a number of products lists vitamin intoxication as a hazard, managed by Operating SOPs. A review of the operating SOPs does find statements indicating control. It would be better to address this potential hazard in a prerequisite program.
7. The records documenting the ongoing application of the PP's are adequate and complete. The NCIMS HACCP Committee needs to provide additional guidance in the HACCP document on standardizing and formalizing procedures for checking the prerequisite performance
8. The hazard analysis for eggnog lists microbiological contamination as potential for caps and cartons as "Not likely to occur" but justification does not list purchase from an IMS listed source, yet this is a requirement of their HACCP pilot program
9. Common portions of flow charts should be standardized to reduce duplication and minimize the possibility of discrepancies when changes are made. Common sections of flow charts could be represented by one flow chart to simplify things. The current duplications requires one change to be reflected in many flow charts and hazard analysis, increasing the possibility of forgetting to update one of the affected flow charts.
10. The hazard analysis identifies biological hazards for pathogens at the raw receiving step. The monitoring temperature is described as being for the storage area, but it should be for milk in tankers.
11. The hazard analysis does not flow to the prerequisites and monitoring. The prerequisites are often very generally stated so it is difficult to determine what is monitored, by who, and what is acceptable.
12. The hazard analysis did not include rework.
13. Rework is not included on any flow diagrams for products. Potential hazards are not identified or evaluated and also because of storage of rework and possible temperature abuse.
14. The pests and rodents are noted as a hazard in the hazard analysis with the Transportation and Storage prerequisite listed as the control for most products at the "Receiving" step. The prerequisite that does address pest and rodent management is the Sanitation and Pest Control prerequisite.
15. Internal audit procedures-- not using plant-initiated internal audits. The plant using the results of their third party audits
16. In the hazard analysis for a number of the products with the raw milk receiving step, drug residues are listed as a significant hazard by indicating a "Yes". This conclusion requires control through a CCP at this step or a future step, yet the justification is listed as Appendix N. Use of form not straight forward and NCIMS HACCP committee needs to address use of this form, additional instruction and training or scrap form for another form.
17. The first three columns for the hazard analysis are used appropriately but the rest of the columns appear to be misunderstood and need to be addressed by the HACCP Committee since this has been a consistent problem by most plants, i.e. receiving step with biological pathogens. The 4^th column on justification states "A subsequent step will eliminate pathogens." This is not adequate justification. 5^th column on control measure does not need to be filled out if column #3 is "No", but most of the time column #5 is filled out.
18. Foreign material is listed as a hazard in a number of product "Receiving" steps and controlled by the Transportation and Storage and Equipment prerequisites. A review of these programs shows that foreign material is not directly addressed in either. It is more common for foreign material to be controlled through the use of strategically located filters and screens which are not listed in any flow diagrams or product hazard analysis.
19. Sanitizer residues listed as a hazard in the raw milk storage step under the hazard analysis for a number of products notes that it is controlled by the Sanitation and Pest control program and is addressed there in general terms, i.e. #9 on page 2. However records documenting this to prove sanitizer or cleaning solutions are not left in storage tanks during filling with product do not exist. There is a procedure for inspecting a tank prior to filling but this is not referenced in the Sanitation and Pest Control prerequisite.
20. Allergenic contamination with eggnog base is not listed as a chemical hazard for any products. The Allergen prerequisite describes in general terms the management of allergic ingredients but does not address frequency of cleaning, separation of lots or batches or sequence of processing to manage these items.
21. Ammonia contamination is listed as a hazard for a number of product hazard analyses under the raw milk storage step but not specifically mentioned in the Equipment prerequisite program. Found in two different places. This is not generally a hazard in food products, but a quality issue.
22. For a number of product hazard analyses, "contamination from raw milk side of unit" for the HTST is listed as a hazard with control occurring through the Equipment prerequisite. This is a cross contamination issue; however, it is not addressed in the Equipment prerequisite other than general references to calibration of equipment.
23. The flow diagrams were not complete and did not include Salvage Milk.
24. The cooling step for heat-treated cream (this step was shown on the white milk flow diagram but not on the cream flow diagram).
25. Water and air are not included for any product.
26. The hazard analysis for large curd products under the step for storage, identifies sanitizer residue as a hazard in the Transportation and Storage and Personnel PP but these do not address the issue. Should indicate the Sanitation and Pest control PP.
27. Vitamins are not addressed as a potential hazard for any products run on HTST #1.
28. The control of the hazard for introduction of pathogens at the blending step of a number of products indicates post pasteurization contamination and control is through the Personnel prerequisite. This prerequisite does not adequately address measures to be used to control post-pasteurization contamination. This hazard is more likely controlled through GMPs, employee training and the Sanitation prerequisites.
29. No hazards identified except biological hazards and chemical antibiotics. No physical hazards such as metal noted.
30. No guidance on sensitive ingredients in pilot document. Should be included in pilot document, in training and in hazard guide.
31. Recommendations might be made that the flow diagrams include conditions such as time, temp and pH, if useful.
32. Hazard analysis of different products noted where hazards were controlled, but no hazards were identified. No evidence of consideration of certain ingredients and packaging.
33. PP7 Employee health conditions-- high risk conditions were not addressed, diseases not addressed.
34. Flow chart lacked several steps, fruit receiving, raw milk pasteurization and separation, cream, blend, culture media blending and rehydration, leading it to be incomplete and misleading. Because of this, these steps were not considered in the hazard analysis.
35. Other flow diagrams were mostly complete, but missing some steps (packaging).
36. Hazard analysis for triblender did not consider that it was used for raw and pasteurized product on the same day.
37. Ingredients were held to be proprietary, no letter of guarantee for cert. of analysis. Need an FD&C letter of guaranteed compliance.
38. Letters of guarantee were present.
39. Sensitive ingredients and allergens not evaluated on the hazard analysis.
40. Hazard analysis did not show complete consideration of hazards on the summary table.
41. The following steps including raw receiving- physical, silos - chemical, cream storage -biological and chemical, and rework list no potential biological hazards.
42. Hazard analysis column 3 heading used the heading "does this hazard need to be documented in the HACCP Plan?"
43. Eggnog flow chart is very thorough.
44. Evaluation Team liked the way the role of the PP was identified in HA worksheet but would like to see a little more on individual procedures.
45. The written HACCP plan does not include all processing steps (washer cooler for fruit on the bottom yogurt), ingredients (concentrated milk for eggnog), and potential hazards (i.e. juice into milk, listeria in pasteurized product, curd washing for cottage cheese, HTST cross-connections). In addition, the some sub-processes should have their own flow chart, i.e. concentrated milk and raw milk.
46. Usually the HA does not list PPs for justification when no hazard has been identified.In this case, they were. However, allergens and sensitive ingredients as well as vitamins should be considered at this step. The same is true of the next step on non-dairy storage.
47. Drug residues are identified as the hazard but the control step is a ß-lactam test. Drug residues should refer to specific drug residues, i.e. ß-lactam test. Hazards need to be specifically addressed. Drug residues as a hazard without being specific and if this nets out a CCP, then what are the critical limits. Do you set critical limits for all possible drug residues?
48. Staph. toxins are identified as a potential hazard, but when evaluated, they are determined to be not likely to occur. The justification indicates time and temperature for tankers. This becomes a required prerequisite program and must have evidence and records of implementation, monitoring, corrections. What is the scientific basis that the occurrence is unlikely? The Receiving Operation Checklist does not necessarily indicate parameters and show that a procedure is in this place to make this decision.
49. The Receiving PP is supplementary justification and should appear as a justification rather than a control measure. Prerequisite programs which justify the determination that a potential hazard is not likely to occur should be clearly identified in the hazard analysis. This should point to the implemented procedures and their documentation.
50. The Grade A storage step on the HA Worksheet again lists pathogens as a potential hazard. In this case the hazard seems to be presence and growth of vegetative pathogens, including production of staph toxin. Growth is inhibited by time and temperature control in the PP entitled Transportation and Storage. The HA erroneously indicates a PP which is Time and Temperature Storage. The PP should indicate a documentation of these procedures.
51. The raw milk receiving step as well as step 1 in the HA indicates heat-treating will destroy pathogens. We probably should use the term pasteurization to be clear and consistent in our terminology. Several observations noted above for condensed also apply to this step. Physical hazards were not identified, but the justification indicates they will be filtered out by tankers. In addition the next step is a filter. The assumption is this is a quality step and does not indicate the presence of a hazard.
52. The raw milk storage step results in many of the same observations as HA step 2. In this step and in step 2, calibration is listed as a PM control measure. In HACCP, by definition, calibration is considered to be a validation step under the verification program.
53. Blending identifies no potential hazards, but justification mentions several PPs including cross-contamination, which addresses allergens. The PP preventing cross-contamination addresses egg nog as a potential allergen. It does not address the potential for milk to contaminate juices run on Grade A equipment. The plant operates under a HACCP plan for juice (outside of NCIMS responsibility).
54. It seems unusual, because within the last 20 years, major outbreaks have occurred from the contamination and growth of Listeria in refrigerated, pasteurized milk. These have resulted in deaths. Therefore, pasteurized milk products should probably have Listeria identified as a potential pathogen. If the likelihood of occurrence is low due to the presence of properly implemented prerequisite programs, then identify the programs and their documentation, but don't ignore an obvious potential hazard.
55. Non-dairy ingredients and packaging, especially compounded ingredients, labels and packaging need to be considered individually to focus on specific hazards such as allergens and sensitive ingredients.
56. Fruit on Bottom HA - Washer Cooler is listed process step but not considered in HA.
57. HTST Past. does not reference PP on cross connections. PP on cross connections is not complete enough to have provided meaningful guidance regarding problems found on past. #4.

HACCP Plan

Findings

The HACCP plan itself usually listed only the control of vegetative pathogens at pasteurization. In most cases other identified potential hazards were justified as not likely to occur in light of prerequisite programs.

1. Pasteurization models were found to be helpful. They need to be updated to include HTST, HTST with magnetic flow meters, and vat pasteurization.
2. There was some inconsistency in applying the pasteurization model that was furnished during training.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. Document entitled "CCP Summary Table" under the section entitled recordkeeping only refers to temperature charts reviewed by supervisor but not flow rate records. Both HTSTs have mag. Flow meters.
2. The overall HACCP plan was very good, but implementation appeared to be a hybrid between unmodified prior or existing programs and the needed HACCP plan items.
3. One operator's documentation of pasteurization charts was incomplete over a three year period without correction. This was a very long period of time for lack of correction of such an important record
4. HACCP Summary Table does not address the pasteurizer seal integrity.
5. Quarterly calibration of pasteurizer done by state regulatory.
6. PP# 3 Prevention of Cross Contamination uses cut in and cut out as a separation of raw and pasteurized. Does not include the differential pressure, doesn't mention proper interwiring.
7. No PP for adequate holding time in past. Should this be incorporated in the model?
8. Critical limits are intended to reflect scientifically based maximum or minimum values to represent control of the CCP. The plant has used operational times and temperatures for its critical limits. Recommend modifying identified pasteurization times and temperature critical limits to reflect minimum acceptable pasteurization times and temperatures. A CCP is identified as 175 degrees F. This critical limit should not be identified on HA but on CCP Summary Table also called HACCP Plan form. When noted, we should call this critical limit "pasteurization above 175 degrees F" to distinguish it from a simple heating step.
9. The plant has chosen to lump eggnog and 2% low fat together on the same flow chart. The following ingredients were not broken out: Sugar, Eggnog Base, Guar Gum, Carrageenan, Packaging - gallon jugs, caps, and labels. By grouping nondairy ingredients together, the egg yolks, an allergen in the eggnog base may not be accounted for on the hazard analysis. Packaging, together with the labels, need to identify egg yolk as a potential allergen, may be overlooked. All components of eggnog should be entered on the flow chart as ingredients. The individual ingredients would therefore have their own flow chart and hazard analysis.
10. Pathogens should be more clearly identified, such as vegetative or spore-forming. Clearly both may be present in the food. More clearly indicating which vegetative pathogens are expected, gives the opportunity to assess the likelihood of occurrence and properly design and choose control measures. In this particular situation, most vegetative pathogens are destroyed by the control measure of pasteurization, but the type of occurrence, such as presence, contamination contributed at this step, growth, destruction, or toxin production cannot be assessed by simplistic annotation.
11. Computer Controls: HACCP program is silent on computer controls for past. except for general statement requiring PMO compliance. HTST #4 computer controls adequate, operational computers appear appropriately isolated from Public health safe guards, however HACCP system should be used to monitor and verify continuation for these controls.
12. The HACCP Committee should consider modifying its pasteurization model to place all traditional pasteurization public health control measures in a unified model.

Corrective Actions

Recommendations

- None

Findings

Corrective actions to the violation of critical limits were generally in order. This was probably due to the fact that there are considerable automatic controls to pasteurization and these were familiar to both the plant and the regulatory agency through long implementation. Plants rarely included CCPs in their plan other than pasteurization.

Observations - None

Verification and Validation

Findings

In the HACCP pilot, the verification program shall evaluate whether the HACCP system is being implemented according to design. In addition, there are requirements to validate that the HACCP plan is adequate to control hazards that are reasonably likely to occur.

1. Calibration of process monitoring instruments under the verification program sometimes did not extend to those instruments used in the HACCP system other than the pasteurization equipment tests and thermometers.
2. Sometimes, in a few plants, basic calibration information such as comparison of indicating and recording thermometers and checking of cut-in and cut-out were not always entered on the records
3. The volume of records generated in some plants resulted in the review of records that was very time consuming for plant personnel and difficult to complete, i.e. monitoring of critical control points, taking of corrective action, calibration of process monitoring instruments, etc.
4. The Phase II pilot document was not clear resulting in some confusion about the need to verify regulatory seal integrity and the frequency.

Validation of the HACCP plan and the hazard analysis shall occur at least once in 12 months after implementation and at least annually thereafter.

1. Annual verification and required reassessments were not observed being accomplished in several plants.
2. The validation is to be performed by a "qualified" individual who has been trained. There is a question whether just having the Dairy HACCP Core Curriculum training produces a qualified individual capable of conducting a hazard analysis and determining controls in the absence of technical training and experience without specific guidance.
3. Records of validation of the hazard analysis after product and process changes were not common in plants.
4. Plants were often not in a position to validate that an identified potential hazard was not likely to occur to the satisfaction of the regulatory agency. This caused conflicts with the regulatory agency when the regulatory auditor did not accept the findings of the hazard analysis. An example of this was the potential hazard of staph. toxin production in the cream pot. This is one example of the need for the Technical Resource Team.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. In HACCP Plan under CCP planning, the document states equipment tests are done by the state. This is not in compliance with the NCIMS HACCP pilot document.
2. The calibration of pasteurization equipment referenced that the state regulatory agency is responsible for accuracy and timeliness of equipment tests.
3. Pasteurizer flow charts are not verified by individual that has been appropriately trained. The operator on HTST #2 recorded a cutout as 165.8°F. This would indicate that the state seal registering a cutout of 166 °F was no longer valid, which is not likely. Note: It appears that the operator was doing this test too rapidly and incorrectly reading the temperature from the recorder instead of the indicating thermometer.
4. Charts overlap repeatedly and noted by verifier but not corrected.
5. HACCP plan needs to be revalidated since it has been slightly over 1 year since last validation.
6. CIP charts are not dated by the verifier as per NCIMS HACCP document.
7. Audit for 8a included in HACCP plan records. Specific item requires event pen to indicate forward flow based on flow rate but charts did not show clear movement between forward and divert events. May take up to 7 minutes to complete movement.
8. CIP charts: Some the pens on the recorder were not working for several days in a row. Stamped as being verified but did not correct pen problem until a day after being verified.
9. Some of CIP charts overlap. Was not noted by operator or verifier. One of the pasteurizers was wired with the cleaning computer so part of the public health controls could be overridden, preventing the separator from valving out and overriding the public health controls. Also, valving out for the separator requires the control of three valves, one of which has the spring operating in the wrong direction.
10. The plant did not address the role of regulatory seals in its HACCP program.
11. The "verification" column for the CCP or HACCP Plan Summary table is not complete, i.e.:
12. Nothing addressing the comparison of the indicating and recording thermometers when changing charts.
13. Integrity of the seals not addressed.
14. The pasteurization equipment controls tests conducted on HTST #2 by the state regulatory auditor did not adequately address all public health requirements. Specifically: The cream separator that was an integral part of the system did not "valve out" as required.
15. The use of a master-cleaning schedule pulled many disparate things together but was a little hard to follow because of the varying frequency for all the various individual activities.
16. To improve flow of information between records we suggest the verifier or monitor be trained to mark through those places on forms that are superfluous so there are no blank spaces. Examples of this are checks only done weekly when there is a space for every day, or in those cases when the facility may be down and not processing.
17. The product flow diagrams should be verified annually and each time there is a change which might affect the hazard analysis in order to make sure they are complete and include inputs such as water, air under pressure, and rework or salvage. These flow diagrams are key to the conduct of the hazard analysis. Process steps or ingredients not accounted for on the flow diagram will probably not be considered in the hazard analysis. Consequently, they may be left off the HACCP plan or out of the mandatory prerequisite programs. It had been a little over a year since the last verification
18. It appears there was uncertainty by the plant verifiers about the correctness of information on the HTST recorder-controller charts and magnetic flow meter charts.
19. The NCIMS HACCP committee needs to modify the training on verification to include specific direction for the verifier on the HTST charts and what items to verify.
20. No requirement in the document for calibration of non CCP monitoring devices.
21. The centralized approach to maintaining the HACCP written program, oversight and verification creates a huge burden on one individual resulting in some deficiencies caused by lack of time and failure to delegate.

Records

Findings

The HACCP pilot generated numerous records. In some cases, HACCP was integrated with corporate quality control systems or with ISO 9000 or other management systems adding to the deluge of documents.

1. Document control is a big issue in HACCP plants. Some documents were inadequately identified causing the paper trail to be confusing.
2. A master list of monitoring records found in a couple of plants was extremely helpful.
3. Some plants included copies of blank monitoring forms with the HACCP plan. This was also found to be helpful.
4. Documents referred to in the hazard analysis had been changed or their use had been discontinued.
5. Sometimes the current revision of documents was not clear due to failure to date the form and properly identify the revision.
6. Pasteurization records sometimes did not contain information required by the PMO. The Phase II pilot document was not completely clear regarding this.
7. In some plant, the large numbers of automatic records produced as in the case of mechanical cleaning records made them difficult to verify. Some plants were not taking advantage of the ability of their systems to produce automated mechanical cleaning exception reports to alleviate this deluge of records that needed to be checked.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. Each page of the written PPs, hazard analysis, HACCP plan and other pages of the HACCP program should be dated, with a new date added whenever the page is updated.
2. Prerequisites are decentralized and not readily available in one area
3. Include a summary page at the beginning of the HACCP program that dates and briefly describes each change to the written HACCP program.
4. There is a strong need for a standard format for records produced in the plant. Most formats for records in this plant are similar and relatively easy to follow. A suggestion would be that each record have an identifying name and a place for comments and verifier's remarks and signature.
5. HACCP certification has directly and positively affected the plants ability to acquire new business.
6. Pilot document-- master list of monitoring records would be especially helpful.
7. Pasteurizer charts do not record low flow alarms or loss of signal or high flow rate alarms.
8. The cleaning charts show cleaning circuits being repeatedly not identified (5 of same circuit charts had one or more missed circuits in last 20 days). Need to be addressed by HACCP team and production personnel to achieve permanent correction. Five unidentified cleaning circuits in last 25 days on CIP charts.
9. A centralized deviation log was complete and accurate but could have been better organized.
10. Silo temperature monitoring instruments were malfunctioning and had done so for an extended period of time. Log being used to monitor temperatures was not adequate.
11. A ledger should be established to record the details of all pasteurized equipment checks on one page. State and plant should maintain ledgers for each pasteurizer in order to accurately and easily determine whether all required tests are being done at required frequency.
12. The HACCP Committee should provide more written guidance on what is expected on HTST pasteurization charts and computer controls related to public health controls (use PMO as guidance).
13. Cut in and cut out not being conducted properly
14. Charts have improved substantially since last visit
15. Last four state equipment tests required adjusting the recording thermometer. No adjustments noted on the chart by operators
16. Regulatory tests show the cutout between 167.5°F and 167.8°F but charts do not reflect that. Charts show at 167°F. Temperature changes too fast to discern.
17. For one of the pasteurization equipment control tests conducted by the state regulatory auditor, the cut out temperature was recorded as 164° F but the legal "cutout" temperature should have been at least 166° F. There were no indication that the regulatory auditor recognized this was below acceptable pasteurization temperatures. The cut out temperature on the daily operational recorder charts was listed as 166° F before and after the incident. The cut out temperatures on the pasteurization equipment tests conducted by the state, prior to and after the incident, were recorded as 166° F.
18. A document control program should be put in place to identify PPs and procedures as well as their related documents and records. This should include all HACCP plan documentation. The Table of Contents for the various PPs has numerous mistakes, as mentioned before, titles are inconsistent.
19. Recommend summary of records for PP and CCP monitoring and verification records be included in the HACCP program. The Evaluation Team had some difficulty locating all HACCP program documents. The NCIMS HACCP Committee needs to be more specific in the pilot document and training regarding the listing of documents used to monitor prerequisite programs and CCPs. Some ideas include a summary records table or examples of blank documents to facilitate verification and auditing. Also, the storage of these records should be organized and identified.
20. The HACCP Committee should add language or recommendations to improve the effectiveness of the employee HACCP training program so when a form is provided for employee monitoring, a short description of the procedure to be carried out should be documented and training of the employee should be documented. This training should be based on the procedure for carrying out the program, how to fill out the form, correcting entry errors, when comments are required on the form, and who to notify or which corrective actions are to be taken when there is a departure from the procedure.

Regulatory Sample Testing

Findings

Regulatory samples were being tested in accordance with the current PMO and associated documents.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. State regulatory sampling was complete and timely, except for fruit on the bottom yogurt which was produced for 3 months before the first samples were taken.
2. HA lists drug testing for condensed milk as CCP. Charm SL test being used for this and had not been evaluated and accepted by FDA for this product.
3. Regulatory samples met minimum frequencies for products and water with a few shortcomings listed below.
   1. a) Warning letters not sent as necessary.
   2. b) Condensed milk only 3 samples from 2/2000 to 4/2001.
   3. c) Eggnog twice sampled in Dec. 2000.
   4. d) No samples for fruit on the bottom cottage cheese.

Pasteurization and Aseptic Processing Equipment Testing and Sealing

Findings

Protocols for who conducted the testing and sealing of pasteurization systems varied from state to state. It was the intention of the pilot program to allow the plants to be responsible for routine testing and sealing of the pasteurization system, with the regulatory agency present to verify that the tests were being properly conducted at least semiannually.

1. Some states were reluctant to allow the plants to conduct testing and sealing on their own.
2. A few plants wished to rely on the regulatory agency to conduct testing and sealing.
3. In states where testing and sealing was conducted by qualified plant personnel, there seemed to be no more issues with plant personnel conducting pasteurization checks than with the regulatory person performing these procedures.
4. Criteria for testing vat pasteurizing equipment needs to be delineated. There is a question as to whether state regulators must physically supervise vat pasteurization recorder tests semiannually.
5. There was confusion about how to deal with aseptic processing critical factors. Should they be a CCP?

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. How to deal with regulatory agencies and 6 month test for aseptic and vat pasteurizers.
2. Process records were complete, accurate and properly documented. This is substantially improved from the last time.
3. Timing pump was not sealed.
4. Product balance tank was uncovered. Note: if the lid had been in place and the return line properly connected, there would have been a cross connection with raw product.
5. Water balance tank chemical line was left connected during processing. If disconnected, this swing line is blocked by pipes and cannot be swung free from the tank
6. The NCIMS HACCP Committee needs to resolve a difference of interpretation regarding the authority and responsibility of pasteurized equipment checks by the state regulatory agency and the plant.
7. Pasteurizer and processing charts do not record low flow alarms or loss of signal or high flow rate alarms
8. How to address critical factors under a low acid canned food program in HACCP.

Training and Standardization

Findings

Prior to the Evaluation Team's work, training seemed to be adequate for the pilot, but the Evaluation Team found that there are gaps in the training of plant and regulatory personnel across both HACCP and the traditional system.

1. Some regulatory personnel, especially state auditors and listing officers, expressed a lack of confidence in fully understanding their roles and responsibilities, particularly with regard to regulatory and listing decision points.
2. Some regulators expressed a lack of understanding in how to properly fill out the audit report form.
3. Opportunities to address the findings mentioned in other categories in training courses abound.
4. While industry seems to have adequate resources for training, some state regulatory agencies could not spare the personnel time to send people to training even when the FDA was paying the travel and expenses.
5. Verifiers were often overwhelmed because of the requirement that only those who have been trained under the Dairy Foods HACCP Core Curriculum may perform required records reviews.
6. Prerequisite program records verification under the HACCP system is time consuming and may not be appropriate to the importance to the record being reviewed. Other plant personnel might be able to accomplish the task as well as the HACCP-trained individual, especially for those PP's that are not used to reduce the likelihood of occurrence of a potential hazard.
7. Those plants that had an on-going and periodic training program with comprehensively documented programs had more robust and better functioning HACCP programs.
8. The team noted some cases where the training of the regulator and the plant personnel seemed to be inadequate in pasteurizing system installation, operation, controls and testing.
9. National training with the pilot plants and participating state agencies receiving the same training under the Dairy HACCP Core Curriculum has been helpful. Since industry has available to them other training, and the FDA is planning to use standardized training on the HACCP principles, minimum requirements for this training and specialized training under the core curriculum will need to be considered.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. The plant had not established a documented system for training of individuals with monitoring responsibilities for CCP(s); however, annual HACCP training for all plant employees was conducted and documented.
2. HACCP Committee needs to change the document to clarify that prerequisite document monitoring does not have to be conducted by nationally trained personnel.
3. Record review takes a lot of time for the verifier, the in-house auditor, the state regulator and other third party auditors because of the organization and storage of the records as well as the centralization of the records and verification is primarily assigned to one individual.
4. The state listing audit and the state regulatory audits adequately and comprehensively address the plant sanitation requirements but fail to adequately evaluate the plant's HACCP System. Specifically, the audits failed to identify some deficiencies, non-conformities and organization of verification activities. The state HACCP audit reports indicated the state was inspecting the plant for floors, walls and ceilings rather than to audit against the plant HACCP system's ability to continuously assure dairy product safety.
5. The centralized approach to maintaining the HACCP written program, oversight and verification as practiced in this plant creates a burden on one individual. It may be a better use of plant resources to spread the responsibilities for training, records verification and HACCP program maintenance over the entire HACCP team. Freeing up the HACCP coordinator should allow more time to verify the system as a whole for complete implementation and thoughtful analysis.
6. HACCP team was divided into development and compliance teams; the evaluation team found this useful.
7. Plant stated that state demanded use of specific HA form that proved difficult to understand and state provided no assistance in filling out.
8. The prerequisite program and CCPs in the written HACCP program were written and implemented at an appropriate level of detail resulting in almost complete documentation and verification. Frequent HACCP written program updating resulted in some inconsistencies in terminology that create unnecessary difficulties for auditors. There were some minor inconsistencies in the written HACCP program and hazard analysis with prerequisite names, within products, between products with the same hazard profile and between the hazard analysis and prerequisite programs.
9. Both the state and FDA regulatory personnel expressed the opinion that a HACCP listing implies increased expectations beyond the traditional PMO program. The NCIMS HACCP Committee needs to clarify that the intent of the HACCP pilot is to provide an equivalent level of product safety, noting that there is a recognized potential under the HACCP pilot to achieve a higher level of assurance or confidence in the safety of the product which may result from increased documentation, verification, resources, regulatory presence, etc.
10. Does a person who verifies critical factors have to go to HACCP training when they are a certified supervisor?

HACCP Audits, Listing Procedures, and Follow-up Actions

Findings

Mainly, the routine regulator is responsible for auditing the HACCP system although the listing officer and the regional milk specialist have less frequent roles.

1. The audit frequencies delineated in the HACCP document are confusing and need to be clarified.
2. There is a perception among some state and FDA regulatory personnel that HACCP plants are to be held to a higher standard of public health assurance, with increased expectations beyond the traditional program, rather than being a true alternative.
3. Some states did not implement the 6-month audit interval and the plant equipment testing provisions. This somewhat hindered the ability of the Evaluation Team to fully evaluate these provisions of the pilot although there were no obvious negatives encountered where they were implemented. It was noted that state regulators were in the plant often for purposes other than scheduled audits.
4. There was a question about how to deal with repeat violations from audit to audit and even what constituted a repeat violation.
5. One state used the HACCP Audit Verification Form found on the web site for the pre audit interview. This seemed to give the regulator a better basis to plan the regulatory audit.
6. Correction timelines were being used for the findings of most regulatory audits. They were being noted differently from state to state. Follow-up varied substantially in practice. Some states noted follow-up on the ensuing audit.
7. The audit form does not have a space to note that it is a follow-up audit.

Observations: 

Note - These observations are very site and date specific, representing individual team member's observations. Individual observations were captured for further discussion and later evaluation by the complete Evaluation Team. Many of the observations are based on requirements listed in the plant's HACCP written program and are not requirements of the NCIMS HACCP pilot program.

1. How should the committee deal with repeat violations; answer is - same as existing system.
2. Question for committee-- how should the committee deal with repeat violations?
3. Should repeat violations result in a notice of intent to suspend?
4. Dealing with repeat violations should be added to audit form as a notice of intent to suspend.
5. Use of the regulatory summary sheet is a useful tool and we like it.
6. Protection from contamination marked 7of 7 times.
7. The HACCP Audit Verification form should be used for pre-audit interview.
8. Recommend that the committee change section 12 on audit form to reflect "previous audit findings corrected" so corrections and failures to correct will be documented on supplemental page.
9. All the recent state audits had correction timelines established.
10. Recommend that the Training Subcommittee recommend the HACCP Audit Verification form be used for pre-audit interview auditing.
11. Received goat milk at 47 °F and held 33 hours before processing ending at 50°F.
12. Plant not listed for goat milk on the listing audit and receives goat milk from unlisted source, we think
13. Priority for addressing problems that was high for state regulatory under traditional system now needs to be shifted to HACCP team verification. This shift has not occurred yet at this plant
14. The plant has very good documentation of most activities in HACCP. There are cases where it is hard to navigate the system because it is hard to move from one document to another. Improving the connections between documentation pieces should reduce verification time and regulatory time spent.
15. The HACCP Committee may want to include a box at the top of the audit form which lists the various types of audits and allows the user to check the applicable purpose of the audit.
16. The HACCP Committee may want to develop a summary sheet of observations and findings to be included with the audit report so the auditor has a place to put details of their findings and observations.
17. The level of commitment, oversight and auditing by the state auditor and state listing officer demonstrate strong state regulatory oversight of the HACCP pilot plant, using advisory visits, corrective action time lines, and audit follow-up visits.
18. The over commitment of resources (maintaining four month audit frequencies and monthly plant visits) to monitor this HACCP pilot plant has caused the state regulatory agency to be very reluctant to participate with another volunteer HACCP pilot plant. The state has an opportunity to reduce the frequency and length of follow-up visits freeing resources based on the observations of the Evaluation Team. The state was not willing at this time to accept other plants into the HACCP pilot primarily because of resource limitations.
19. The Evaluation Team notes the intent of the state regulatory agency to upgrade its computerized dairy records system. The current computerized records system results in data summaries, which are less than complete and reliable. This creates difficulty in the evaluation of product and water sampling records.
20. Audit reports appeared to be well done and good follow-up but on wrong form (Phase I). Listing audit was done before Jan. 2002 training using Phase I form without attached flow diagram or product description. If the new Phase II form been used, observation regarding HTST #4 not included in HACCP program would have been considered a critical listing element.
21. Mold was noted in cottage cheese caused by condensate.

FDA State Program Evaluations & Follow-up Actions to be Taken

Findings

The state program evaluations for the HACCP pilot have not been completed by FDA Regional Milk Specialists to make them available for the review by the Evaluation Team.

Observations: 

- None

Acknowledgments

The evaluation committee wishes to acknowledge the participating pilot plants, state regulatory agencies and FDA in both Phase I and Phase II of the pilot. The commitment of the participants was considerable in both time and resources. The state and industry participants endured a heightened level of scrutiny as they had to produce and document programs and procedures, justify decisions, rewrite documents and commit to training. In addition these brave souls had to endure the imposition, questionnaires, and probing of an Evaluation Team determined to evaluate every aspect of implementation of the HACCP program. The participants accomplished this with grace and hospitality. The Evaluation Team salutes them for their contribution to the understanding of how this program should be implemented.

We wish to thank the NCIMS HACCP Committee for its guidance and moral support. Thanks also to the FDA for support of travel without which the team could not have conducted its evaluations.

The Team acknowledges the commitment of its members who donated their time, endured a summer of airports and hotels, late night evaluation sessions and innumerable conference calls and report revisions. We also wish to thank Richard Graham who came out of what could otherwise have been a peaceful retirement to encourage, cajole, and otherwise lead a diverse group of dairy industry professionals with a common mission.

Evaluation Team Members:
Richard Graham-Chairman	Dr. John Rushing	Randy Arbaugh
Steve Sims	John Beers	Dr. Bill Sveum
Allen Sayler

Appendices

Appendix I. Industry, State Auditor, State Rating Officer, State Program Administrator and FDA Regional Milk Specialist Questionnaires- Responses, and Summaries

Appendix II. Listening Session Summary and Response

Appendix III. Summary of Regulatory Information (Inspections, HACCP Audits, State Ratings, Pre and Post HACCP Evaluations, FDA Check-Ratings, Regulatory samples)

Appendix IV. Positive Observations and Comments from Plant Visits

---

* Since HACCP is science-based, members of the HACCP Team must have available adequate technical resources to assist in the identification of hazards, choosing controls, and system verification. In order to provide assistance in conducting the hazard analysis and developing the rest of the plan, the Evaluation Team recommends the development of a "Hazards and Controls Guide" and models of HACCP plans. The Evaluation Team also recommends that a model prerequisite program as well as guidelines for the implementation of required PPs should be developed and made available for use in future training.