NISTIR 6014

THE ABC's OF THE U.S. CONFORMITY ASSESSMENT SYSTEM

Maureen A. Breitenberg
Office of Standards Services
National Institute of Standards and Technology
Gaithersburg, MD 20899

April 1997

TABLE OF CONTENTS

• Foreword
• Acknowledgments
• Abstract
• Introduction
• Standardization
• Conformity Assessment
• Inspection
• Testing
• Criteria for Laboratory Evaluation
• Laboratory Accreditation
• Laboratory Accreditation Evaluation Criteria
• Other Considerations
• U.S. Laboratory Accreditation System
• Certification
• Types of Certification
• Third Party Certification Programs in the United States
• Choice of Standards
• Certification Methodology
• Certificates or Marks of Conformity (Certification Marks)
• Certification Program Evaluation Criteria
• Other Considerations
• U.S. Certification Program Accreditation System
• Management System Assessment/Registration
• The ISO 9000 Standards
• The ISO 14000 Standards
• Management System Assessors
• Management System Registration
• Accreditation of U.S. Registrars
• Accreditation Program Recognition
• International/Regional Cooperation in Conformity Assessment Summary
• Appendix I --- List of Relevant ISO/IEC Documents
• Appendix II -- List of Acronyms
• Appendix III - Information Available from the Office of Standards Services

FOREWORD

The Office of Standards Services periodically publishes information related to standards and conformity assessment as a service to producers and users of such systems -- both in the government and in the private sector. This report provides those not fully familiar with these fields with an introduction to some of the complexities. We hope that this material will be informative and will serve to stimulate wider understanding of the purpose and nature of the various aspects of conformity assessment, as well as interrelationships among related activities. The interested reader may wish to take advantage of other available publications and services provided by the Office of Standards Services, some of which are described in Appendix III.

ACKNOWLEDGMENTS

I would like to thank John Donaldson, ANSI; Charles Hyer, The Marley Organization; Walter Leight, NIST; and Krista Johnsen Leuteritz, NIST; and many others for their careful review of and comments on this document.

Maureen A. Breitenberg
Office of Standards Services

ABSTRACT

This report is designed to provide the reader with an introduction to conformity assessment and information on how the various conformity assessment activities are interlinked. It highlights some of the field's more important aspects and serves as background for using available documents and services.

Key Words: accreditation; certificates of conformity; certification; certification marks; conformance testing; conformity; environmental management systems; inspection; laboratory accreditation; product listing; quality assurance; quality system; quality system registration; recognition; registration; standardization; standards; testing

INTRODUCTION

The U.S. marketplace is becoming increasingly globalized as evidenced by the vast array of products made in foreign countries which are available to U.S. consumers. This growing complexity has made buyers increasingly dependent on formal methods and procedures for ensuring that the products, services, and systems that they purchase -- whether domestic or foreign -- consistently meet their needs.

Some product characteristics are vital for safe and effective performance, and many of these characteristics cannot be evaluated simply by picking up and examining the product in the marketplace. Such characteristics need to be determined, assessed, and assurance provided to the buyer (or other interested party) that the product conforms to requirements and that conformance is consistent from product to product. For example, if a buyer determines that it is critical that an airplane part be able to withstand pressure of at least 5000 kilograms per square centimeter, then the buyer needs assurance that none of the purchased parts will fail to withstand such pressure. The buyer cannot determine whether a part meets this requirement by simply looking at it.

Conformity assessment is defined in ISO/IEC (1) Guide 2: 1996 as: "any activity concerned with determining directly or indirectly that relevant requirements are fulfilled." Conformity assessment procedures provide a means of ensuring that the products, services, or systems produced or operated have the required characteristics, and that these characteristics are consistent from product to product, service to service, or system to system. Conformity assessment includes: sampling and testing; inspection; certification; and quality and environmental system assessment and registration. It also includes accreditation of the competence of those activities by a third party and recognition (usually by a government agency) of an accreditation program's capability.

While each of these activities is a distinct operation, they are closely interrelated. The inclusion or absence of any of these activities, as well as the quality with which any one of them is performed, can have a significant effect on the confidence and reliance that can be placed on the results of the entire conformity assessment process. In addition, standards, which underlie each of these activities, can also have a major impact on the outcome of each specific conformity assessment activity as well as a cumulative effect on the outcome of the entire process. Conformity assessment activities form a vital link between standards (which define necessary characteristics or requirements for products) and the products themselves. Together standards and conformity assessment activities impact almost every aspect of life in the United States.

This impact is especially evident when one considers specific product examples. A state-of-the-art computer is of no benefit without compatible software. A new and technologically superior appliance is useless if its plug does not fit the outlet; or (worse yet!) appears to fit, but actually increases the potential for fire or electrical shock. We do not want to purchase a product that appears to meet our needs and then discover that it has potentially dangerous undetected defects. If we purchase products on a regular basis, we also don't want to find that they are periodically unacceptable because of unexplained or unexpected variations in production or delivery processes.

Conformity assessment can verify that a particular product meets a given level of quality or safety, and provide the user with explicit or implicit information about its characteristics, the consistency of those characteristics, and/or performance of the product. Conformity assessment can also increase a buyer's confidence in a product, furnish useful information to a buyer, and help to substantiate a company's advertising and labeling claims regarding a product (2). Conformity assessment is therefore an important marketplace communications device -- a means of exchanging information between buyer and seller. It is vital for buyers, sellers, and other interested parties to understand the conformity assessment process to competently judge the value of a particular assessment scheme and to use the information resulting from that scheme to make intelligent marketplace choices.

The quality of the conformity assessment information conveyed depends on: the impartiality and competence of the assessment body; the types of assessment activities included in the scheme; and the adequacy and appropriateness of the standards against which the product is evaluated. Improperly conducted conformity assessment activities may result in widespread buyer deception. If properly conducted, however, conformity assessment can furnish valuable information to the marketplace and can serve as the basis for increased opportunities for national and international trade.

The impact of conformity assessment on both domestic and international trade was prominently noted in the 1994 Agreement of Technical Barriers to Trade (TBT Agreement) of the international General Agreement on Tariffs and Trade of the World Trade Organization (WTO). The TBT Agreement recognized that conformity assessment activities can expedite or seriously hinder the free flow of goods in international commerce and established procedural requirements for conformity assessment schemes to avoid the establishment of unnecessary obstacles to trade. The agreement requires that conformity assessment procedures be "prepared, adopted and applied so as to grant access for suppliers of like products originating in the territories of other Members [signatories to the agreement] under conditions no less favorable that those accorded to suppliers of like product of national origin or originating in any other country... ." The Agreement also requires that such procedures (3) not be "prepared, adopted or applied with a view to or with the effect of creating unnecessary obstacles to international trade." Ideally, a properly conducted conformity assessment program benefits, not hinders, the free flow of goods into the marketplace.

This report will provide the reader an overview of the topic of conformity assessment in order to better understand its impact on the marketplace. The report will: (1) discuss some concerns regarding the standards used in conformity assessment activities; (2) describe each type of conformity assessment activity and examine some of the issues involved in assessing the activity's competence; and (3) identify some of the interrelationships among conformity assessment activities.

To improve readability, this report will use the term product to refer to a product, service, process, and/or system. The reader should also note that while many examples in this paper refer to products (because such examples are generally easier to understand), the implications are equally applicable to services, processes, and systems.

STANDARDIZATION

A standard was defined by the National Standards Policy Advisory Committee (4) as:

"A prescribed set of rules, conditions, or requirements concerning definitions of terms; classification of components; specification of materials, performance, or operations; delineation of procedures; or measurement of quantity and quality in describing materials, products, systems, services, or practices."

Standards (5) are known to have existed as early as 7000 B.C. when cylindrical stones were used as units of weight in Egypt. In more modern times, the great blaze in downtown Baltimore in February 1904 and other similar catastrophes provided tragic and undeniable evidence of the importance of standards. While the fire in Baltimore burned, fire engines from as far away as New York rushed to the scene only to discover that their hoses would not fit Baltimore hydrants. Those "alien" fire engines were useless! The inferno raged for more than thirty hours, destroying 1526 buildings covering more than seventy city blocks. All electric light, telephone, telegraph, and power facilities were also razed. In contrast, 23 years later, help from 20 neighboring towns saved Fall River, Massachusetts from destruction since hydrants and hose couplings had by then been standardized in those communities.

As late as 1927, a color-blind motorist had as good (or as bad) a chance as anyone else when trying to interpret traffic signals. Purple, orange, green, blue, yellow, and red lights greeted him as he drove from state to state. In some states, green meant "Go," in others "Stop." Red, not yellow, lights meant caution in New York City. In 1927 a national code for colors was established through the work of the American Association of State Highway Officials, the National Bureau of Standards (now the National Institute of Standards and Technology (NIST)) and the National Safety Council. Imagine the chaos that would occur during rush hour in any major U.S. city today if newcomers and tourists did not know what traffic signals meant!

Probably the most significant standard ever developed in the United States, however, was the railroads' standard track gage. This standard (now also used in Great Britain, the United States, Canada and much of continental Europe) enables railroad rolling stock to cross the country.

It was the Second World War, however, that brought the urgency of extending domestic standardization to the international level. Allied supplies and facilities were severely strained due to the incompatibility of tools, replacement parts, and equipment. The War highlighted the need for standards aimed at reducing inventories and increasing compatibility.

Standards may be classified in numerous ways, some of which are described here. Standards can define words so that an industry or parties to a transaction may use a common, clearly understood language. Test method standards define the process/procedures to be used to assess the performance or other characteristics of a product. Product standards establish qualities or requirements for a product (or related group of products) to ensure that it will function safely and/or effectively. Process standards specify requirements to be met by a process (e.g., an assembly line operation) in order to function effectively. Service standards (e.g., standards for servicing or repairing a car), establish requirements to be met in order to achieve the designated purpose of this service. Interface standards (e.g., a standard for the point of connection between a telephone and a computer terminal), are concerned with the compatibility of products. Another type of standard lists product data requirements for which values must be obtained.

Still another classification scheme distinguishes between voluntary standards, which by themselves impose no obligations regarding use, and mandatory standards. A mandatory standard is generally published as part of a code, rule or regulation by a regulatory government body and imposes an obligation on specified parties to conform to it. However, the distinction between these two categories may be lost when voluntary consensus standards are referenced in government regulations, effectively making them "mandatory" standards. Voluntary consensus standards may also become "quasi-mandatory" due to conditions in the marketplace. For example, the health care industry is sensitive to the need to use the safest products available to ensure patient safety and to protect manufacturers, vendors and health care providers against lawsuits. Informed buyers of health care products will frequently insist that products meet all appropriate voluntary consensus standards. If manufacturers wish to compete effectively, their products must conform to such standards.

It is clear, then, that standards cover a broad range of types and serve a wide variety of purposes. In the United States alone, there are approximately 49 000 private sector voluntary standards developed by more than 620 organizations (6). This number does not include approximately 44 000 regulatory and procurement standards (developed and used by approximately 80 federal regulatory and procurement authorities) or other codes, rules and regulations containing standards which have been developed and adopted by state and local government authorities. There are also company standards, developed for use by a company or organization for its own products or for the products it purchases. However, some company standards gain such widespread marketplace acceptance that they can and do become de facto industry standards, such as the architecture for the personal computer established by IBM and widely used in the personal computer industry.

In addition, numerous foreign national, regional and international organizations produce standards of interest and importance to U.S. manufacturers and exporters. The International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) produce the largest number of international standards. In its 1996 catalog, ISO had 10 189 international standards, in addition to other technical documents. In 1995 alone, the IEC issued 429 standards totalling 17 000 bilingual pages.

Standards are vital tools of industry and commerce because they promote understanding between buyers and sellers and make possible mutually beneficial commercial transactions. As noted before, buyers cannot always evaluate product specifications or characteristics by inspection or even from prior experience. Information on a product's conformance (or nonconformance) to a particular standard can provide an efficient method of conveying information needed by a buyer on the product's safety and suitability.

While physical measurement standards tend to be objects (e.g., a standard meter, kilogram, etc.,) rather than pieces of paper, these types of standards are outside the scope of this paper. Most standards are written documents. Such standards have little or no significance -- unless they are used. Some standards never receive widespread acceptance and use. Others may have been accepted by industry at one time, but now apply to technologies that have become outdated. One of the most important uses for standards is within a conformity assessment process. Standards provide the basis for conformity assessment activities which, in turn, are the basis for many buyer-seller transactions. Hence standards used in conformity assessment activities can have tremendous impact on companies and nations and even on the economic fabric of the world market. Because standards have such an impact on the validity of conformity assessment activities, it is important to have some familiarity with them.

Standards can cover many aspects of the conformity assessment process. They can describe characteristics of the product for which conformity is sought; the methodology (e.g., test, inspection or other assessment methods) used to assess that conformity; or even the conformity assessment process itself (e.g., how a certification program should be operated). Standards used in conformity assessment should be clearly and concisely written, readily understood, precise, technically credible, and contain only unambiguous requirements - the absence or presence of which can be objectively verified. The use of well written standards in a conformity assessment process lends credibility and validity to the process, increasing its usefulness. Requirements in such standards should be stated in terms of "shall" or "will," rather than "may." In addition, standards for conformity assessment methods (e.g., test methods) used in the conformity assessment process should be capable of evaluating the conformity of a product to the specified requirements in a manner that produces test results that are within an acceptable accuracy range. The results should also be consistent with results produced by the same laboratory when it repeats the test using the same or a similar test method. The results should also be reproducible, i.e., capable of being duplicated by other testing bodies using the same or similar test methods.

Standards used in conformity assessment should not impede innovation. For this reason, performance standards, which describe how a product is supposed to function, are preferred over design (also called prescriptive) standards, which define how the product is to be designed or constructed (7). For example, a performance standard for water pipe might set requirements for the pressure per unit area that a pipe must withstand, along with a test method to determine if a pipe sample meets the requirement (8). Manufacturers are free to choose any product design, material, and manufacturing process as long as the pipe can perform in the specified manner. On the other hand, a standard that requires that a pipe be made of a given gage of copper and have a given diameter is a design standard. Manufacturers trying to comply with such as standard are not free to make the pipe out of stainless steel, for example, or to vary the size of the diameter no matter how such changes impact the pipe's performance. For this reason, including design requirements in standards can discourage innovation.

The example above is rather simplified. Few standards are purely design or performance in nature. Most are a mix of requirements of both types. Requirements in a standard, for example, may be mostly written in terms of performance; while the test method for ensuring conformance might be written in design terms. In fact, design requirements are frequently more appropriate for test methods where the need for accuracy and reproducibility usually outweigh other considerations. It should also be noted that the determination of conformance to performance standards may be more difficult than for design standards. For example, it is usually more difficult to determine that a pipe can perform in the specified manner than it is to determine that a pipe is made of a given gage of copper and has a given diameter. Performance standards are also more difficult to write. Therefore, the use of performance standards, while desirable, may not be practical in all situations. In some cases, the disadvantages associated with the use of performance standards may outweigh other considerations.

However, in general, when products can be defined in terms of required performance characteristics, the resulting performance standards tend to be less restrictive than design standards. Performance standards are also more likely to allow the inclusion of technological innovations in the product and to prevent unnecessary barriers to trade. This is why the United States and the other signatories to the WTO Agreement are encouraged to write technical regulations and standards in terms of performance rather than design (9).

It should also be noted that writing a poor performance standard is as easy as writing a poor design standard. A poorly written standard of EITHER type is unlikely to lead to greater technological innovation, increased trade, or to an acceptable outcome when used in a conformity assessment process.

Standards used in conformity assessment should also be chosen so that they specify all essential characteristics of a product necessary for achieving the objective of the conformity assessment activity. For example, if assurance of the electrical safety of a coffee pot is the objective, a standard that covers only the electrical safety of the coffee pot's cord and does not cover the pot's heating element would not meet the objective. Knowing what aspects of the product will be evaluated in a conformity assessment process and whether there are other aspects which might impact quality, safety, or performance allows the user of the conformity assessment data to evaluate the data's significance.

Given the large number of national and international standards, it is not surprising that a number of standards are redundant or overlapping. In a few areas (particularly building and construction), there are sometimes five or six standards that, while not identical, define functional requirements for the same type of product or material. This redundancy makes it especially important to know not only which aspects of the product are covered in a conformity assessment process, but which standard(s) was used. Requirements in two different standards covering the same characteristics may be very different. To understand the results of a conformity assessment process, the user needs to know what standard(s) was used.

CONFORMITY ASSESSMENT

In addition to information on the standards used, those who rely on conformity assessment results also need to know and understand which types of conformity assessment activities were included in the process. The following sections cover a number of conformity assessment activities including: inspection; testing; laboratory accreditation; certification programs and their accreditation; management system assessment/registration and accreditation; and the recognition (usually by a government agency) of the competence of accreditation programs.

INSPECTION

Inspection is defined in ISO/IEC Guide 2 as "conformity evaluation by observation and judgement accompanied as appropriate by measurement, testing or gauging." In the European Standard, EN 45004, inspection is defined as the "(e)xamination of a product design, product, service, process or plant, and determination of their conformity with specific requirements, or on the basis of professional judgement..." While a number of people regard inspection as an activity which is distinct from other types of conformity assessment activities, most other conformity assessment activities do involve inspection to some degree. For example, it would be unusual to test a product without first looking at it (inspecting it) to determine if it is intact or has undergone rough handing that might have damaged it and could therefore affect the outcome of the testing. Likewise, it would be unusual to assess a company's quality system, without inspecting the elements of the facility which could impact the system's operation. Because some type of inspection is included in most forms of conformity assessment and because the topic of conformity assessment is so complex, this paper will not deal with inspection as a separate type of conformity assessment activity. However, the reader should be aware that this activity exists and that inspection can and often does provide information that is used in conformity assessment. ISO has also published a guide on the topic, ISO Guide 39: 1988, "General requirements for the acceptance of inspection bodies," which is currently being updated.

TESTING

Testing laboratories support billion dollar industries and affect the entire operation of U.S. industry and the U.S. regulatory system. Each day major corporate and regulatory decisions are made based on data produced by testing laboratories.

Test data are used in many tasks including:

o product design and research;

o quality control prior to acceptance of incoming materials/components, during production, and prior to shipment/sale;

o insurance underwriting;

o meeting contractual agreements;

o satisfying government regulatory requirements;

o certification and labeling;

o buyer protection and information;

o product comparisons;

o building and structure design, construction and related engineering tasks;

o medical and health services;

o environmental protection;

o product operation, maintenance and repair;

o legal proceedings; and

o forensic work.

Flawed test data can result in defective products capable of causing serious injury or harm to the user or the environment. Defective products (such as fire detection and mitigation equipment and systems, security alarms, aircraft, and autos) can also result in serious injury or death - not only to users, but also to unsuspecting bystanders.

Testing can be performed by laboratories differing widely in size, legal status, purpose, range of testing services offered, and technical competence. They may be government regulatory laboratories, government research laboratories, or government supported laboratories -- at the federal, state or local levels. They can also be college/university laboratories, independent private sector laboratories, laboratories affiliated with or owned by industrial firms or industry associations, or manufacturers' in-house laboratories. Test laboratories can be for-profit or nonprofit. Laboratories can operate facilities in one or multiple locations; and may even operate in multiple countries. Laboratories can offer only a limited range of testing services or services in many fields. There are almost as many different types of laboratories as there are different types of users of the test data that the laboratories produce.

A test is defined by ISO/IEC Guide 2 as a: "technical operation that consists of the determination of one or more characteristics of a given product, process or service according to a specified procedure." Test data result from the performance of a test. If the test method is well written, it is sufficient that the test data comply with the test method's requirements for accuracy and variability requirements.

Accuracy (or bias) refers to the degree of departure of the test result from the "true value." For example, if a product is weighed and the result is 5.1 kg (when the actual weight is 5.0 kg), the test or measurement is inaccurate by .1 kg. The required degree of accuracy will depend on the characteristic being tested and the impact of test data accuracy on the ability of the product being tested to perform in an acceptable manner.

Variability (or precision) refers to the degree of difference between the results from several repetitions of the same test. For example, if that same product (weighing 5.0 kg) were measured three times and the weights were recorded as 5.1 kg, 4.9 kg, and 5.0 kg., these results vary less than if measurements for that product were 4.5 kg, 5.0 kg and 5.5 kg.

Variability can be further defined in terms of repeatability, which is a measure of the variation among the test results when the same or similar test is repeated within ONE laboratory. Reproducibility (or replicability) is a measure of variation of test results from similar tests conducted in DIFFERENT laboratories. Reproducibility can be a key concern in conformity assessment programs which use multiple laboratories.

Problems in the accuracy and variability of test results occur not only due to errors by the laboratory staff or defects in the test equipment, but also arise from other factors, such as flaws or variables in the test method or in the sample selection process. As noted elsewhere, the selection of good test methods is vital to the production of good test results. Because test results are a vital component of most conformity assessment programs, the use good test data is essential for the credibility of any certification program.

Standards organizations have long recognized the importance of the link between testing and laboratory competence. For example, ISO and IEC have published Guide 25, "General Requirements for the Competence of Calibration and Testing Laboratories," which establishes general requirements for laboratory competence to conduct specific calibrations or tests. Laboratory requirements in this guide include: having an appropriate organizational structure; having adequate resources and using only properly trained staff; having a good quality system; using equipment which has been adequately maintained and calibrated; conducting tests under acceptable environmental conditions and using appropriate test methods; and producing accurate, clear, unambiguous, and objective test reports. The compliance of a laboratory with ISO/IEC Guide 25 or its equivalent provides some assurance of the competence of that laboratory.

CRITERIA FOR LABORATORY EVALUATION

The following list contains general criteria which may be used in evaluating laboratories.

1. Laboratory Organization/Independence (No Conflict- of-Interest)

The laboratory should be a legal entity organized in a manner which permits satisfactory performance of all required functions. In addition, a laboratory and its staff should be impartial or independent (that is free from any outside influence), which might bias the integrity and objectivity of the work performed.

2. Financial Stability

The laboratory should have sufficient resources to enable it to properly use and maintain the test equipment and facility, to satisfactorily perform all required functions, and to adequately indemnify itself against financial liabilities/penalties resulting from its operations. The laboratory's business should also be sufficiently diversified so that the loss of a single client does not seriously jeopardize its financial well being.

3. Staff Qualifications Requirements

Each staff member in the laboratory should have the education, training, knowledge, and experience necessary to perform the tasks assigned and an appropriate level of supervision should be maintained. The training of each staff member should be kept current and documented.

4. Adequate Quality System

The laboratory should have a quality system appropriate to the type and amount of work performed. The system should be reviewed periodically by management and revised as needed to ensure continued acceptable performance. The quality system should be suitably documented in a comprehensive, up-to-date quality manual, which is readily available for consultation by staff. Internal audits should be conducted and documented by the laboratory as needed to evaluate its compliance with the requirements in its quality documentation, as well as to assess the efficiency and effectiveness of the system's operation. The results of such audits should be reported to top management and appropriate action taken and documented.

5. Sampling Requirements

If a laboratory receives test materials in quantities larger than the amount required for the test, the laboratory should sample the material in such a manner as to ensure that the sample tested is representative of the entire quantity of material received. Where sample selection is the responsibility of the test laboratory, the laboratory should use appropriate sampling methods and/or techniques.

6. Sample Control/Integrity Requirements

The laboratory should have an effective system which ensures both the identity and integrity of the test samples. Maintaining the integrity of the sample involves preventing it from being damaged during any stage of its collection, shipment, storage, or handling. Such damage can include: physical damage; loss of part of the sample due to leakage or other causes; contamination by foreign materials; failure to maintain the sample within appropriate temperature or atmospheric conditions; or other deterioration, such as deterioration resulting from samples being held too long before testing. Where the sample may be used as legal evidence, a complete record may be required on who had custody of the sample from collection through testing, and (when needed) up until its disposal.

7. Statistical Methods Requirements

The statistical methods used to interpret or to provide additional information about test data should be appropriate and adequate for the type and level of testing undertaken. Control charts, which help distinguish random errors from systematic (assignable cause) errors or variations, should be employed as needed to alert laboratory personnel to potential problems in test procedures or equipment.

8. Recordkeeping Requirements

A laboratory should maintain all test records, observations, calculations and derived data for all tests it performs for an appropriate time period or as required by law.

9. Test Report Content/Format Requirements

Test reports should include all information relevant to sample selection, test performance, and test results. It may also be useful for a laboratory that is involved in related conformity assessment work or research to note if any relationship exists between that work/research and the results contained on the test report. Test data should be displayed in a format that is easy to read and understand. Data included in such reports should be routinely audited and validated, i.e., checked for questionable values and accepted or rejected based on an established set of criteria. Audit levels (the amount of work subject to review and the extent of those reviews) should be appropriate for the type and amount of work being performed as well as the skill of the analyst or technician conducting the tests.

10. Available Operational Manuals/Instructions

The laboratory should have readily available instructions on the operation and maintenance of all materials and equipment, copies of the test methods and standards employed with any additional instructions needed on their application, sample selection and handling procedures, and any other relevant information necessary to ensure the quality of the work performed.

11. Participation in Proficiency Testing Program

The laboratory should participate in proficiency testing to the extent appropriate to ensure the competence of its testing processes. Proficiency testing allows a laboratory to compare its test results with results obtained from other laboratory(s) from tests performed on the same or similar items. Proficiency testing can provide the laboratory with valuable feedback on the competence of its testing processes.

12. Adequacy of Facilities and Equipment

The laboratory should own or have access to all equipment required to perform all test methods it conducts. In addition, the facility should require test methods to be conducted in a controlled environment to prevent any adverse effects on the accuracy of the test result. Specifically, the testing environment should be free from excessive temperatures, temperature fluctuations, dust, moisture, dryness, vibration, and electromagnetic or other interference. The laboratory should also have adequate lighting, heating and ventilation. When needed, specialized facilities such as clean rooms should be available.

13. Equipment Maintenance/Repair/Calibration Requirements

Equipment calibration, preventative maintenance and repair procedures and the choice of reference materials (10) used for calibration should be appropriate for the nature and amount of work being performed. Equipment calibrations should be traceable (11) to some ultimate or national reference standard (12). Complete records should be maintained on all calibration, maintenance and repair procedures performed.

14. Adequate Control over Subcontractors

The laboratory should have a system to ensure that testing and related work performed by any of its subcontractors is at an acceptable level of quality. The nature of the system should be appropriate for the type and amount of work being performed by the subcontractor.

15. Appeals Procedure

The laboratory should have a mechanism to deal with technical questions, appeals, complaints and challenges, originating either from the customer or from interested regulatory or other parties.

LABORATORY ACCREDITATION

While a laboratory can self-declare or self-certify its compliance with requirements such as those listed above, or laboratory users can validate a laboratory's compliance for themselves, another method to ensure the quality and accuracy of laboratory test results is through the use of an independent third party. A third party (independent of the laboratory and the purchasers or users of laboratory testing and test data) can accredit the competence of a laboratory to conduct specific tests or to operate in specific fields.

Accreditation is defined in ISO/IEC Guide 2 as a: "procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks." In the case of laboratory accreditation, the tasks are tests or specific types of tests. Laboratory accreditation does not provide any guarantees about the test results obtained from the performance of any specific test procedures. Laboratory accreditation means that the laboratory is capable of performing specified test methods and procedures correctly, not that the laboratory has competently tested all products in each and every instance. In addition, accreditation provides assurance about a laboratory's capability SOLELY within the scope or areas for which accreditation was granted. If a laboratory is accredited to test concrete, no assurance is provided regarding that laboratory's ability to test any product other than concrete. It should also be noted that not being accredited does not necessarily imply that the laboratory is not technically competent since not all laboratories seek or require accreditation, and accreditation programs may not exist in the laboratory's field of operation.

Users of test results must understand the laboratory accreditation process, the criteria used, and the scope of accreditation to assess the value of that accreditation. This knowledge is necessary to make intelligent choices regarding the use of test results produced by the accredited laboratories. While laboratory accreditation is related to other aspects of conformity assessment, it is nonetheless a distinct procedure. Unlike product or service certification (which is described later in this report), laboratory accreditation involves neither the review nor assessment of products or services. While accredited laboratories may be used in certification programs, laboratories may be accredited to conduct tests (such as EPA's accreditation program for laboratories testing drinking water) in fields where no certification program exists. Some certification programs also use laboratories that are not accredited.

Accreditation may recognize both the technical competence and impartiality (lack of conflicts-of-interest) of a testing laboratory, or only its technical competence. The inclusion or exclusion of a requirement for impartiality is only one of many differences among the various approval criteria used in U.S. laboratory accreditation schemes. Such differences in requirements or criteria for accreditation must be considered in evaluating a particular scheme.

Since most U.S. laboratory accreditation/designation schemes were designed to meet particular governmental or private sector needs, such schemes tend to take distinctive forms and use different sets of procedures to ensure that a laboratory has sufficient competence to perform the specified testing. Some schemes involve only a simple review of data submitted by a laboratory with no attempt at verification. Others require a full scale on-site evaluation of the laboratory's facilities, staff and equipment, and include a review of the results of internal audits, quality system reviews, and proficiency testing. Comparable programs, even those which are conducted by the same organization or government agency, may include different types or number of assessment procedures and may provide different degrees of assurance regarding a laboratory's competence.

Meeting regulatory requirements is probably the primary reason for a laboratory to participate in an accreditation program. Congress, state, and local governments can mandate that required testing be done by an accredited laboratory. Federal, state and local government agencies may also impose a requirement for laboratory accreditation through regulations issued under their own legal authority. Laboratories may also require accreditation to meet testing requirements imposed by foreign governments on products imported into their countries.

However, laboratories may also wish to participate in an accreditation program as: (1) an outside check of their internal quality control program; (2) proof of competence to higher level management within the organization; (3) a competitive advantage over other unaccredited laboratories; (4) a means of protection in liability proceedings; or (4) a means of establishing credibility with the public. In addition, contracts or procurement requirements may mandate the use of an accredited laboratory to conduct any required testing. A laboratory desiring to compete for such work would have to seek accreditation. Laboratories can also sometimes secure reduced medical care and related insurance rates by providing proof of a safe working environment through accreditation. An accreditation requirement could also be imposed as a condition of sale by a purchaser of a laboratory.

The reasons for and benefits of seeking accreditation are as diverse as the laboratories themselves. In general, however, laboratories participate in accreditation programs in expectation of some type of economic return (such as increased business) on the resources they invest in obtaining the accreditation.

Eligibility requirements for laboratories seeking accreditation vary among programs. Some federal programs restrict eligibility to state government laboratories, such as NIST's program to accredit state weights and measures laboratories. Other programs place different restrictions on eligibility. For example, the U.S. Department of Agriculture (USDA) program for inspecting grain for export restricts the number of approved laboratories in a particular geographical area to ensure that each approved laboratory has an adequate market share.

Some accreditation programs are restricted to laboratories operated by the body doing the accreditation, such as the Department of Veterans Affairs' (VA) program to accredit the laboratories of its medical centers. This type of program falls into a categorical gray area somewhere between laboratory accreditation and internal quality assurance -- since elements of both are present. In other programs, eligibility may be restricted to those holding membership in the accrediting organization. However, it is generally desirable that a program impose as few restrictions on eligibility as possible.

The terminology used to refer to a laboratory which satisfies the criteria established by a laboratory evaluation program also varies greatly. Some programs use the term "accredited," while others use the term "designated." Other programs refer to such a laboratory as "inspected," "accepted," or even "nationally recognized." Even the same term, such as "accredited," can be used by different programs to mean very different types of assessments. Different terms may also have different legal implications or may reflect differences between various agencies' or organizations' legal authority.

The term laboratory "designation" is increasingly being used rather than "accreditation" for schemes in which government agencies, public authorities, certification bodies, companies and others identify or "designate" one or more laboratories to perform specific types of testing for their own use or for the purpose of implementing regulations, standards, or specifications in which the organization or agency has an interest. Regulators, certifiers and others sometimes designate test laboratories through a contractual or similar relationship with the laboratory. The use of the term "designation" may be preferable, particularly if the depth of technical competence assessment is less than what is usually encompassed under accreditation. A designation type program is usually conducted by a body for its own use, rather than for use by others.

One should note the distinction between the acceptance body (the organization responsible for accepting and using the data produced by an accredited laboratory) and the accrediting body (the body that administers the laboratory accreditation program and issues the accreditation). For example, the accreditation body for a laboratory which tests radiation dosimeters may be NIST's National Voluntary Laboratory Accreditation Program (NVLAP), but the acceptance body -- the body which uses the data to regulate dosimeters -- is the Nuclear Regulatory Commission (NRC). This is an important difference that the reader should be aware of when reviewing information on laboratory accreditation programs.

Laboratories can be accredited in several ways. One way is for the laboratory to be accredited to test in an entire field of testing. The American Society for Testing and Materials (ASTM) Standard E 1224-94: Categorizing Fields of Capability for Laboratory Accreditation Purposes defines a field of testing as a "broad sphere of science, engineering, or technology used to describe a general area of testing for classification purposes." ASTM E 1224-94 lists these fields as acoustic and vibration testing, biological testing, chemical testing, electrical testing, ionizing radiation, mechanical testing, metrology,

non-destructive testing, optics and photometry, and thermal testing.

A laboratory can also be accredited in a scientific discipline, such as biochemistry, or for a specific technology (such as gene splicing), or in relation to specific products, such as blood product testing or concrete sample testing. Accreditation can also be limited to the conduct of specified test methods.

In general, the broader the scope of approval, the more difficult and time consuming it is for the accrediting body to thoroughly assess the laboratory's ability to perform all test methods and to test all sizes and types of products within that scope. However, the narrower the scope of accreditation, the more likely it is that a laboratory performing a broad range of testing will have to obtain multiple accreditations.

There are a number of standards or guidelines on how to operate a laboratory accreditation program, such as ISO/IEC Guide 58: 1992 - Calibration and testing laboratory accreditation systems - General requirements for operation and recognition. In the United States, a number of organizations have also attempted to address the issue of judging technical competence of laboratory accreditation bodies through standards documents, such as the American Society for Testing and Materials (ASTM) E 994-1995, "Guide Calibration and Testing Laboratory Accreditation Systems: General Requirements for Operation and Recognition."

Not all laboratory accreditation programs are equally thorough and not all laboratory accreditation bodies are equally competent. The users of accreditation, accredited laboratories, and the test data produced by accredited laboratories should be aware of the following factors when evaluating a laboratory's accreditation or an accreditation process. These factors may affect the ability of the accreditation process to provide assurance that an accredited laboratory is capable of producing accurate and precise test data that is well accepted by all users.

LABORATORY ACCREDITATION EVALUATION CRITERIA

1. Accreditation Body/Independence (No Conflict-of- Interest)

The accreditation body should be a legal entity organized in a manner which permits the satisfactory performance of all required functions. If the process is to be credible, the accrediting body and its assessors should be free from any outside influence which might bias the results of the accreditation process. No relationship should exist between the laboratory and the accreditation body or its assessors which might influence the objectivity and outcome of the accreditation process.

2. Accreditation Body Procedures

The laboratory accreditation body should have published procedures on its operations, including procedures for granting, maintaining, modifying, suspending and withdrawing accreditation; and for maintaining the confidentiality of proprietary information. In general, participation in the accreditation process should not be conditional upon a laboratory's membership in any association or organization. Unjustified fees, financial requirements, or other conditions for application, which restrict participation and are not relevant to the competence of the laboratory, should also be avoided.

3. Financial Stability

The body should have sufficient resources to satisfactorily perform all required functions, and to adequately indemnify itself against financial liabilities/penalties resulting from its operations. The body should have sufficient financial resources to allow it to refuse accreditation should that prove necessary.

4. Staff Qualifications Requirements

The accreditation body should be able to demonstrate that its personnel are qualified. Assessors should have the knowledge and experience necessary to perform all required assessments in the accreditation process, including detailed knowledge of the assessment criteria and the standards, test methods, and equipment involved. Assessors should also be able to effectively communicate the results of the assessment. Each staff member's training should be kept current and documented. Financial benefit accruing to the accreditation body from assessor training should be avoided to discourage the training of marginally qualified assessors.

5. Adequate Quality System

The body should have a quality system appropriate to the type, range and number of accreditations performed. The presence of an effective quality system in the accreditation scheme is important for maintaining confidence in the process. The system should be reviewed periodically by management and revised as needed to ensure continued acceptable performance. The quality system should be suitably documented in a comprehensive, up-to-date quality manual, which is readily available for consultation by staff. The body should conduct and document the results of internal audits as necessary to evaluate compliance with the requirements in its quality documentation and to assess the efficiency and effectiveness of the system's operation.

6. The Application Process

The accreditation body's assessment process should include a detailed application process. The application should include: the scope of accreditation being sought; an agreement by the applicant to comply with all requirements and fees imposed by the accreditation body; information on the laboratory, its staff and facilities; as well as the laboratory's quality manual and other quality documentation as appropriate.

7. Standards/Test Methods Used

The choice of standards and test methods used in a laboratory accreditation scheme has a significant impact on the validity of the scheme and the accuracy, representativeness, and reproducibility of the results. Accreditation programs should only accept and use test methods that are capable of producing accurate and precise test results.

8. The Assessment Process

The accrediting body should have a means of ensuring fair and equitable assessor selection and assignment. The accreditation body should appoint only competent and impartial assessors to conduct the assessment and provide them with any instructions and/or procedures needed. The assessment process should include: an initial meeting between the assessment team and the applicant to discuss the assessment; an evaluation of all areas of the laboratory required to ensure compliance with the accreditation requirements; documentation of the evaluation in a written report; and a follow up meeting between the team and the applicant to discuss the report and results of the assessment.

9. Adequate Control over Subcontractors

The accreditation body should have a system to ensure that any work related to the accreditation performed by a subcontractor is of acceptable quality. The nature of the system should be appropriate for the type and amount of work performed by the subcontractor(s). The body should also be able to provide interested parties with adequate evidence of the competence of all subcontractors.

10. Accreditation Decision

The accreditation body should make a decision on accreditation based on all evidence collected during the accreditation process. Responsibility for accreditation decisions should not be delegated by the accreditation body to another party.

However, in some cases, it may be useful for the accreditation body to accept as equivalent to its own accreditation, an accreditation issued by another accreditation body(s). This should only be done after the accreditation body has thoroughly reviewed the requirements, procedures, and process used by the other accreditation body(s), and the two (or more) bodies have established a formal agreement listing the terms and conditions under which such recognition will occur.

11. Recordkeeping Requirements

The accrediting body should maintain adequate records on the data collected, findings and reports related to all assessments performed; the qualifications, training and experience of each assessor; the assessment procedures used; and the laboratories accredited. All documentation regarding a laboratory should be restricted to persons or organizations considered by the accreditation body as having a legal right to such records.

12. Surveillance System

The accreditation body should have a documented system in place for laboratory surveillance and periodic reassessment (including participation by the laboratory in a proficiency testing program) to ensure that accredited laboratories continue to comply with all accreditation requirements. The laboratory should be required to assist the body as necessary in gauging continued compliance. The accrediting body should also have procedures which permit the swift withdrawal of accredited status from laboratories which fail to comply with the terms and requirements for accreditation, as well as procedures for notifying other interested parties of the change in accreditation status. The body should also require that any significant changes in an accredited laboratory's operations be promptly reported. Based on such information, the body should then decide in a timely manner whether accreditation should be continued or terminated.

13. Control of the Accreditation Document/Logo

The accrediting body should have mechanisms in place which minimize the potential for fraud or deception by a laboratory regarding its accreditation status or the meaning of that status. The body should have requirements regarding the use of its mark and the accreditation document, as well as a system in place to enforce those requirements. The body should take appropriate action against an accredited laboratory which misuses or misrepresents its mark or its accreditation or fails to continue to meet the requirements for accreditation. The body should also take action against unaccredited laboratories which misrepresent their accreditation status.

14. Appeals Procedure

Disagreements may occasionally arise among parties involved in an accreditation program. The program should include an impartial appeals mechanism to handle disagreements that cannot otherwise be resolved. Procedural requirements for appeals should be documented with minimal limitations on their timing and on who may file.

15. Accredited laboratory listing

An accreditation body should periodically publish a listing of the accredited laboratories, the scope of accreditation, and any limitations/restrictions on the availability of the testing laboratories' services.

OTHER CONSIDERATIONS

There are other considerations for evaluating the competence and usefulness of an accreditation program, these include:

1. Recognition of Laboratory Accreditation Program

Widespread recognition or acceptance of a laboratory accreditation program can reduce the need for extensive retesting of a product. Programs that are well accepted are usually more cost effective since the need for multiple and duplicative assessments is reduced. Lack of recognition can result in the need for a laboratory to obtain multiple accreditations of the same (or essentially the same) testing services, sometimes based on all or most of the same accreditation criteria.

In some cases, lack of recognition can result from inadequate knowledge or information on the program, rather than problems in the program itself. Based on information collected by Mr. Charles Hyer, the author of a number of NIST publications in this area, laboratories desiring to be accredited/designated nationwide to conduct electrical safety- related testing of construction materials, for example, have to gain the acceptance of at least 43 states, over 100 local jurisdictions, the International Conference of Building Officials (ICBO), the Building Officials and Code Administrators (BOCA), the Southern Building Code Congress International (SBCCI), a number of federal agencies, as well as of several large corporations.

Recently Congress recognized the importance of problems arising from redundant requirements in laboratory accreditation and other conformity assessment areas in its passage of the Technology Transfer and Advancement Act of 1996. Section 12 of the Act requires that federal agencies coordinate their conformity assessment activities with each other and with those of state government agencies and the private sector with the goal of "eliminating unnecessary duplication and complexity in the development and promulgation of conformity assessment requirements and measures." As a result, an effort is underway in the United States to upgrade the quality of and to facilitate mutual recognition among accreditation programs, known as the National Council for Laboratory Accreditation (NACLA). This joint government/industry program is composed of those in government and industry who actively support development of a system for recognizing the competence of testing and calibration laboratories -- both nationally and internationally. The work of this Council began in early 1997. Its efforts may help to decrease requirements for redundant accreditations and facilitate mutual recognition in the U.S. laboratory accreditation system.

In addition, the National Environmental Laboratory Accreditation Conference (NELAC), sponsored by the U.S. Environmental Protection Agency (EPA), concerned state agencies, and others involved in environmental testing, is working to promote mutual acceptance of environmental test data among states.

2. Program Scope

The scope of a laboratory accreditation program can impact the program's usefulness. Laboratory accreditation programs which are very narrow in scope -- designating or accrediting laboratories to test only a narrowly defined range of products within the agency's or organization's area of responsibility -- are usually of interest to only a few organizations. A laboratory which offers a broad range of testing services may need multiple accreditations which can increase the lab's cost of doing business and may restrict entrance into additional testing areas. On the other hand, accreditation programs which accredit laboratories to conduct too broad a scope of testing may not be able to adequately assess the competence of the laboratory for all test methods or testing areas within the scope, reducing the value of the accreditation.

3. Accreditation Costs

Accreditation costs are of concern to all laboratories, particularly smaller laboratories or laboratories that offer a broad range of testing services. Such costs must be balanced against the amount of new testing work likely to result from accreditation.

Multiple accreditations can be required to perform testing for different countries, or even for more than one state and/or local jurisdiction. Multiple accreditations may also be required for different products or testing fields -- all of which can considerably increase total accreditation costs. These costs are generally passed on to the users of the testing services.

THE U.S. LABORATORY ACCREDITATION SYSTEM

The U.S. laboratory accreditation system is different from that of most foreign countries. The majority of foreign accreditation bodies are public organizations or organizations with some direct government involvement. There is also a growing tendency in foreign countries to run national laboratory accreditation schemes in a coordinated fashion. In the United States, laboratory accreditation schemes are operated by all levels of government and by the private sector. No centralized coordinating body exists, though some coordination does take place between specific programs on their own initiative or through the intervention of other interested bodies, such as government agencies, trade associations or professional societies. This lack of coordination may be reduced as NACLA's and NELAC's work progresses.

The different types of U.S. programs are briefly described below.

1. Federal Government Laboratory Accreditation Programs

Requirements for laboratory accreditation/designation programs within the federal government vary greatly by program. While some programs, such the National Voluntary Laboratory Accreditation Program (NVLAP), are quite comprehensive; others involve only minimal review of a laboratory's qualifications. The requirements and scope of each accreditation program have generally been tailored to meet specific agency needs. With the exception of NVLAP, which was established to provide laboratory accreditation services, regulation or procurement (rather than laboratory accreditation/designation) is usually the agency's primary goal. In some cases, the accredited or designated laboratories provide only an initial product screening, with federal laboratories maintaining final responsibility for producing the test data used in enforcing regulations, such as the Mine Safety and Health Administration's (MSHA) programs for conformity assessment of mining products and equipment.

Eligibility requirements for accreditation also vary among programs. Some programs restrict applications to state laboratories only, such as NIST's program for accrediting state weights and measures laboratories. As noted above, the VA program accredits only the laboratories of its medical centers. Some agencies, such as DOD's Defense Logistics Agency (DLA), designate or accredit laboratories found to be capable of performing specific tests on products procured by that agency. Such programs are open only to potential government suppliers.

Terminology also varies by program. A 1989 GAO report noted the use of 10 different terms for accreditation with at least 18 different meanings by the 20 programs it reviewed. The same term, such as "accredited," used by different programs can mean very different types of assessments and assessment procedures. Different terms may also have different legal implications or may reflect differences between various agencies' legal authority to conduct specific programs (13).

2. State/Local Government Laboratory Accreditation Programs

States administer many laboratory accreditation programs for a variety of reasons. In most cases, state and local government agencies accredit laboratories to test regulated products and require that such products be inspected and/or tested by an approved laboratory. An example is the regulation of building and electrical products at the state or local government level by requiring that the products be tested/inspected and bear the mark of a recognized or approved testing laboratory. In other cases, the state or local government programs, like some federal programs, accredit or designate laboratories that provide a surveillance/screening service for testing product compliance. In such cases, state/local government laboratories usually maintain responsibility for producing the test data used in enforcement situations.

States also evaluate laboratories for other purposes, such as to assist them in enforcing federal regulations. For example, states have primary enforcement responsibility for the national requirements for ensuring the quality of public water systems. In carrying out this obligation, states accredit local laboratories to test drinking water. State and local authorities also accredit or designate laboratories to test products prior to their procurement by a state agency.

As with federal programs, state and local requirements for laboratory accreditation/designation vary greatly by program. Some programs are quite comprehensive, while others involve only minimal review of a laboratory capabilities. The terminology used for accreditation also varies extensively.

3. Private Sector Laboratory Accreditation Programs

Private sector accrediting bodies also administer laboratory accreditation programs for a variety of reasons. These reasons range from assisting laboratories in defending their competence in professional malpractice matters to assisting an industry to avoid government regulation by undertaking self-policing efforts. Many private sector laboratory accreditation programs are operated as an integral part of a private sector certification program. In addition, accreditation programs can be established to assist government agencies in enforcing regulations, such as in the testing of building and construction products. The criteria and terminology used in private sector laboratory accreditation/designation schemes again vary greatly.

Private sector programs also differ in size and scope. The American Association of Blood Banks (AABB) operates an accreditation program for blood bank and transfusion services which is recognized by many state authorities. The College of American Pathologists (CAP) program accredits hospitals and independent medical laboratories, and their program is used by the Department of Veterans Affairs (VA) to ensure the competence of VA medical facilities. The American Association for Laboratory Accreditation (A2LA) operates an accreditation program which accredits laboratories in a number of testing fields.

At the international level, the IEC Quality Assessment System for Electronic Components (IECQ) accredits testing facilities capable of demonstrating compliance of electronic components to prevailing requirements and standards, which are based on IEC standards. This accreditation program is part of a reciprocal worldwide certification system, in which certificates of conformity issued by an approved laboratory in one participating country are accepted by all other participating countries.

The IEC also operates the IEC System for Conformity Testing for Safety of Electrical Equipment (IECEE). The IECEE's objective is the reciprocal recognition of test results (not certification marks or certificates of conformity) among the participating countries. Equipment is tested by approved laboratories against prevailing specifications and requirements, including safety, based on IEC standards. Any electrical products for which there are appropriate IEC standards can be included within the program.

CERTIFICATION (14)

While a close interrelationship exists among standardization, testing, laboratory and certification, these areas are distinct. Certification is the process of providing assurance that a product conforms to a standard or specification or that a person is competent to perform a certain task. The use of certification in the marketplace dates back to the earliest times. An inscribed stone of the 4th Century BC, found in Eleusis, Greece in 1893, cited a decree regarding the manufacture of bronze fittings for the erection of the columns of a new stoa (portico) which became the Philonion Stoa. The bronze specified was copper/tin in a ratio of 11 parts copper to 1 part tin. Professor Varoufakis in his book, Materials Testing in Classical Greece, Technical Specifications of the 4th Century BC, argued that the discovery of such a specification implies the existence of some type of check or verification. Professor Varoufakis tested the possibility that this check could have been conducted based on color differences between polished specimens of bronze differing in tin content. He noted a distinct color difference between two specimens when the tin content differed by only 2%. A similar process for identifying and differentiating gold alloys was already in use in ancient Greece. These may have been two of the earliest forms of certification -- metals of unknown content certified as to their composition on the basis of inspections for color (15).

From 1890 to 1900, when the use of gas for domestic cooking and heating began to gain widespread popularity, many new gas appliances were developed and sold. Early records indicate that some thought was given to establishing a gas appliance testing laboratory as early as 1903. The Baltimore investigation, conducted by the National Bureau of Standards in 1923, analyzed accidents resulting from the use of domestic gas appliances and flexible gas tubing. The City of Los Angeles started a similar laboratory and field study in 1924. Both investigations pointed to the need for a certification program for gas appliances. In 1925, the American Gas Association established a laboratory to test and certify gas appliances. The first certified gas heaters and ranges appeared on the market in 1926.

Just 50 years ago, U.S. drug manufacturers could produce and sell drugs without testing them on either animals or humans and without any kind of governmental approval. Governmental action could only be taken against drugs which were misbranded or adulterated. In 1937, physicians in Tulsa, Oklahoma reported to the American Medical Association (AMA) the deaths of six patients from a liquified version of the then wonder drug sulfa. Ultimately this drug killed 107 people, mostly children, before doctors realized what was happening and the drug was recalled. This tragedy led to the enactment of the 1938 Food, Drug and Cosmetic Act, which requires that drugs be tested and approved by the Food and Drug Administration before marketing. From such early beginnings, government and private sector certification programs have grown both in size and complexity.

Some (but not all) certification programs mandate that accredited laboratories conduct any required testing, but (as noted above) there are laboratory accreditation schemes that are not associated with a certification program. Certification and laboratory accreditation programs both use standards, but not all standards are intended for these uses. Because standards, testing, laboratory accreditation, and certification are linked; strengths as well as deficiencies in any one area can have significant consequences for the other areas. For example, improvements in test method standards can significantly increase the capability of a laboratory to produce valid test data, which may then be used in a certification program.

Product certification schemes range from the simple to the complex. Many private organizations, as well as federal and state agencies in the United States, certify products ranging from electrical cords to meat products. In addition, many certification programs are operated at local government (city, township, county, etc.) levels. Consumers see evidence of the extensiveness of certification-related activities when they note the Underwriters Laboratories (UL) mark on such products as electric coffee pots and fire extinguishers, the U.S. Department of Agriculture (USDA) mark on poultry and other agricultural products, and the International Wool Secretariat's Woolmark or Woolmarkblend on wool and wool blend textile goods - only a few of the many certification marks which may appear on consumer products.

TYPES OF CERTIFICATION

1. First Party Certification or self-certification is the process by which a manufacturer or supplier declares that the product meets one or more standards based on: (1) the manufacturer's confidence in the quality control system, or (2) the results of testing or inspection the manufacturer undertakes or authorizes others to undertake on his/her behalf. This process is also known as a manufacturer's or supplier's declaration of conformity. The manufacturer's capability, integrity, and reputation determine the degree of confidence that can be placed in this type of certification.

In the United States, the criteria and procedures recommended for a self-certification program are established in American National Standards Institute (ANSI) (16) Standard Z-34.2-1987, "American National Standard for Certification - Self-Certification by Producer or Supplier." At the international level, ISO/IEC Guide 22: 1996 - "General Criteria for Supplier's Declaration of Conformity," also contains a list of such criteria and recommended procedures. Such procedures frequently include the requirement that the self-certification or manufacturer's declaration be based on conformance with all requirements of a standard unless full disclosure is made of any limitations of the certification. Usually there are additional requirements for the adequate use and maintenance of test equipment, an effective quality control system, fully-trained staff, written certification procedures, and adequate record keeping.

One of the most familiar certification programs in the United States involves the identification of the weight of motor oils by the manufacturer of conformance to the Society of Automotive Engineers (SAE) standards. Consumers will recognize such weight designations as SAE 10W-40W or 10W-30W on the motor oils that they buy. These SAE designations are placed on the motor oils by the manufacturer based on his/her own testing and quality control mechanisms. This is primarily a self-certification program, though SAE does audit manufacturers' self- certifications to ensure that SAE standards are not being misused.

2. Second Party Certification is also common in the United States. In this case, it is usually the buyer who requires and certifies that the products he/she wishes to purchase from suppliers meet one or more standards. The nature of these programs varies greatly depending on the type of product being purchased and the needs of the buyer. These certifications are generally only available to and mandatory for those companies wishing to become suppliers to that buyer.

3. Third party certification is a type of certification in which the producer's claim of conformity is validated by a technically and otherwise competent third party (a body not controlled by or under the influence of the producer or buyer. The sponsor of the third party program (the certifier) may be responsible for collecting the required data, generating test results or conducting inspections, in addition to reviewing the results of such activities and making a final determination on the product's conformance or lack of conformance. The certifier may also delegate all or part of the data collection and review activities to another party or parties. The degree of confidence that can be placed in third party certification programs varies greatly depending on (1) the number and types of testing/inspection methods used within the program to ensure product conformance, (2) the adequacy of the manufacturer's quality control system, and (3) the competence of the body which conducts the testing and/or inspection and evaluates the test results.

Recommended criteria and procedures for third party certification programs in the United States are established in ANSI Standard Z-34.1-1993, "American National Standards for Certification - Third-Party Certification Programs for Products, Processes, and Services." However, the recently issued ISO/IEC Guide 65: 1996 "General requirements for bodies operating product certification systems," developed by the ISO CASCO Committee (17) is likely to gain considerable acceptance nationally and internationally in the future.

THIRD PARTY CERTIFICATION PROGRAMS IN THE UNITED STATES

1. Private Sector Certification Programs

U.S. private sector third party certification programs, described below, are operated by many different types of organizations including:

o Professional or technical societies, such as the American Dental Association (ADA) or the American Society of Sanitary Engineering (ASSE);

o Trade associations, such as the Association of Home Appliance Manufacturers (AHAM) or the International Safe Transit Association (ISTA);

o Independent testing/inspection organizations, such as Underwriters Laboratories (UL), Factory Mutual Research Corp. (FMRC), ETL Testing Laboratories, and the MET Electrical Testing Company;

o Organizations oriented toward consumers, industrial buyers, or users of the product, such as Good Housekeeping magazine, which offers a consumer protection program for products it approves;

o Organizations composed of producers, testing laboratories, and others concerned with the well- being of an industrial group and/or its customers, such as the Solar Rating and Certification Corp. (SRCC), a nonprofit organization founded in 1980 to test and certify solar thermal panels and systems;

o Organizations comprised of government officials involved in the regulation of an industry, such as the Building Officials and Code Administrators International (BOCA), the International Conference of Building Officials (ICBO), the Southern Building Code Congress International (SBCCI), and the International Association of Plumbing and Mechanical Officials (IAPMO); and

o Other miscellaneous organizations, such as a ship classification society or nonprofit organizations representing the kosher food consumer.

One of the oldest certification programs has been operated since 1870 by the American Bureau of Shipping (ABS), a ship classification society. It is concerned with the design, construction and periodic survey of merchant ships and other marine structures. ABS certifications cover over 12,400 vessels. Other certified products include: cargo containers, mobile offshore drilling and production units, fixed offshore structures, work boats, cranes, yachts, cargo handling engines, and related machinery and materials.

Another well established industry trade association certification program is that of the American Gas Association (AGA), operated since 1925, and which has joined with its Canadian counterpart to form International Approval Services (IAS). Over 700 U.S. and Canadian manufacturers participate in IAS's program for gas appliances and accessories. IAS currently certifies a wide range of products including gas appliances and related accessories and electric and oil appliances. Products are tested in IAS facilities and requirements for certification include: a review of the construction and performance information for the product, factory and quality control inspections and annual product testing/inspections. Additional documentation may also be required. IAS publishes an annual directory of certified appliances and accessories.

More recently established appliance certification programs for room air conditioners, refrigerator-freezers, dehumidifiers, and humidifiers are operated by the Association of Home Appliance Manufacturers (AHAM). The Air-Conditioning and Refrigeration Institute (ARI) certifies appliances ranging from air conditioning equipment to water coolers and solar collectors.

A program familiar to many consumers is that of the American Dental Association (ADA), a professional society which operates a program for certifying dental materials, instruments, and equipment, including toothpaste and tooth brushes. The ADA certification program uses ADA standards and requires the submittal of a detailed application describing the product's specifications. ADA also requires self-certification by the manufacturer that the product meets the standard. Product samples obtained on the open market are tested by ADA. Upon approval, ADA authorizes the use of the ADA Seal of Certification and publishes approved product lists.

Most consumers take for granted the familiar "UL" mark on a variety of products from electrical appliances to fire extinguishers. The Underwriters Laboratories (UL), an independent testing laboratory founded in 1894, is not only a major standards writer, but also tests products and materials with respect to potential hazards to life or property, listing (18) those items which appear to pose no significant hazards. The Factory Mutual Research Corporation (FM), is another "product listing" organization, similar to UL. It is a non- profit testing and research organization whose purpose is to minimize industrial property damage through a number of activities, including product approval. FM issues lists of approved products which can vary from industrial trucks to building materials.

Gardeners might recognize the AARS (All-America Rose Selections) mark on the rose bushes they purchase. Over 40% of the roses sold in the United States bear the AARS symbol indicating that the rose bush meets certain quality requirements. The AAS (All-America Selections) mark is used in connection with seed for flowers and vegetables which meet similar requirements.

The work of other major certification organizations, although equally vital, may be less well known outside the testing/inspection community. The Maple Flooring Manufacturers Association certifies floor finishes and sealants. The National Cotton Council of America, a trade association, certifies wrapping and banding materials for cotton bales. MEA Certifications certifies carpet under the Department of Housing and Urban Development (HUD) carpet certification program developed by the Carpet and Rug Institute.

The American Society of Mechanical Engineers (ASME) accredits manufacturers of boilers, water heaters, pressure vessels and nuclear components, which, in turn, certify that their products meet ASME Code requirements through the completion of a Code data report. The National Board of Boiler and Pressure Vessel Inspectors (NBBI), an organization composed of chief boiler inspectors in the United States and Canada, registers manufacturers Code data reports and commissions the inspectors who provide independent third party inspection of products manufactured under the Code. The value of certifying these products to ASME Codes is recognized internationally.

Several building code organizations, such as the Building Officials and Code Administrators International (BOCA), the International Conference of Building Officials (ICBO), and the Southern Building Code Congress International (SBCCI), engage in certification-related activities. Composed of building, construction, zoning, and inspection officials, these organizations have developed model building codes which have been adopted by hundreds of state and local governments. The organizations evaluate products and materials and approve them for conformance with the requirements established in their building codes.

Over one hundred seventy eight private sector organizations in the United States certify or list different types of products (19).

Many others certify services. The broad range of organizations involved in certification reflects the impact of certification on a vast spectrum of interests and disciplines.

2. Federal Government Certification Programs (20)

Federal government certification programs can be classified into several general categories:

o Programs to certify products directly affecting the health or safety of the user or the general public.

o Programs which test products in order to avoid the necessity for retesting at local levels or prior to each procurement.

o Programs to provide a uniform basis for trade by assessing the quality and condition of products offered for sale.

Examples of the first type of certification program include the evaluation and approval by the Food and Drug Administration (FDA), U.S. Department of Health and Human Services, of new animal and human drugs, medical devices, biologicals, and other products; the certification by the Federal Aviation Administration (FAA), U.S. Department of Transportation, of airplanes, major airplane components, and related services; and the certification by the Mine Safety and Health Administration (MSHA), Department of Labor, of electrical equipment used in mines.

An example of the second type of program is the Department of Defense's (DoD) Qualified Products Listing (QPL) Program for parts, materials and components used in military systems. This program reduces retesting prior to each government purchase by testing products and placing those approved on appropriate QPL's. An extension of this concept also underlies the DoD Qualified Manufacturing Lists (QML's) Program, in which a manufacturer's process controls and manufacturing capabilities are evaluated and approved for an entire range of products.

An example of the third type of program is the U.S. Department of Agriculture's (USDA) voluntary program to grade and certify meat and meat products (on a voluntary basis) using uniform quality grading standards for the buying and selling of such products. The USDA also certifies dairy products, fresh and processed fruits, vegetables, nuts and related products. The National Oceanic and Atmospheric Administration (NOAA), U.S. Department of Commerce, likewise inspects and grades processed fish and shellfish at a seafood processor's request.

3. State Certification Programs

States administer many certification programs which cover a diversity of products for a variety of reasons. In some cases, states inspect or test products under authority delegated by the federal government. For example, many states inspect meat and meat products, certifying those that meet standards established by the USDA. These states then authorize the use of the appropriate USDA marks. Many states also inspect and issue certificates of conformity for manufactured homes under authority delegated by the U.S. Department of Housing and Urban Development (HUD). States may also impose additional state requirements and simultaneously check for conformity to these and federal requirements.

States also regulate products under their own authority for health and safety reasons, including amusement rides and insulation, depending on each state's perception of the health and safety impact of such products on its population. Products may be inspected and/or tested directly by the states themselves, or indirectly through a requirement that such products be inspected and/or tested and certified by an approved body, such as a nationally recognized laboratory. An example of the latter is the regulation of electrical building products by imposing a state requirement that they be tested/inspected and bear the mark of a "nationally recognized testing laboratory." The term "nationally recognized laboratory" is currently defined by each state and/or municipality.

States regulate products of direct or indirect economic importance. Florida and California, for example, inspect products important to their citrus fruit industry. Nebraska, with a considerable agricultural industry, regulates tractors through a testing program at the University of Nebraska and issues certificates of conformity for approved models. California, with its air pollution problem, stringently regulates auto emissions equipment.

States also inspect/test/certify materials, products, systems and services they procure, such as materials for the construction of state roads and bridges. In yet other cases, the states establish standards and leave enforcement (testing, inspection, etc.) to local authorities. This is sometimes true for building and construction materials.

States operate certification (also called licensing programs) to regulate the services offered by professionals in fields ranging from medicine to insurance.

CHOICE OF STANDARDS

As noted before, the standards chosen for use in certification (or other types of conformity assessment) programs should specify accurate and reproducible methods of testing for the specified characteristics if the certification program is to have any validity. The standards should also be clearly written and be capable of being uniformly interpreted. ISO/IEC Guide 7: 1994 has stated that standards suitable for product certification "....should be written in such a way that they facilitate and do not retard the development of technology." However, there are many other elements to consider when choosing standards for a certification program.

One of the most obvious considerations in selecting standards is the determination of what characteristics need to be assessed to achieve the objectives of the program. The standards should specify all those and only those characteristics and requirements that are necessary to define the properties of the product or its performance; if this is not possible, then reference should be made to other relevant standards. Unless the chosen standards do, in fact, adequately specify all product characteristics that need to be assessed, it will not be possible to fully achieve the program's objectives and the program's value may be questionable.

Certification schemes are normally established to indicate that a product meets one or more sets of acceptable or minimum requirements. But what is acceptable? What may be acceptable to one buyer may not be acceptable to another. Certification programs may choose standards that result in certified products that satisfy a particular group or organization, such as the government, or the program may choose standards expected to satisfy a larger group of buyers or users. The choice of standards, therefore, may depend on the needs of the parties likely to use the certification.

Standards containing only one set of requirements and which define only one level of safety or quality are known as "pass- fail" standards. Either the product meets the minimum requirements (it passes and is certified) or it does not (it fails). Little or no attempt is made to convey information about the relative safety or quality of one certified product compared to other certified products (such as is conveyed through a grading system). Unless other methods are used to convey such information to buyers/users, a supplier may be less motivated to produce or supply products which exceed the minimum requirements specified in the standard.

Certifiers may therefore wish to provide information on the quality or safety of one certified product relative to other certified products. This may be accomplished by selecting standards which define several levels of safety or quality and by including a grading scheme in the certification program. Information can be conveyed by stating the expected life of the product, such as the expected number of miles that tires are supposed to travel, or the versatility of the product, such as portable fire extinguishers rated for several specified types of fires, or the efficiency of the product, such as the energy efficiency rating for appliances. Buyers, assured of at least a minimum level of safety and/or quality, can then choose among certified products based on any additional level of quality or safety that they desire, other characteristics or features, and/or price.

A choice must also be made as to whether the establishment of conformity should be based on an assessment of a product's performance or its design. As noted above, performance standards, tend to be difficult to develop and difficult to use in assessing compliance. Problems may also arise in establishing reliable test methods for assessing conformity, thus potentially increasing certification costs.

The choice of standards for a certification program has a significant impact on the validity of the program, the value of the information conveyed by the certification, and the program's cost.

CERTIFICATION METHODOLOGY

Third party certification programs differ greatly from one another, and the degree of confidence in the resultant certification depends on the type of program and its comprehensiveness (the number and types of test/inspection methods used within the program to assess conformity).

The methods used in third party certification programs can be classified as follows:

1. Type-testing/Initial Inspection - This method attempts to determine if the supplier's design for the product can produce a product that conforms to a particular standard. Products from a preproduction run are inspected and/or tested or services or service providers are inspected/tested before being allowed to offer services to the public, but this method provides no information on whether the products or services supplied subsequent to the initial testing and inspection also consistently meet the specification.

2. Surveillance of the Supplier's Process - Assessment of a supplier's materials, production and control processes can, at relatively low cost, provide assurance that the supplier's quality control procedures are adequate. In some cases, a supplier's quality system must have been assessed and approved (registered) by a third party as a pre-requirement for application.

3. Audit-Testing - In this procedure, samples of products, services or individuals are selected at random from the marketplace or population and tested. Extensive testing is usually required to provide adequate assurance that group of objects/populations from which the samples are taken meet the referenced standard.

4. Field Investigations - Alleged failures of products during actual use are investigated to determine the cause of failure and to suggest appropriate corrective action.

5. Batch-testing - A sample is selected from a population or batch and tested for conformance to the standard. If the sampling procedure and the sample size are adequate, batch-testing may be used to estimate, with a specified degree of confidence, that all objects in that batch or population conform to the standard. It does not, however, ensure that an untested product or individual in the batch/population will meet the standard, nor does it furnish information on the quality of products produced in earlier or subsequent batches, or of individual in other populations. Batch testing is used in many certification programs for building products.

6. 100 Percent Testing - In this method, each product/individual is tested to determine compliance with the designated standard. If the testing procedures are adequate, the procedure provides the highest possible level of assurance that the product/individual conforms to a particular standard. It is also usually the most expensive method and can be applied only where the test has no adverse effect on the product/individual.

Many certification programs rely on two or more of these methods for their approval process. The choice of methods depends on the needs of both the buyer and the seller and on the nature of the product or service provided. The chosen methods can greatly affect both the cost of the program and the level of confidence that can be ascribed to it.

CERTIFICATES OR MARKS OF CONFORMITY (CERTIFICATION MARKS)

ISO/IEC Guide 2 defines a certification mark (a mark of conformity for certification) as a: "protected mark, applied or issued under the rules of a certification system, indicating that confidence is provided that the relevant product, process or service is in conformity with a specific standard or other normative document." The marks used in U.S. certification programs are usually (but not always) registered with the U.S. Patent and Trademark Office (PTO) under the Trade Marks Act of 1946 (Lanham Act) as amended (1994). Some of these marks are registered with PTO as certification marks, however, others may be registered as trade or collective marks (21). The requirements established and enforced by PTO for obtaining and using these different types of marks vary by type of mark.

ISO/IEC Guide 2 defines a certificate of conformity, as a: "document issued under the rules of a certification system, providing confidence that a duly identified product, process, or service is in conformity with a specific standard or other normative document."

The marks or accompanying information should indicate the identity of the certification body (and any other testing body if applicable) and any relationship that the body(s) may have to the manufacturer. In addition, the certificate of conformity should contain information on: (1) the lot, batch or other production information to allow traceability to the production source and time of production; (2) the date when the certificate was issued; and (3) the officer of the company responsible for its issuance. Labeling included with the product should identify the supplier, and contain information on the name, type or model number and all instructions necessary for the correct and safe use and maintenance.

Certification marks and certificates of conformity should be used to indicate that all essential characteristics of the product have been assessed. In cases where only one of several aspects of the product have been evaluated, such as flammability or electrical safety, this information should be conveyed in some manner to the buyer lest the mark mislead the buyer into placing more reliance on the certification than is justified. To the extent possible, the symbols used in connection with the certification mark should be capable of being interpreted without further definition.

CERTIFICATION PROGRAM EVALUATION CRITERIA

While certification programs benefit both the seller and buyer of a certified product, certification programs also have the potential for misleading users and for other abuses. Some of the more significant elements to consider when evaluating certification programs are the similar to those for evaluating laboratory accreditation programs including:

OTHER CONSIDERATIONS

There are also other considerations for evaluating the usefulness of a certification program, including:

Despite public perception and assumptions that certification implies legal responsibility for the quality and/or safety of the product certified by a third party, most certifiers are not willing to accept such responsibility. Responsibility for this liability generally rests with the supplier. Buyers and users of certification should understand these limitations. Buyers and users of a certification scheme should be adequately informed as to the purpose, scope, and technical limitations of the resultant certification so as not to be misled as to the value and degree of confidence that can be placed on the mark or certificate of conformity.

2. Compatibility with and Recognition by Other Certification Schemes.

When U.S. certification schemes for the same product differ significantly, this may result in the need for extensive retesting at considerable expense in terms of both time and money, such as having to have a product certified by multiple certifiers to meet various state requirements. The potential barriers to trade which can result from differences among U.S. and foreign national certification programs can also be considerable. Preference should be given to the use of certification programs which are compatible with other national or foreign national certification schemes while still meeting the objectives of the scheme.

In some cases, lack of recognition results from inadequacies in specific programs which need to be addressed by the responsible certifiers. However, lack of recognition can also result from inadequate knowledge or information on the program by certification users. It may also result from the vested interests of competing certifiers or others, rather than problems in the program itself. Widespread recognition or acceptance of competent certification programs should be encouraged to reduce the need for extensive retesting/inspection/certification of a product.

3. Appropriate Use of Certification in Conjunction with Quality System Requirements

As W. Edwards Deming (22) often pointed out, quality must be designed into the product and ensured through an effective and efficient manufacturing process. Inspection, testing, and certification will provide information on whether the desired end result has been achieved, but usually that information comes too late in the supplier's process to improve the quality of the inspected/tested products. Certification should

not be used as a substitute for a supplier having an effective quality system.

The costs involved in the certification process should be reasonable and in balance with the benefits likely to accrue to the supplier from having the product certified. Excessive costs in a certification process can be cause for serious concern for all suppliers (particularly for smaller suppliers), and can serve to restrict trade and competition. In most cases, certification costs are ultimately passed on to the buyers of the product.

U.S. CERTIFICATION PROGRAM ACCREDITATION SYSTEM

One of the criticisms of self-certification or manufacturer's declaration, namely that the manufacturer does not have a third party checking the competence and integrity of the results, fails to recognize that many third party certification programs also do not have anyone looking over their shoulder to ensure the quality of their programs. Like a laboratory, a certifier can self-declare or self-certify its compliance with the requirements, such as those listed in ISO Guide 65: 1996 "General requirements for bodies operating product certification systems." Users of certification results can also validate a certifier's compliance for themselves. Yet another method for ensuring the competence of certifiers is through an accreditation program conducted by an independent third party.

A third party (independent of the certifier and the purchasers or users of the certification) can accredit the competence of a certifier to conduct specific certification tasks for specific types of products. Like laboratory accreditation, certification program accreditation does not necessarily give any guarantees about the certification results obtained from the performance of specific certification procedures. Certification program accreditation means that the certifier is capable of performing specified certification procedures correctly, not that the certifier has competently certified all products in each and every instance. Again, like laboratory accreditation, certification program accreditation only provides assurance about a certifier's capability within the scope or areas for which accreditation was granted. It should also be noted that lack of accreditation does not necessarily imply that the certifier is not technically competent since not all certifiers want or require accreditation. In some cases, accreditation programs may not exist in the certifier's field of operation.

Meeting regulatory requirements is again probably the primary reason for a certifier to participate in an accreditation program. Congress, state and local governments can mandate that certifications be done by accredited certifiers. OSHA, for example, is responsible for the regulation of all electrical products used in the work place. Electrical products under OSHA's jurisdiction require approval and marking/listing by certifiers accredited by OSHA, known as Nationally Recognized Testing Laboratories (NRTL).

State and local government agencies may also impose such a requirement through regulations issued under their own legal authority. Certifiers may also have to be accredited to meet mandatory requirements imposed by foreign governments on products imported into their countries.

Like laboratories, certifiers may also participate in an accreditation program as an outside check of their internal quality control program, as proof of competence to higher level management within the organization, as a competitive advantage over other unaccredited programs, as a means of protection in liability proceedings, or as a means of establishing credibility with the public.

The reasons for and benefits of seeking accreditation are many. In general, however, certifiers (and laboratories) participate in accreditation programs in expectation of some type of economic return on the resources they invest in obtaining the accreditation, such as new or expanded business opportunities.

Again like laboratory accreditation programs, not all certifier accreditation programs are equally thorough and not all accrediting bodies are equally competent. Some factors that can be used to evaluate a certification program accreditation process were previously discussed under laboratory accreditation. These factors affect the ability of the certification program accreditation process to provide assurance that an accredited certifier is capable of competently conducting certain certification procedures for one or more products as defined in the scope of the accreditation.

The U.S. certification program accreditation system is less complex than that for laboratory accreditation, primarily because the U.S. has significantly fewer programs for accrediting certifiers than for accrediting laboratories. The two most prominent programs are the ones run by OSHA (which was mentioned above) and by ANSI.

OSHA's program covers electrical equipment/materials used in the work place. All electrical products used in the work place must be tested and listed or labelled by a certifier (known as a Nationally Recognized Testing Laboratory or NRTL) which is recognized/approved by OSHA. Participation in the OSHA program is voluntary for certifiers; however, if they wish to test such products, they must be recognized/approved by OSHA (23).

ANSI also conducts an accreditation program for certifiers which was established in the 1970s. ANSI's policies and procedures were revised in 1992 to be consistent with both relevant international standards and guides and with anticipated obligations under the WTO Agreement. ANSI accredits certifiers in a number of areas ranging from construction materials to drinking water additives to gas and electrical appliances and accessories.

MANAGEMENT SYSTEM ASSESSMENT/REGISTRATION

At the national and international level, interest has mushroomed in quality and environmental management systems. Having a quality management system helps a supplier ensure that he/she is able to produce products which consistently conform to a given set of standards or expectations. Product quality depends on many variables, such as the caliber of the components or materials used; type of equipment used in design, production, handling, installation, testing and shipping; the equipment calibration and maintenance procedures employed; the training and experience of production and supervisory personnel; the level of "workmanship;" and sometimes the environmental conditions (temperature, humidity, level of dust particles) in the area where the product is produced. The process, organizational structure, procedures, and resources that suppliers use to control these variables to produce a product of consistent quality which meets defined specifications is called a quality system (24). The standards for quality systems that are best known and that are being adopted globally are the ISO 9000 standards.

Environmental management systems (EMS) are designed to provide a comprehensive approach which can be used by businesses and other organizations to manage the impact of their activities on the environment. Environmental management system standards generally do not establish requirements for environmental compliance nor for specific levels of pollution prevention or performance (this being essentially the responsibility of the appropriate regulatory agency(s) and/or the organization's management).

However, management system standards do establish a framework which an organization can use to evaluate its performance against such levels. EMS standards, such as ISO 14001, generally stress identifying key environmental impacts (aspects) associated with the operations of an organization, setting targets and objectives for these aspects, and implementing the system via training and documentation of procedures. EMS standards, like ISO 14001, also require that the company implement a program to conduct internal audits of its EMS, to conduct periodic management reviews of the system's effectiveness, and to have a commitment to continual improvement of the EMS.

Recent publication of the first two international environmental management system standards (ISO 14001 and ISO 14004) is likely to increase awareness in the United States of the usefulness of management systems in overseeing and improving environmental performance.

THE ISO 9000 STANDARDS

The original ISO 9000 standards were a series of five international standards (ISO 9000, 9001, 9002, 9003, and 9004), developed by ISO Technical Committee (TC) 176 and published in 1987 to provide guidance on selection of an appropriate quality management program (system) for a supplier's operations. The Series is generic in scope covering both service and manufacturing industries. The 1994 edition of ISO 9001 contains requirements in twenty areas covering: management responsibility; quality system requirements; contract review; design control; document and data control; purchasing; requirements for customer-supplied product; product identification and traceability; process control; inspection and testing; control of inspection, measuring and test equipment; inspection and test status; control of nonconforming product; corrective and preventative action; handling, storage, packaging, preservation and delivery; control of quality records; internal quality audits; training; servicing; and the use of statistical techniques. ISO 9002 covers all elements except design control. ISO 9003 (which is not used to any extent in the United States) covers significantly fewer elements. The titles and content of the 1994 edition of the series are as listed in Appendix I.

The United States has been an active participant in ISO TC 176 since 1987, and the 1994 edition of the ISO 9000 Standard Series has been adopted in the United States as the ANSI/ASQC Q 9000 Standard Series. The ISO 9000 standards, which were initially intended to be advisory in nature and which were developed primarily for use in two- party contractual situations or for internal auditing, are currently being applied under a much broader range of conditions and circumstances.

The reader should also note that some national and regional standards bodies are developing supplemental guidance for the application of the ISO 9000 series to specific industries. For example, CEN and CENELEC are developing more specific requirements for the application of the ISO 9001 to the medical device industry; and the U.S. Food and Drug Administration (FDA) is revising its Good Manufacturing Practice (GMP) (25) regulations for medical devices to follow ISO 9001 with appropriate additional requirements.

THE ISO 14000 STANDARDS

In June 1992, the United Nations' Conference on Environmental Development was held in Brazil. At that Conference, ISO made a commitment to continue the work it had begun the prior year in the area of environmental management. The previous year, ISO had formed the Strategic Advisory Group on the Environment (SAGE) to develop an international consensus of what needed to be done in this area. The work of SAGE ultimately resulted in the formation of TC 207 in 1993 to develop standards in the field of environmental management tools and systems.

The United States has been an active participant in ISO TC 207 since 1993. U.S. participation in TC 207 and its subcommittees is jointly managed by the American Society of Testing and Materials (ASTM) and ASQC under the aegis of ANSI. The work of ISO TC 207 is divided into seven areas: (1) management systems; (2) audits; (3) labeling; (4) environmental performance evaluation; (5) life cycle assessment; (6) terms and definitions, and (7) environmental aspects in product standards.

Work is ongoing in all of these areas. In the management system area, ISO 14001 and 14004 were released in the fall of 1996, and have been the subject of much review and analysis in the United States. Some U.S. companies have already begun to seek conformance with the standards, with a smaller number seeking third party registration of that conformance. While it is still unclear as to the extent that these standards will ultimately gain acceptance with U.S. industry, there appears to be growing acceptance of these standards in Europe and elsewhere.

MANAGEMENT SYSTEM ASSESSORS

1. First party assessment or self-assessment is the process by which a manufacturer or supplier declares that the quality or environmental management system conforms to the requirements of ISO 9001, 9002 or 9003; ISO 14001; or some comparable standard. Such self-audits are usually major components of the system itself. Such self-audits can increase the confidence of management in its system and demonstrate to its personnel that the firm is committed to quality or environmental management. This process is also known as a manufacturer's declaration of conformance.

2. Second Party Assessment is also common in the United States. In this case, it is usually the buyer who requires and assesses the quality or environmental management systems of the suppliers from which the buyer wishes to purchase products. The nature of these programs varies greatly depending on the type of product being purchased and the needs and concerns of the buyer. These assessments are generally only available to and mandatory for those companies wishing to become suppliers to that buyer.

3. Third party Assessment (better known as management system registration), is a form of assessment in which the supplier's claim of conformity is validated by a technically competent body. "Third party" quality or environmental management system evaluations and registrations may be voluntary or mandatory and are conducted by persons or organizations independent of both the supplier and the buyer.

4. A new type of assessment has also been proposed within the United States and internationally -- the Supplier's Audit Confirmation (SAC) route -- which involved elements of both first party and third party evaluations. The SAC route would include a supplier's declaration of conformity to the appropriate ISO 9000 or ISO 14000 standard combined with a third party assessment of the maturity of the manufacturer's quality or environmental management system and the effectiveness of the internal audit program. The details of how this new type of assessment would operate are still under development.

MANAGEMENT SYSTEM REGISTRATION

Management system registration or approval (sometimes misnamed "management system certification" (26)) involves the assessment and periodic audit of the adequacy of a supplier's management system by a third party, known as a registrar. When a supplier's system conforms to the registrar's interpretation of an ISO 9000 or 14000 standard, the registrar issues the supplier a "certificate of registration."

The reader should note that it is the supplier's system that is registered, not an individual product or a company's compliance with an established level of environmental performance. Consequently, management system registration does not imply product conformity to any given set of requirements or compliance with any established level of environmental performance. Management system registration is not a replacement for an assessment of the product to determine its quality or suitability nor is it a replacement for assessment of the organization's compliance with established environmental requirements.

For example, when a product certification program is conducted in conjunction with a registration program, the buyer is provided with information on the characteristics of the product and on the likelihood that the product is being produced consistently. When an environmental management system assessment is conducted in conjunction with a compliance audit, information is provided not only about the organization's compliance with legal environmental requirements at the time of the compliance audit but also about the organization's ability to sustain or improve upon that compliance level over time.

Currently, approximately fifty U.S. based organizations conduct quality system registration. In addition, according to Quality System Update (QSU) (27), as of December 1996, 11,738 company sites had obtained registration; and the number of registered U.S. companies continues to grow at a rapid pace.

Some of the impetus for U.S. companies to seek quality system registration results from European Union (EU) requirements. Having an approved quality system will not be a blanket requirement for all products. Nevertheless, for suppliers of some products, having an approved quality system will be a key component of the EU's legal requirements. For most of these regulated products, however, ISO 9000 registration is but one alternative for proving compliance to the essential requirements contained in the so-called "new approach" directives, not an absolute requirement.

However, most of the demand U.S firms are experiencing for ISO 9000 registration in Europe and elsewhere seems to be coming less from regulatory bodies than from the marketplace. As the importance of ISO 9000 registration becomes recognized and required by foreign and domestic buyers and as registration is seen and used by manufacturers as a competitive marketing tool, the demand for ISO 9000 compliance is expected to increase in non-regulated areas.

The degree of interest and pressure felt by U.S. manufacturers to seek registration currently varies significantly by industry. In many of the "high tech" or "high safety and health risk" areas where product reliability is crucial, the market pressure on U.S. manufacturers to seek registration is likely to be considerable. In addition, international companies are becoming increasingly compliant with, and requiring their suppliers to become compliant with, the ISO 9000 standards and/or to have their quality systems registered.

For example, the so-called "Big Three" U.S. automakers -- Ford, Chrysler, and General Motors (GM) -- have adopted a uniform set of quality system requirements (referred to as the Chrysler, Ford and General Motors Quality System Requirements QS-9000 Standard), which incorporates ISO 9001 in its entirety. Previously each company had its own expectations for supplier quality systems and corresponding assessment documents. In addition to the Big Three, the QS- 9000 standard has been adopted by Mack Trucks, Volvo GM Heavy Truck, and others.

The QS-9000 will be applied to all internal and external suppliers of production and service parts and materials. GM and Chrysler have already indicated that they will require third party registration by their primary suppliers and have provided them with a time table for completing that registration. Ford is still considering the issue of third party registration and has indicated that -- while it will accept third party registration -- it will not require it at this time.

Another ISO 9000 based standard, TE-9000, has also been developed by the "Big Three" and applies to suppliers of both capital equipment and perishable tools for the following processes: assembly; balancing; casting; deburring; forging; forming; gaging; heat treating; machining; material handling; measuring; molding; packaging; painting; plating; robotics; stamping; tooling; washing; and welding. Registration is not currently required, but that may change.

Other large U.S. firms also require or encourage suppliers to become compliant with the ISO 9000 standards or their equivalent, such as the aerospace industry's ARD-9000 standards.

In the environmental management system area, five U.S. registrars were approved in early 1997 by the U.S. accreditation system to conduct ISO 14001 registrations. Additional registrars are involved the approval process. This is still a new program in the United States, but it is expected to grow in coming years. While some companies have or are beginning to seek conformance to these standards, and even third party registration, it is still too early to speculate on the extent to which U.S. industry and U.S. government agencies will endorse the use of these standards or the effect that international marketplace considerations will have on the ISO 14000 standards acceptance and use. U.S. Federal Agencies Activities Related to ISO 9000

While currently there are no mandatory requirements in the United States for quality system approval or registration, some federal agencies are considering how they might use them within their regulatory programs. For example, in October 1996, FDA issued a final rule to revise its medical device good manufacturing practice (GMP) regulations to ensure compatibility with quality system specifications contained in ISO 9001. FDA has also begun to look at how the ISO 9000 Standard Series can make a useful contribution in the other product areas it regulates. Other federal agencies are also studying this issue.

Federal agencies are also looking at the usefulness of the ISO 9000 standards in the procurement area. On June 29, 1994, Secretary of Defense William Perry issued a memo which mandated greater use of performance and commercial specifications and standards and encouraged greater partnership between DoD and industry associations in the development of non-government standards. It directed the reduction of government oversight by substituting process controls and non-government standards in place of development and/or production testing and inspection, and military unique quality assurance standards. This memo reinforced the earlier DoD memo dated February 14, 1994 in which DOD issued a policy statement designed to remove barriers within DOD to the use of the ISO 9000.

DoD has taken a leadership role along with National Aeronautics and Space Administration (NASA) in the Government Industry Quality Liaison Panel (GIQLP), which is exploring the use of a government-wide single set of basic quality system requirements based on the twenty elements of ISO 9001, the use of "advanced quality concepts" in government procurement, and mutual recognition of agency audits of the basic quality system by other agencies. They have been joined in this effort by other federal agencies, including the General Services Administration (GSA), The Federal Aviation Administration (FAA), the U.S. Coast Guard, the National Maritime Administration (MARAD), the National Oceanic and Atmospheric Administration (NOAA), and the National Institute of Standards and Technology (NIST).

U.S. Federal Agencies Activities Related to ISO 14000

Because the issuance of ISO 14001 and 14004 is so recent, no federal endorsement of these standards had been issued as of early 1997. However, many federal agencies have begun to review the potential usefulness of these standards within their regulatory and programs activities. The Interagency Committee on Standards Policy (ICSP), chaired by NIST, has established an ISO 14000/EMS Workgroup to provide information and recommendations to the ICSP regarding development and implementation of the ISO 14000 standards in the federal sector. The working group has been: sharing information; identifying current uses of ISO 14000 by agencies; as well as identifying agency capabilities and initiatives and ways to link agency efforts. In addition, the Environmental Protection Agency (EPA) has begun using management system standards in several pilot programs designed to collect data on such issues as how having an EMS affects or does not affect compliance with regulatory requirements and overall environmental performance. Such data is expected to be used by EPA in formulating its policy on the use of such standards.

ACCREDITATION OF U.S. MANAGEMENT SYSTEM REGISTRARS

In 1989, the Registrar Accreditation Board (RAB) was established as an affiliate of the American Society of Quality Control (ASQC) to develop a program to evaluate the quality of services offered by registrars. RAB issued its first approval in March 1991, and several more firms have been approved since then. In December 1991, RAB and ANSI agreed to form a joint U.S. quality system registration program called the American National Accreditation Program for registrars of Quality Systems (ANAPRQS). This system was recently been expanded to include the accreditation of environmental management system registrars and is now referred to as the National Accreditation Program (NAP).

In February 1992, RAB established an ISO 9000 auditor certification program. RAB recently expanded this program to include the certification of EMS auditors. In 1992, RAB also established requirements for the accreditation of auditor

training course providers. The course provider accreditation program is currently operated under the ANSI/RAB NAP and was expanded in 1996 to include the approval of EMS auditor training course providers.

At least two European accreditation bodies also accredit registrars outside their geographical borders. The Dutch Council for Accreditation (RvA), in particular, has accredited a number of registrars in the United States. RAB is currently working towards an agreement with RvA regarding mutual recognition of each other's accreditations.

In addition, the Standards Council of Canada (SCC) has established an accreditation program for registrars, which is open to U.S. registrars. Programs similar to those of the RAB, RvA, and SCC have been established in a number of other European countries and elsewhere in the world, though many are currently not open to U.S. registrars. However, some U.S. registrars with parent bodies in Europe have had their parent bodies accredited by (28) their national accreditation bodies. For example, TUV America's parent body is accredited to conduct quality system registrations by German Accreditation Board (DAR) in Germany.

ACCREDITATION PROGRAM RECOGNITION

The recognition of laboratory, certifier, and registrar accreditation programs within the United States is a relatively new field. Congress mandated the use of such a program in conjunction with fastener testing under the Fastener Quality Act of 1990. Section 6 of the Act requires the Secretary of Commerce, acting through the Director of NIST, to issue regulations under which private laboratory accreditation entities may apply to NIST for approval to accredit laboratories in accordance with the requirements of the Act, which are to be consistent to the extent practical with requirements of national or international consensus documents.

NIST also offers to recognize accreditation programs under its National Voluntary Conformity Assessment System Evaluation Program (NVCASE). This program will enable the Department of Commerce, acting through NIST, to evaluate and recognize accreditation bodies (as well as other conformity assessment bodies as appropriate) which meet established requirements. The program is designed to provide the EU and other countries with any assurances they might require regarding the competence of U.S. based testing, certification and quality system registration bodies to conduct conformity assessment activities mandated by the EU or other country under one or more MRAs. The final rule on the NVCASE program was published in the Federal Register on April 22, 1994, and a number of applications are now pending.

Both the Fastener Quality Act recognition program and the NVCASE program will conduct assessments against general requirements established in appropriate international standards and guides (See Appendix I). The programs will also use specific technical requirements that are established for fastener testing or within the scope of the applicable MRA.

While other studies have also called for increasing the use of government recognition programs (29), it is still too early to determine the extent to which recognition programs will be used as part of the U.S. conformity assessment system.

INTERNATIONAL/REGIONAL COOPERATION IN CONFORMITY ASSESSMENT

A number of mechanisms are used to facilitate the acceptance of conformity assessment data in more than one country. In some cases, a conformity assessment body in one country will buy or establish a subsidiary within the geographical boundaries of a foreign country. Conformity assessment data can then be exchanged and accepted between the parent body and its subsidiary.

In other cases, an agreement can be established between two or more organizations located in different countries to accept each other's conformity assessment data or the agreement may be based on the acceptance of conformity assessment marks or certificates of conformity. An example of the former type of mutual recognition arrangement is the IEC's System for Conformity Testing to Standards for Safety of Electrical Products (the IECEE scheme), which is designed to promote the reciprocal recognition of test results among the participating countries and to simplify the certification of electrical products at the national level. An example of the latter is ISO's/IEC's Quality System Assessment Recognition Program (QSAR) which is currently being developed in cooperation with and will be operated by the International Accreditation Forum (IAF) (30).

That program is designed to encourage mutual recognition of ISO 9000 registration certificates by participating national accreditation bodies based on audits of each other's operations. The IAF peer reviews are expected to begin in March 1997 with the assessment of the ANSI-RAB NAP Program.

In most mutual recognition schemes, such as the IECEE Scheme, parties to the agreement agree to accept each other's test results or reports rather than each other's certification marks. Test results or reports prepared in one participating country are accepted in other participating countries for the purpose of obtaining certification or government marketing approval by the importing country.

Mutual acceptance of data may also result from the establishment of bilateral or multilateral agreements between governmental authorities or between a governmental authority and a private sector body (31). These agreements require each party to accept conformity assessment results from entities which are recognized or accredited by the other party or parties. However, these agreements are usually very difficult to reach for many reasons -- especially legal restrictions imposed on the extent to which agencies can transfer their authority for approving products to foreign governments or other entities. For that reason, most mutual recognition agreements between governments involve the mutual acceptance of test data, not certifications or approvals; each government retains the authority for final approval under such agreements.

However, one example of a governmental multilateral agreement for the acceptance of certifications is the European Union's (EU) regional mutual recognition scheme for regulated products. Under the so-called "new approach to technical harmonization of standards," each EU country provides the EU Commission with a list of certifiers (as well as other conformity assessment bodies) which the country considers to be competent to perform certain conformity assessment tasks listed under specified directives. These certifiers (as well as other conformity assessment bodies) are referred to as "notified bodies." Notified bodies can declare (certify) that a regulated product conforms to the "essential requirements" spelled out in the applicable directive. Acceptable methods for conformity assessment, including certification, are listed in each directive. The certifications and subsequent product approval marking of any notified body must be accepted by all other EU countries unless there is cause to believe that the product was improperly certified. Each EU country is responsible for ensuring that the certification bodies it notifies comply with the criteria for competence spelled out in the EN 45000 series.

Another example of such an agreement is the Agreement on the International Carriage of Perishable Foodstuffs and on the Special Equipment to be Used for Such Carriage (ATP). The ATP is an international agreement which sets standards for the testing and uses of equipment which carry perishable foodstuffs. USDA certifies test stations and laboratories in accordance with requirements of the ATP. These bodies can then issue U.S. ATP certificates which are recognized by other ATP signatories (countries).

An example of an agreement between a governmental entity and a private sector body is NIST's National Voluntary Laboratory Accreditation Program (NVLAP) bilateral agreement with the Standards Council of Canada (SCC) -- a governmentally recognized, though private sector, body. Under this agreement, laboratories accredited by one party are recognized by the other party to the agreement as being competent within the scope of accreditation.

While such efforts have done much to reduce the need for multiple assessments of products sold internationally, there can be many problems with mutual recognition agreements, including lack of clarity or the inclusion of poorly defined terms and responsibilities in the written agreement. The reader should also be cognizant of who signed the agreement when reviewing information on mutual recognition agreements (MRAs) between conformity assessment programs.

An acceptance body is an organization responsible for accepting and using the conformity assessment data produced by a conformity assessment body. This body may be the same as or different from the conformity assessment body -- the body that administers the conformity assessment program. As previously cited, the accreditation body for a laboratory which tests radiation dosimeters might be NIST's National Voluntary Laboratory Accreditation Program (NVLAP), while the acceptance body -- the body which uses the data to regulate dosimeters -- is the Nuclear Regulatory Commission (NRC). This is an important distinction that readers should be aware of when reviewing information on mutual recognition arrangements between conformity assessment programs. If acceptance bodies are not parties to the agreement, they are not obligated and may not be willing to accept data generated from all signatories to the agreement. The value of the agreement is then questionable.

This problem may be overcome if each conformity assessment body is willing to extend the use of its own conformity assessment mark or certificate to a conformity assessment body that has been accredited by the other party(ies). For example, as noted above, if laboratories (registrars or certification bodies) accredited by Accreditation Body A are also provided with an accreditation certificate from Accreditation Body B and are able to use Body B's accreditation mark, and vice versa, this problem can be overcome.

However, under this arrangement, each party may be assuming responsibility for (and the liability and risks associated with) the other party's work. Another alternative is to allow parties to label a conforming product with a mark common to all parties to the agreement. An example of this type of scheme is the IEC's Quality Assessment System for Electronic Components (the IECQ scheme). This scheme is designed to facilitate trade in electronic components by requiring participating countries to recognize without discrimination all IECQ approvals of manufacturers, distributors and test laboratories and the IECQ certification of components.

Yet another alternative is to have all acceptance bodies also sign the agreement. This can be difficult to accomplish if regulatory agencies are involved and issues of statutory authority required to enter into such agreements are raised. Such problems can make the establishment of an effective MRA a very complex undertaking.

Nonetheless, mutual recognition schemes are vital to trade because they can help to: (1) ensure an objective assessment of a product; (2) increase the efficiency of international or regional economic cooperation through the removal of technical barriers to trade; (3) quicken the circulation of products entering international or regional markets; (4) eliminate the need for recertification and thus reduce the costs incurred; and (5) ensure safety, health and environmental protection.

It is likely that ISO Guides 65, 61, and 62 (which cover the operation of certification and quality system registration programs and the accreditation schemes for those activities) together with ISO/IEC Guides 25 and 58 (32) (which cover the operation of testing and calibration laboratories and laboratory accreditation schemes) will help facilitate the process of establishing mutual recognition agreements by serving as the basis for many if not most of such mutual recognition schemes in the future.

SUMMARY

The U.S. system for conformity assessment is complex, multifaceted and comprised of many diverse activities including: testing, certification, management system registration, accreditation and accreditation program recognition. While each of these activities is a distinct operation, they are closely interrelated. The inclusion or absence of any of these activities, as well as the quality with which any one of them is performed, can have a significant effect on the confidence and reliance that can be placed on the results of the entire conformity assessment process. In addition, standards, which underlie each of these activities, can also have a major impact on the outcome of a specific conformity assessment activity as well as a cumulative effect on the entire process.

Standards and conformity assessment activities together impact almost every aspect of life in the United States. Conformity assessment is an important marketplace communications device -- a means of exchanging information between buyer and seller. It is therefore important for buyers, sellers, and other interested parties to understand the conformity assessment process to competently judge the value of a particular assessment scheme and to use the information resulting from that scheme to make intelligent choices. The quality of the information conveyed depends on: the impartiality and competence of the assessment body; the types of assessment activities included in the scheme; and the adequacy and appropriateness of the standards against which the product is evaluated.

Conformity assessment may result in widespread buyer deception if the performance characteristics or test methods contained in a standard are insufficient to ensure adequate product performance or if the buyer is misinformed as to the competence of the conformity assessment body or the extent to which the product characteristics have been evaluated. If properly conducted, however, conformity assessment can furnish valuable information to the marketplace and can serve as the basis for increased or diminished opportunities for national and international trade.

APPENDIX I

LIST OF ISO/IEC CONFORMITY ASSESSMENT DOCUMENTS (as of March 1997)

ISO/IEC GUIDE 2: 1996

STANDARDIZATION AND RELATED ACTIVITIES -- GENERAL VOCABULARY

ISO/IEC GUIDE 7: 1994

GUIDELINES FOR DRAFTING STANDARDS FOR CONFORMITY ASSESSMENT

ISO/IEC GUIDE 22: 1996

GENERAL CRITERIA FOR SUPPLIER'S CONFORMITY

ISO/IEC GUIDE 23: 1982

METHODS OF INDICATING CONFORMITY WITH FOR THIRD-PARTY CERTIFICATION SYSTEMS (under revision)

ISO/IEC GUIDE 25: 1990

GENERAL REQUIREMENTS FOR THE COMPETENCE CALIBRATION AND TESTING LABORATORIES (under revision)

ISO GUIDE 27: 1983

GUIDELINES FOR CORRECTIVE ACTION TO BE TAKEN BY A CERTIFICATION BODY IN THE EVENT OF MISUSE OF ITS MARK OF CONFORMITY (under revision)

ISO/IEC GUIDE 28: 1982

GENERAL RULES FOR A MODEL THIRD-PARTY CERTIFICATION SYSTEM FOR PRODUCTS (under revision)

ISO/IEC GUIDE 39: 1988

GENERAL REQUIREMENTS FOR THE ACCEPTANCE INSPECTION BODIES (under revision)

ISO/IEC GUIDE 42: 1984

GUIDELINES FOR A STEP-BY-STEP APPROACH TO AN INTERNATIONAL CERTIFICATION SYSTEM

ISO/IEC GUIDE 43: 1984

DEVELOPMENT AND OPERATION OF LABORATORY PROFICIENCY TESTING (under revision)

ISO/IEC GUIDE 44: 1985

GENERAL RULES FOR ISO OR IEC INTERNATIONAL THIRD-PARTY CERTIFICATION SCHEMES FOR PRODUCTS

ISO/IEC GUIDE 53: 1988

AN APPROACH TO THE UTILIZATION OF A QUALITY SYSTEM IN THIRD PARTY PRODUCT CERTIFICATION (under revision)

ISO/IEC GUIDE 56: 1989

AN APPROACH TO THE REVIEW BY A CERTIFICATION BODY OF ITS OWN INTERNAL QUALITY SYSTEM

ISO/IEC GUIDE 57: 1991

GUIDELINES FOR THE PRESENTATION OF INSPECTION RESULTS (under revision)

ISO/IEC GUIDE 58: 1993

CALIBRATION AND TESTING LABORATORY SYSTEMS - GENERAL REQUIREMENTS FOR OPERATION AND RECOGNITION

ISO/IEC GUIDE 60: 1994

ISO/IEC CODE OF GOOD PRACTICE FOR CONFORMITY ASSESSMENT

ISO/IEC GUIDE 61: 1996

GENERAL REQUIREMENTS FOR ASSESSMENT AND ACCREDITATION OF CERTIFICATION/REGISTRATION BODIES

ISO/IEC GUIDE 62: 1996

GENERAL REQUIREMENTS FOR BODIES OPERATING ASSESSMENT AND CERTIFICATION/REGISTRATION OF QUALITY SYSTEMS

ISO/IEC GUIDE 65: 1996

GENERAL REQUIREMENTS FOR BODIES OPERATING PRODUCT CERTIFICATION SYSTEMS

ISO/IEC TR 13233: 1995

(TECHNICAL REPORT) INFORMATION TECHNOLOGY -- INTERPRETATION OF ACCREDITATION REQUIREMENTS IN ISO/IEC ACCREDITATION OF INFORMATION TECHNOLOGY AND TELECOMMUNICATION TESTING LABORATORIES FOR SOFTWARE AND PROTOCOL TESTING SERVICES

ISO/IEC DIS 50043-1

(DRAFT INTERNATIONAL STANDARD) PROFICIENCY TESTING BY INTERLABORATORY COMPARISONS -- PART 1: DEVELOPMENT AND OPERATION OF PROFICIENCY TESTING SCHEMES

ISO/IEC DIS 50043-2

(DRAFT INTERNATIONAL STANDARDS) PROFICIENCY TESTING BY INTERLABORATORY COMPARISONS -- PART 2: SELECTION AND USE OF PROFICIENCY TESTING SCHEMES BY LABORATORY ACCREDITATION BODIES

LIST OF ISO QUALITY MANAGEMENT AND QUALITY ASSURANCE DOCUMENTS (as of March 1997)

ISO GUIDE 34: 1996

QUALITY SYSTEM GUIDELINES FOR THE PRODUCTION OF REFERENCE MATERIALS

ISO 8402: 1994

QUALITY MANAGEMENT AND QUALITY ASSURANCE -- VOCABULARY

ISO 9000-1: 1994

QUALITY MANAGEMENT AND QUALITY ASSURANCE STANDARDS -- PART 1: GUIDELINES FOR SELECTION AND USE

ISO 9000-2: 1993

QUALITY MANAGEMENT AND QUALITY ASSURANCE STANDARDS -- PART 2: GENERIC GUIDELINES FOR THE APPLICATION OF ISO 9001, ISO 9002 AND ISO 9003

ISO/FDIS 9000-2

(FINAL DRAFT INTERNATIONAL STANDARD) QUALITY MANAGEMENT AND QUALITY ASSURANCE STANDARDS -- PART 2: GENERIC GUIDELINES FOR THE APPLICATION OF ISO 9001, ISO 9002 AND ISO 9003 (REVISION OF ISO 9000-2: 1993)

ISO 9000-3: 1991

QUALITY MANAGEMENT AND QUALITY ASSURANCE STANDARDS -- PART 3: GUIDELINES FOR THE APPLICATION OF ISO 9001 TO THE DEVELOPMENT, SUPPLY AND MAINTENANCE OF SOFTWARE

ISO/DIS 9000-3: 1991

(DRAFT INTERNATIONAL STANDARD) QUALITY MANAGEMENT AND QUALITY ASSURANCE STANDARDS -- PART 3: GUIDELINES FOR THE APPLICATION OF ISO 9001 TO THE DEVELOPMENT, SUPPLY AND MAINTENANCE OF SOFTWARE (REVISION OF ISO 9000-3: 1991)

ISO 9000-4: 1993

QUALITY MANAGEMENT AND QUALITY ASSURANCE STANDARDS -- PART 4: GUIDE TO DEPENDABILITY PROGRAMME MANAGEMENT ISO 9001: 1994 - QUALITY SYSTEMS -- MODEL FOR QUALITY ASSURANCE IN DESIGN, DEVELOPMENT, PRODUCTION, INSTALLATION AND SERVICING

ISO 9002: 1994

QUALITY SYSTEMS -- MODEL FOR QUALITY ASSURANCE IN PRODUCTION, INSTALLATION AND SERVICING

ISO 9003: 1994

QUALITY SYSTEMS -- MODEL FOR QUALITY ASSURANCE IN FINAL INSPECTION AND TEST

ISO 9004-1: 1994

QUALITY MANAGEMENT AND QUALITY SYSTEM ELEMENTS -- PART 1: GUIDELINES

ISO 9004-2: 1991

QUALITY MANAGEMENT AND QUALITY SYSTEM ELEMENTS -- PART 2: GUIDELINES FOR SERVICES

ISO 9004-3: 1993

QUALITY MANAGEMENT AND QUALITY SYSTEM ELEMENTS -- PART 3: GUIDELINES FOR PROCESSED MATERIALS

ISO 9004-4: 1993

QUALITY MANAGEMENT AND QUALITY SYSTEM ELEMENTS -- PART 4: GUIDELINES FOR QUALITY IMPROVEMENT

ISO 10005: 1995

QUALITY MANAGEMENT -- GUIDELINES FOR QUALITY PLANS

ISO/FDIS 10006

(FINAL DRAFT INTERNATIONAL STANDARD) QUALITY MANAGEMENT -- GUIDELINES TO QUALITY IN PROJECT MANAGEMENT

ISO 10007: 1995

QUALITY MANAGEMENT -- GUIDELINES FOR CONFIGURATION MANAGEMENT

ISO 10011-1: 1990

GUIDELINES FOR AUDITING QUALITY SYSTEMS -- PART 1: AUDITING

ISO 10011-2: 1991

GUIDELINES FOR AUDITING QUALITY SYSTEMS -- PART 2: QUALIFICATION CRITERIA FOR QUALITY SYSTEMS AUDITORS

ISO 10011-3: 1991

GUIDELINES FOR AUDITING QUALITY SYSTEMS -- PART 3: MANAGEMENT OF AUDIT PROGRAMMES

ISO 10012-1: 1992

QUALITY ASSURANCE REQUIREMENTS FOR MEASURING EQUIPMENT -- PART 1: METROLOGICAL CONFIRMATION SYSTEM FOR MEASURING EQUIPMENT

ISO/FDIS 10012-2

(FINAL DRAFT INTERNATIONAL STANDARD) QUALITY ASSURANCE REQUIREMENTS FOR MEASURING EQUIPMENT -- PART 2: CONTROL OF MEASUREMENT PROCESSES

ISO 10013: 1995

GUIDELINES FOR DEVELOPING QUALITY MANUALS

ISO/DIS 10014

(DRAFT INTERNATIONAL STANDARD) GUIDELINES FOR MANAGING THE ECONOMICS OF QUALITY

ISO/DTR 13352

(DRAFT TECHNICAL REPORT) GUIDELINES FOR INTERPRETATION OF ISO 9000 SERIES FOR APPLICATION WITHIN THE IRON ORE INDUSTRY

ISO 13485: 1996

QUALITY SYSTEMS -- MEDICAL DEVICES -- PARTICULAR REQUIREMENTS FOR THE APPLICATION OF ISO 9001

ISO 13488: 1996

QUALITY SYSTEMS -- MEDICAL DEVICES -- PARTICULAR REQUIREMENTS FOR THE APPLICATION OF ISO 9002

LIST OF ISO ENVIRONMENTAL SYSTEM MANAGEMENT DOCUMENTS (as of March 1997)

ISO GUIDE 64:1997

GUIDE FOR THE INCLUSION OF ENVIRONMENTAL ASPECTS IN PRODUCT STANDARDS

ISO 14001: 1996

ENVIRONMENTAL MANAGEMENT SYSTEMS -- SPECIFICATION WITH GUIDANCE FOR USE

ISO 14004: 1996

ENVIRONMENTAL MANAGEMENT SYSTEMS -- GENERAL GUIDELINES ON PRINCIPLES, SYSTEMS AND SUPPORTING TECHNIQUE

ISO 14010: 1996

GUIDELINES FOR ENVIRONMENTAL AUDITING -- GENERAL PRINCIPLES

ISO 14011: 1996

GUIDELINES FOR ENVIRONMENTAL AUDITING -- AUDIT PROCEDURES -- AUDITING OF ENVIRONMENTAL MANAGEMENT SYSTEMS

ISO 14012: 1996

GUIDELINES FOR ENVIRONMENTAL AUDITING -- QUALIFICATION CRITERIA FOR ENVIRONMENTAL AUDITORS

ISO/DIS 14021

(DRAFT INTERNATIONAL STANDARD) ENVIRONMENTAL LABELS AND DECLARATIONS -- SELF-DECLARATION ENVIRONMENTAL CLAIMS -- GUIDELINES AND DEFINITION AND USAGE OF TERMS

ISO/FDIS 14040

(FINAL DRAFT INTERNATIONAL STANDARD) ENVIRONMENTAL MANAGEMENT - LIFE CYCLE ASSESSMENT - PRINCIPLES AND FRAMEWORK

ISO/DIS 14050

(DRAFT INTERNATIONAL STANDARD) ENVIRONMENTAL MANAGEMENT - VOCABULARY

APPENDIX II - LIST OF ACRONYMS

The following is a list of initials/acronyms that may be of interest/use in reviewing literature in the conformity assessment field.

A

A2LA

AMERICAN ASSOCIATION FOR LABORATORY ACCREDITATION

AABB

AMERICAN ASSOCIATION OF BLOOD BANKS

AARS

ALL-AMERICA ROSE SELECTIONS

AAS

ALL-AMERICA SELECTIONS

ABBS

ANTIGUA AND BARBUDA BUREAU OF STANDARDS

ABNT

ASSOCIACAO BRASILEIRA DE NORMAS TECNICAS (BRAZILIAN ASSOCIATION FOR TECHNICAL STANDARDS)

ABS

AMERICAN BUREAU OF SHIPPING

ACCSQ

ASEAN CONSULTATIVE COMMITTEE FOR STANDARDIZATION AND QUALITY

ADA

AMERICAN DENTAL ASSOCIATION

AENOR

ASOCIACION ESPANOLA DE NORMALIZACION Y CERTIFICACION (SPANISH ASSOCIATION FOR STANDARDIZATION AND CERTIFICATION)

AFAQ

ASSOCIATION FRANCAISE POUR L'ASSURANCE DE LA QUALITE (FRENCH ASSOCIATION FOR QUALITY ASSURANCE)

AFNOR

ASSOCIATION FRANCAISE DE NORMALISATION (FRENCH STDS. ASS'N.)

AGA

AMERICAN GAS ASSOCIATION

AHAM

ASSOCIATION OF HOME APPLIANCE MANUFACTURERS

AMA

AMERICAN MEDICAL ASSOCIATION

ANSI

AMERICAN NATIONAL STANDARDS INSTITUTE

ANSI/ASQC Q 9000 SERIES

U.S. EQUIVALENT OF THE ISO 9000 SERIES

APEC

ASIA-PACIFIC ECONOMIC COOPERATION

APLAC

ASIA-PACIFIC LABORATORY ACCREDITATION COOPERATION/APEC AGREEMENT SIGNED BY THE U.S. AND 15 OTHER SIGNATORIES TO COOPERATE IN THE DEVELOPMENT OF INFRASTRUCTURES FOR CALIBRATION, TESTING, AND QUALITY SYSTEMS AND ACCEPTANCE OF LABORATORY ACCREDITATION

APMP

ASIA-PACIFIC METROLOGY PROGRAM

AQAP

ALLIED QUALITY ASSURANCE PUBLICATION, NATO

ARI

AIR-CONDITIONING AND REFRIGERATION INSTITUTE

ASEAN

ASSOCIATION OF SOUTHEAST ASIAN NATIONAL-FOUNDED IN 1967 AND COMPOSED OF BRUNEI, INDONESIA, MALAYSIA, PHILIPPINES, SINGAPORE, THAILAND & VIETNAM

ASME

AMERICAN SOCIETY OF MECHANICAL ENGINEERS

ASMO

ARAB ORGANIZATION FOR STANDARDIZATION AND METROLOGY

ASQC

AMERICAN SOCIETY FOR QUALITY CONTROL

ASTM

AMERICAN SOCIETY FOR TESTING AND MATERIALS

ATP

AGREEMENT ON THE INTERNATIONAL CARRIAGE OF PERISHABLE FOODSTUFFS AND ON THE SPECIAL EQUIPMENT

B

BBN

BUREAU BURUNDAIS DE NORMALISATION ET CONTROLE DE LA QUALITE (BURUNDI BUREAU OF STANDARDIZATION AND QUALITY CONTROL)

BCIQ

BUREAU OF COMMODITY INSPECTION AND QUARANTINE (TAIWAN)

BDS

COMMITTEE FOR STANDARDIZATION AND METROLOGY AT THE COUNCIL OF MINISTERS (BULGARIA)

BELST

BELARUS COMMITTEE FOR STANDARDIZATION METROLOGY AND CERTIFICATION

BENELUX

ECONOMIC UNION COMPRISED OF BELGIUM, THE NETHERLANDS, AND LUXEMBOURG

BIPM

INTERNATIONAL BUREAU OF WEIGHTS AND MEASURES (CONVENTION DU METRE 1875)

BIS

BUREAU OF INDIAN STANDARDS

BNSI

BARBADOS NATIONAL STANDARDS INSTITUTION

BOCA

BUILDING OFFICIALS AND CODE ADMINISTRATORS INTERNATIONAL

BPS

BUREAU OF PRODUCT STANDARDS (PHILIPPINES)

BS 5750

BRITISH EQUIVALENT OF THE ISO 9000 SERIES

BSI

BRITISH STANDARDS INSTITUTION

BSTI

BANGLADESH STANDARDS AND TESTING INSTITUTION

C

CAC

CODEX ALIMENTARIUS COMMISSION

CANACINTRA

CAMARA NACIONAL DE TRANSFORMATION, MEXICO

CAP

COLLEGE OF AMERICAN PATHOLOGISTS

CASCO

ISO COUNCIL COMMITTEE ON CONFORMITY ASSESSMENT

CB FCS SCHEME

CB FULL CERTIFICATION SCHEME, A PROPOSED EXTENSION OF THE CURRENT CB SCHEME

CB SCHEME

ALSO KNOWN AS THE IECEE SCHEME FOR RECOGNITION OF THE RESULTS OF TESTING TO STANDARDS OF ELECTRICAL EQUIPMENT

CCIR

INTERNATIONAL RADIO CONSULTATIVE COMMITTEE, ITU

CCITT

INTERNATIONAL TELEGRAPH AND TELEPHONE CONSULTATIVE COMMITTEE, ITU

CD

COMMITTEE DRAFT

CE MARK

EUROPEAN UNION MARK INDICATING CONFORMANCE TO THE ESSENTIAL REQUIREMENTS CONTAINED IN AN EU DIRECTIVE.

CEN

EUROPEAN COMMITTEE FOR STANDARDIZATION

CENELEC

EUROPEAN COMMITTEE FOR ELECTROTECHNICAL STANDARDIZATION

CEPT

CONFERENCE EUROPEENE DES ADMINISTRATIONS DES POSTES ET DES TELECOMMUNICATION

CGSB

CANADIAN GENERAL STANDARDS BOARD

CISPR

INTERNATIONAL SPECIAL COMMITTEE ON RADIO INTERFERENCE

CNACR

CHINA NATIONAL ACCREDITATION COMMITTEE FOR QUALITY SYSTEM REGISTRATION BODIES

COFRAC

COMITE FRANCAIS D'ACCREDITATION (FRENCH COMMITTEE FOR ACCREDITATION)

COPANT

PAN AMERICAN STANDARDS COMMISSION

COSMT

CZECH OFFICE FOR STANDARDS, METROLOGY AND TESTING

COVENIN

COMISION VENEZOLANA DE NORMAS INDUSTRIALES (VENEZUELAN INDUSTRIAL STANDARDS COMMISSION)

CSA

CANADIAN STANDARDS ASSOCIATION

CSBTS

CHINA STATE BUREAU OF TECHNICAL SUPERVISION

CSK

COMMITTEE FOR STANDARDIZATION OF THE DEMOCRATIC PEOPLE'S REPUBLIC OF KOREA

CYS

CYPRUS ORGANIZATION FOR STANDARDS AND CONTROL OF QUALITY

D

DANAK

DANISH AGENCY FOR DEVELOPMENT OF TRADE AND INDUSTRY (ACCREDITATION BODY)

DAR

DEUTSCHER AKKREDITIERUNGS RAT (THE GERMAN ACCREDITATION BOARD)

DESC

DEFENSE ELECTRONICS SUPPLY CENTER, DOD

DFARS

DOD FEDERAL ACQUISITION REGULATION SUPPLEMENT

DGN

DIRECCION GENERAL DE NORMAS (MEXICAN STANDARDS/ACCREDITATION BODY)

DHHS

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES

DIN

DEUTSCHES INSTITUT FUR NORMUNG (GERMAN STANDARDS INSTITUTE)

DIS

DRAFT INTERNATIONAL STANDARD

DITI

U.K.'S DEPARTMENT OF TRADE AND INDUSTRY

DLA

DEFENSE LOGISTICS AGENCY, U.S. DEPT. OF DEFENSE

DOC

U.S. DEPARTMENT OF COMMERCE

DOD

DEPARTMENT OF DEFENSE

DOE

U.S. DEPARTMENT OF ENERGY

DOT

U.S. DEPARTMENT OF TRANSPORTATION

DS

DANSK STANDARD (DANISH STANDARDS ORGANIZATION)

DSC

DREJTORIA E STANDARDIZIMIT DHE CILESISE KESHILLI I MINISTRAVE (ALBANIAN STANDARDS BODY)

DSN

DEWAN STANDARDISASI NASIONAL (INDONESIAN STANDARDS BODY)

DSTU

STATE COMMITTEE OF UKRAINE FOR STANDARDIZATION, METROLOGY AND CERTIFICATION

DTR

DRAFT TECHNICAL REPORT

DZNM

CROATIAN STATE OFFICE FOR STANDARDIZATION AND METROLOGY

E

EAC

EUROPEAN ACCREDITATION OF CERTIFICATION, A MEMORANDUM OF UNDERSTANDING SIGNED BY EUROPEAN NATIONAL ACCREDITATION BODIES AT UTRECHT ON MAY 22, 1991 TO ESTABLISH A MULTILATERAL AGREEMENT BETWEEN MEMBERS BASED ON MUTUAL CONFIDENCE. COUNTRIES REPRESENTED IN EAC INCLUDE: DENMARK, FINLAND, FRANCE, GERMANY, ITALY, NETHERLANDS, NORWAY, SWEDEN, SWITZERLAND, AND THE UK.

EAL

EUROPEAN COOPERATION FOR ACCREDITATION OF LABORATORIES (FORMED BY THE MERGER OF WECC AND WELAC)

EC

EUROPEAN COMMUNITY, NOW THE EUROPEAN UNION (EU)

ECE

UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE

EDA

ECONOMIC DEVELOPMENT ADMINISTRATION, DOC

EEA

EUROPEAN ECONOMIC AREA (A TREATY DESIGNED TO ESTABLISH A NINETEEN (EC/EFTA) NATION FREE TRADE AREA)

EFTA

EUROPEAN FREE TRADE ASSOCIATION (AUSTRIA, FINLAND, ICELAND, LIECHTENSTEIN, NORWAY, SWEDEN AND SWITZERLAND)

ELOT

PREDECESSOR OF EOS

EN

EUROPEAN NORM OR STANDARD

EN 29000 SERIES

EUROPEAN EQUIVALENT OF THE ISO 9000 SERIES

ENV

EUROPEAN PRE-STANDARDS

EOQ

EUROPEAN ORGANIZATION FOR QUALITY

EOS

EGYPTIAN ORGANIZATION FOR STANDARDIZATION AND QUALITY CONTROL

EOTA

EUROPEAN ORGANIZATION FOR TECHNICAL APPROVALS

EOTC

EUROPEAN ORGANIZATION FOR TESTING AND CERTIFICATION CREATED BY THE EU IN APRIL 1990 UNDER A MEMORANDUM OF UNDERSTANDING WITH CEN AND CENELEC TO PROMOTE THE MUTUAL RECOGNITION OF CONFORMITY ASSESSMENTS THROUGHOUT THE EU AND EFTA COUNTRIES

E-Q-NET

EUROPEAN NETWORK FOR QUALITY SYSTEM ASSESSMENT AND CERTIFICATION, A BUSINESS AGREEMENT ESTABLISHED IN EARLY 1990 AMONG EUROPEAN NONPROFIT REGISTRARS UNDER WHICH MEMBERS AGREE TO RECOGNIZE EACH OTHER AS BEING COMPETENT BODIES IN CONFORMITY WITH EN 45012 AND THE RECOMMENDATIONS OF THE EAC

EQS

EUROPEAN COMMITTEE FOR QUALITY SYSTEMS ASSESSMENT ESTABLISHED IN 1989 WITH MEMBERSHIP FROM BOTH EU AND EFTA COUNTRIES TO HARMONIZE RULES/PROCEDURES USED FOR QUALITY SYSTEM ASSESSMENT AND REGISTRATION AMONG MEMBERS

ESA

ETHIOPIAN AUTHORITY FOR STANDARDIZATION

ETA

EUROPEAN TECHNICAL APPROVAL (APPROVAL BY AN EU AUTHORIZED BODY WHICH APPLIES TO CONSTRUCTION PRODUCTS FOR WHICH THERE ARE NO EXISTING OR PLANNED STANDARDS)

ETSI

EUROPEAN TELECOMMUNICATIONS STANDARDS INSTITUTE

EU

EUROPEAN UNION, FORMERLY THE EUROPEAN COMMUNITY (EC)

EVS

NATIONAL STANDARDS BOARD OF ETHIOPIA

F & G

FAA

FEDERAL AVIATION ADMINISTRATION, DOT

FAO/WHO

FOOD AND AGRICULTURAL ORGANIZATION/WORLD HEALTH ORGANIZATION CODEX ALIMENTARIUS COMMISSION

FAR

FEDERAL ACQUISITION REGULATION

FDA

FOOD AND DRUG ADMINISTRATION, DHHS

FDIS

FINAL DRAFT INTERNATIONAL STANDARD

FINAS

CENTRE FOR METROLOGY AND ACCREDITATION (FINLAND)

FM

FACTORY MUTUAL RESEARCH CORP.

FTAA

FREE TRADE AREA OF THE AMERICAS, A PROPOSAL TO EXPAND NAFTA TO INCLUDE OTHER COUNTRIES IN THE AMERICAS

GDBS

GRENADA BUREAU OF STANDARDS

GMP

GOOD MANUFACTURING PRACTICE GUIDELINES (FDA)

GOST R

GOSSTANDART OF RUSSIA (COMMITTEE OF RUSSIAN FEDERATION FOR STANDARDIZATION, METROLOGY AND CERTIFICATION)

GSA

GENERAL SERVICES ADMINISTRATION

H, I

HD

HARMONIZED DOCUMENT

HKQAA

HONG KONG QUALITY ASSURANCE AGENCY (ACCREDITATION BODY)

HUD

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

IAA

BUREAU OF ACCREDITATION AND INTERNATIONAL COOPERATION OF THE INDUSTRIAL ADVANCEMENT ADMINISTRATION (KOREAN ACCREDITATION BODY)

IAF

INTERNATIONAL ACCREDITATION FORUM, AN INTERNATIONAL AGREEMENT SIGNED JANUARY 28, 1993 AMONG NATIONAL ACCREDITATION BODIES TO DISCUSS ACHIEVING EQUIVALENCE OF ACCREDITATION PROGRAMS/SYSTEMS AND MUTUAL RECOGNITION OF EACH OTHER'S ACCREDITATION SYSTEMS IN FURTHERANCE OF INTERNATIONAL TRADE. MEMBERSHIP INCLUDES: 21 ACCREDITATION BODIES AND SIX ASSOCIATE MEMBERS

IAPMO

INTERNATIONAL ASSOCIATION OF PLUMBING AND MECHANICAL OFFICIALS

IATCA

INTERNATIONAL AUDITOR AND TRAINING CERTIFICATION ASSOCIATION, COMPOSED OF 21 MEMBER BODIES, WHICH AIMS INCLUDE WORLDWIDE ACCEPTANCE OF AUDITOR CERTIFICATIONS AND AUDITOR TRAINING COURSE ACCREDITATIONS

IBN

INSTITUT BELGE DE NORMALISATION (BELGIAN STANDARDS BODY)

ICAO

INTERNATIONAL CIVIL AVIATION ORGANIZATION

ICBO

INTERNATIONAL CONFERENCE OF BUILDING OFFICIALS

<>dt>ICLAB

IRISH CERTIFICATION AND LABORATORY ACCREDITATION BOARD

ICONTEC

INSTITUTO COLOMBIANO DE NORMAS TECNICAS (CHILE)

ICSP

INTERAGENCY COMMITTEE ON STANDARDS POLICY (CHAIRED BY NIST)

IEC

INTERNATIONAL ELECTROTECHNICAL COMMISSION

IECEE

IEC SYSTEM FOR RECOGNITION OF RESULTS OF TESTING TO STANDARDS OF ELECTRICAL EQUIPMENT

IECEx

IEC SCHEME FOR ELECTRICAL EQUIPMENT INTENDED FOR USE IN EXPLOSIVE ATMOSPHERES DESIGNED TO PROMOTE WORLDWIDE ACCEPTANCE OF A SINGLE STANDARD, A SINGLE CERTIFICATE, AND A SINGLE MARK

IECQ

IEC QUALITY ASSESSMENT SYSTEM FOR ELECTRONIC COMPONENTS

IIOC

INDEPENDENT INTERNATIONAL ORGANIZATION FOR CERTIFICATION, COMPOSED OF 7 CERTIFICATION BODIES WHICH HAVE SIGNED AN MOU TO WORK TOWARDS ELIMINATION OF THE NEED FOR MULTIPLE ASSESSMENTS AND RAISE THE REPUTATION OF ISO 9000

ILAC

INTERNATIONAL LABORATORY ACCREDITATION CONFERENCE

INAPI

INSTITUT ALGERIEN DE NORMALIZATION ET DE PROPRIETE INDUSTRIELLE (ALGERIAN STANDARDS BODY)

INCONTEC

INSTITUTO COLOMBIANO DE NORMAS TECNICAS (COLOMBIAN INSTITUTE FOR TECHNICAL STANDARDS)

INDECOPI

INSTITUTO NACIONAL DE DEFENSA DE LA COMPETENCIA Y DE LA PROTECCION DE LA PROPIEDAD INTELECTUAL (PERUVIAN STANDARDS BODY)

INMETRO

INSTITUTO NACIONAL DER METROLOGIA, NORMALIZACAO E QUALIDADE INDUSTRIAL (BRAZILIAN NATIONAL INSTITUTE FOR METROLOGY, STANDARDIZATION AND INDUSTRIAL QUALITY)

INN

INSTITUTO NACIONAL DE NORMALIZACION (CHILEAN NATIONAL INSTITUTE FOR STANDARDIZATION)

INNORPI

INSTITUT NATIONAL DE LA NORMALISATION ET DE LA PROPRIETE INDUSTRIELLE (TUNISIAN NATIONAL INSTITUTE FOR STANDARDIZATION AND INDUSTRIAL PROPERTY)

IMO

INTERNATIONAL MARITIME ORGANIZATION

INSTA

INTERNORDIC STANDARD CERTIFICATION SYSTEM

IPQ

INSTITUTO PORTUGUES DA QUALIDADE (PORTUGUESE INSTITUTE FOR QUALITY - STANDARDS/ACCREDITATION BODY)

IQA

INSTITUTE FOR QUALITY ASSURANCE

IRAM

INSTITUTO ARGENTINO DE RACIONALIZACION DE MATERIALES (ARGENTINEAN INSTITUTE FOR STANDARDIZATION OF MATERIALS)

IRS

INSTITUTUL ROMAN DE STANDARDIZARE (ROMANIAN STANDARDS/ACCREDITATION BODY)

ISAC

ICELANDIC BOARD FOR TECHNICAL ACCREDITATION

ISO

INTERNATIONAL ORGANIZATION FOR STANDARDIZATION

ISIRI

INSTITUTE OF STANDARDS AND INDUSTRIAL RESEARCH OF IRAN

ISSUAE

DIRECTORATE OF STANDARDIZATION AND METROLOGY (UNITED ARAB EMIRATES)

ITA

INTERNATIONAL TRADE ADMINISTRATION, DOC

ITC

INTERNATIONAL TRADE CENTRE, UNCTAD-GATT

ITQS

RECOGNITION ARRANGEMENT FOR ASSESSMENT AND CERTIFICATION OF QUALITY SYSTEMS IN THE INFORMATION TECHNOLOGY SECTOR

ITU

INTERNATIONAL TELECOMMUNICATIONS UNION, UNITED NATIONS

J, K

JAB

JAPANESE ACCREDITATION BOARD FOR QUALITY SYSTEM REGISTRATION

JAS-ANZ

JOINT ACCREDITATION SYSTEM OF AUSTRALIA AND NEW ZEALAND

JBS

JAMAICAN BUREAU OF STANDARDS

JDS

DIRECTORATE OF STANDARDS AND MEASURES (JORDAN)

JISC

JAPANESE INDUSTRIAL STANDARDS COMMITTEE

KAN

DEWAN STANDARDISASI NASIONAL KOMITE AKREDITASI NASIONAL (INDONESIAN STANDARDS/ACCREDITATION BODY)

KBS

BUREAU OF STANDARDS/INDUSTRIAL ADVANCEMENT ADMINISTRATION (REPUBLIC OF KOREA)

KEBS

KENYA BUREAU OF STANDARDS

L, M

LNCSM

LIBYAN NATIONAL CENTER FOR STANDARDIZATION AND METROLOGY INDUSTRIAL RESEARCH

LST

LITHUANIAN STANDARDIZATION OFFICE

MBS-MALAWI

MALAWI BUREAU OF STANDARDS

MBS-MALTA

MALTA BUREAU OF STANDARDS

MEA

METALLURGICAL ENGINEERS OF ATLANTA

MISM

MONGOLIAN NATIONAL INSTITUTE FOR STANDARDIZATION AND METROLOGY

MOU

MEMORANDUM OF UNDERSTANDING

MRA

MUTUAL RECOGNITION AGREEMENT

MSB

MAURITIUS STANDARDS BUREAU

MSHA

MINE SAFETY AND HEALTH ADMINISTRATION, DEPARTMENT OF LABOR

MSIT

MAJOR STATE INSPECTION OF TURKMENISTAN

MSZH

MAGYAR SZABVANYUGYI HIVATAL (HUNGARY STANDARDS BODY)

MTC

MANUFACTURING TECHNOLOGY CENTERS

N

NA

NORWAY ACCREDITATION

NAC-QS

COMITE NATIONAL POUR L'ACCREDITATION DES ORGANISMES DE CERTIFICATION, BELGIUM ORGANIZATION RESPONSIBLE FOR THE ACCREDITATION OF QUALITY SYSTEM REGISTRARS

NACCB

NATIONAL ACCREDITATION COUNCIL FOR CERTIFICATION BODIES, NOW UKAS

NAC-QS

COMITE NATIONAL POUR L'ACCREDITATION DES ORGANISMES DE CERTIFICATION (BELGIUM ACCREDITATION BODY)

NAFTA

NORTH AMERICAN FREE TRADE AGREEMENT

NATA

ASSOCIATION OF TESTING AUTHORITIES IN AUSTRALIA

NATO

NORTH ATLANTIC TREATY ORGANIZATION

NATSF

NORTH AMERICAN TRILATERAL STANDARDIZATION FORUM, A TRILATERAL EFFORT SPONSORED BY ANSI, CAMARA NACIONAL DE TRANSFORMATION (CANACINTRA) (MEXICO), AND THE STANDARDS COUNCIL OF CANADA (SCC), TO DISCUSS STANDARDS AND CONFORMITY ASSESSMENT ACTIVITIES AND ISSUES AMONG THE THREE COUNTRIES AND TO PROMOTE THEIR HARMONIZATION AS APPROPRIATE

NBBI

NATIONAL BOARD OF BOILER AND PRESSURE VESSEL INSPECTORS

NC

COMITE ESTATAL DE NORMALIZACION (CUBA)

NCB

NATIONAL CERTIFICATION BODY

NCSCI

NATIONAL CENTER FOR STANDARDS AND CERTIFICATION INFORMATION, NIST

NCTL

NATIONAL CERTIFIED TESTING LABORATORIES

NCWM

NATIONAL CONFERENCE ON WEIGHTS AND MEASURES (ADMINISTERED BY NIST)

NELAC

NATIONAL ENVIRONMENTAL LABORATORY ACCREDITATION CONFERENCE (SPONSORED BY THE U.S. ENVIRONMENTAL PROTECTION AGENCY-EPA)

NIST

NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY, DOC

NNI

NEDERLANDS NORMALISATIE-INSTITUUT (NETHERLANDS STANDARDS BODY)

NOAA

NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION, DOC

NORAMET

NORTH AMERICAN METROLOGY PROGRAM

NRC

NUCLEAR REGULATORY COMMISSION

NRTL

NATIONALLY RECOGNIZED TESTING LABORATORIES PROGRAM, OSHA

NSAI

NATIONAL STANDARDS AUTHORITY OF IRELAND

NSF

NORGES STANDARDISERINGSFORBUND (NORWAY STANDARDS BODY)

NVCASE

NATIONAL VOLUNTARY CONFORMITY ASSESSMENT SYSTEMS EVALUATION PROGRAM

NVLAP

NATIONAL VOLUNTARY LABORATORY ACCREDITATION PROGRAM, NIST

O, P & Q

OECD

ORGANIZATION FOR ECONOMIC COOPERATION AND DEVELOPMENT

OIML

INTERNATIONAL ORGANIZATION FOR LEGAL METROLOGY

ON

OSTERRREICHISCHES NORMUNGSINSTITUT (AUSTRIAN STANDARDS BODY)

OSHA

OCCUPATIONAL SAFETY AND HEALTH ADMINISTRATION, DOL

PAC

PACIFIC ACCREDITATION COOPERATION

PASC

PACIFIC AREA STANDARDS CONGRESS

PKN

POLISH COMMITTEE FOR STANDARDIZATION

PNGS

NATIONAL STANDARDS COUNCIL (PAPUA NEW GUINEA)

PSI

PAKISTAN STANDARDS INSTITUTION

QML

DOD QUALIFIED MANUFACTURING LISTS PROGRAM

QPL

DOD QUALIFIED PRODUCTS LISTING PROGRAM

QSAR

QUALITY SYSTEM ASSESSMENT RECOGNITION (QSAR) SCHEME/ISO

QSR

QUALITY SYSTEM REGISTRAR

R & S

RAB

REGISTRAR ACCREDITATION BOARD, U.S. ACCREDITATION BODY FOR QUALITY SYSTEM REGISTRARS/CERTIFIER OF QUALITY SYSTEM AUDITORS, A SUBSIDIARY OF ASQC

RELE

SPANISH ACCREDITATION BODY

RILEM

INTERNATIONAL UNION OF TESTING AND RESEARCH LABORATORIES FOR MATERIALS AND STRUCTURES

RvA

DUTCH COUNCIL FOR ACCREDITATION

RvC

PREDECESSOR OF RvA

SAA

STANDARDS AUSTRALIA

SABS

SOUTH AFRICAN BUREAU OF STANDARDS

SAE

SOCIETY OF AUTOMOTIVE ENGINEERS

SAS

SWISS ACCREDITATION SERVICE

SASMO

SYRIAN ARAB ORGANIZATION FOR STANDARDIZATION AND METROLOGY

SASO

SAUDI ARABIAN STANDARDS ORGANIZATION

SAZ

STANDARDS ASSOCIATION OF ZIMBABWE

SBCCI

SOUTHERN BUILDING CODE CONGRESS INTERNATIONAL

SC

SUBCOMMITTEE

SCC

STANDARDS COUNCIL OF CANADA

SENORCA

PART OF COVENIN RESPONSIBLE FOR ACCREDITATION (VENEZUELA)

SFS

FINISH STANDARDS ASSOCIATION

SISIR

SINGAPORE INSTITUTE OF STANDARDS AND INDUSTRIAL RESEARCH (STANDARDS/CONFORMITY ASSESSMENT BODY)

SII

STANDARDS INSTITUTION OF ISRAEL

SINCERT

SISTEMA NAZIONALE PER L'ACCREDIAMENTO DEGLI ORGANISMI DE CERTIFICAZIONE (ITALIAN ACCREDITATION BODY)

SIRIM

STANDARDS AND INDUSTRIAL RESEARCH INSTITUTE OF MALAYSIA

SIS

STANDARDISERINGS-KOMMISSIONEN I SVERIGE (SWEDISH STANDARDS BODY)

SISIR

SINGAPORE INSTITUTE OF STANDARDS AND INDUSTRIAL RESEARCH

SLSI

SRI LANKA STANDARDS INSTITUTION

SMIS

STANDARDS AND METROLOGY INSTITUTE OF REPUBLIC OF SLOVENIA (STANDARDS/ACCREDITATION BODY)

SNIMA

SERVICE DE NORMALISATION INDUSTRIELLE MAROCAINE (MOROCCAN STANDARDS BODY)

SNV

SWISS ASSOCIATION FOR STANDARDIZATION

SNZ

STANDARDS NEW ZEALAND

SRCC

SOLAR RATING AND CERTIFICATION CORPORATION

STRI

ICELANDIC COUNCIL FOR STANDARDIZATION

SWEDAC

SWEDISH ACCREDITATION BODY

SZS

SAVEZNI ZAVOD ZA STANDARDIZACIJU (DEPARTMENT FOR QUALITY AND CERTIFICATION - YUGOSLAVIA)

T

TAAC

TRADE ADJUSTMENT ASSISTANCE CENTERS

TBS

TANZANIA BUREAU OF STANDARDS

TBT AGREEMENT

1994 AGREEMENT OF TECHNICAL BARRIERS TO TRADE OF THE INTERNATIONAL GENERAL AGREEMENT ON TARIFFS AND TRADE (GATT) OF THE WTO

TC

TECHNICAL COMMITTEE

TC 176

THE ISO TECHNICAL COMMITTEE RESPONSIBLE FOR THE DEVELOPMENT OF THE ISO 9000 AND 10000 SERIES

TCVN

GENERAL DEPARTMENT FOR STANDARDIZATION, METROLOGY AND QUALITY (VIET NAM)

TELARC

TESTING LABORATORY REGISTRATION COUNCIL IN NEW ZEALAND

TGA

TRAEGERGEMEINSCHAFT FUER AKKREDITIERUNG GmbH (GERMAN ACCREDITATION BODY)

TickIT

U.K. QUALITY SYSTEM REGISTRATION SCHEME FOR SOFTWARE COMPANIES STANDARDS

TISI

THAI INDUSTRIAL STANDARDS INSTITUTE

TR

TECHNICAL REPORT

TSE

TURK STANDARDLARI ENSTITUSU (TURKISH STANDARDS INSTITUTE)

TTBS

TRINIDAD AND TOBAGO BUREAU OF STANDARDS

U & V

UEAtc

EUROPEAN UNION OF AGREMENT

UILI

UNION INTERNATIONALE DES LABORATOIRES INDEPENDANTS

UKAS

UNITED KINGDOM ACCREDITATION SERVICE, FORMERLY THE NACCB

UL

UNDERWRITERS LABORATORIES, INC.

UN

UNITED NATIONS

UNI

ENTE NAZIONALE ITALIANO DI UNIFICAZIONE (ITALIAN STANDARDS BODY)

UNIT

INSTITU URUGUAYO DE NORMAS TECNICAS (URUGUAYAN INSTITUTE FOR TECHNICAL STANDARDS)

UNMS

SLOVAK OFFICE OF STANDARDS, METROLOGY AND TESTING (STANDARDS/ACCREDITATION BODY)

USDA

U.S. DEPARTMENT OF AGRICULTURE

UZGOST

UZBEK STATE CENTRE FOR STANDARDIZATION, METROLOGY AND CERTIFICATION (UZBEKISTAN)

VA

DEPARTMENT OF VETERANS AFFAIRS

W

WD

WORKING DRAFT

WHO

WORLD HEALTH ORGANIZATION

WTO

WORLD TRADE ORGANIZATION

APPENDIX III

Information available from:

Office of Standards Services
National Institute of Standards and Technology
100 Bureau Drive, MS 2100
Gaithersburg, Maryland 20899-2100
http://ts.nist.gov/oss

National Technical Information Service (NTIS)
5285 Port Royal Road
Springfield, Virginia 22161, USA
Telephone: (703) 487-4650
Orders Only: (800) 553-6847
Fax: (703) 321-8547

Global Engineering Documents
15 Inverness Way East
Englewood, CO 80112-5704
Telephone: (800) 854-7l79, (303) 397-7956
Fax: (303) 397-2740

ENDNOTES

(1) ISO is the acronym for the International Organization for Standardization, while IEC stands for the International Electrotechnical Commission. The International Organization for Standardization (ISO) is a worldwide federation of over 90 national standards bodies. ISO covers standardization in all fields, except the electrical and electronics fields which are covered by the International Electrotechnical Commission (IEC). IEC has members from over 40 countries which represent some 80% of the world's population. Together ISO and IEC form the world's largest nongovernmental system for voluntary industrial and technical collaboration in the field of standardization.

(2) Under 16 CFR Part 13, misrepresentations of product quality are regarded by the Federal Trade Commission as prohibited trade practices. Conformity assessment programs provide some assurance that claims made regarding a product's conformance to a particular standard are in fact valid.

(3) The Agreement covers conformity assessment procedures for products, processes and services including: "procedures for sampling, testing and inspection; evaluation, verification and assurance of conformity; registration, accreditation and approval as well as their combinations."

(4) This definition is from the National Policy on Standards for the United States and Recommended Implementation Plan, National Standards Policy Advisory Committee, Washington, DC, December, 1978, p. 6. There is also a definition of "standard" in ISO/IEC Guide 2, but that definition is somewhat more complicated. The WTO Agreement defines standard as a "document approved by a recognized body, that provides, for common and repeated use, rules, guidelines or characteristics for products or related process and production methods, with which compliance is not mandatory. It may also include or deal exclusively with terminology, symbols, packaging, marking, or labelling requirements as they apply to a product, process or production method." It is less encompassing than the ISO definition of standard. It is important to know what definition is being used to clearly understand the implications of any conformity assessment discussions.

(5) For further information on the history of standards, see the American Standards Association's, "Through History with Standards," in Rowen Glie, (ed.), Speaking of Standards, Cahner Books, Boston, MA, 1972.

(6) A list of these organizations is contained in the 1996 edition of NIST SP 806, "Standards Activities of Organizations in the United States."

(7) Dr. David Hemenway, NBS/GCR 80-287, "Performance vs Design Standards," NBS, Gaithersburg, MD, October 1980.

(8) Note that the usefulness of a performance requirement depends on the existence of an accurate and reliable method of assessing the conformity of a product or service to that requirement.

(9) The TBT Agreement requires that: "(w)herever appropriate, members shall specify technical regulations based on product requirements in terms of performance rather than design or descriptive characteristics

(10) Reference materials are defined by ISO Guide 30 as a "material or substance one or more properties of which are sufficiently well established to be used for the calibration of an apparatus, the assessment of a measurement method, or for assigning values to materials."

(11) "Traceability means the ability to relate individual measurement results to national standards or nationally accepted measurement systems through an unbroken chain of comparisons." Examples of traceability include: equipment calibrated by NIST; equipment calibrated using NIST transfer standards (materials previously measured by NIST); or equipment calibrated using other NIST calibrated equipment. For further information on traceability, see Brian Belanger's "Traceability: an Evolving Concept," published in ASTM Standardization News, October 1980, pp. 22-28. Also see Ernest Garner and Stanley D. Rasberry's "What's New in Traceability," published in ASTM's Journal of Testing and Evaluation, Nov. 1993.

(12) Reference standards include those involving dimension and mass (i.e., length, weight, diameter, angle, or volume and density); mechanical properties (i.e. flow rate or airspeed); chemical/physical properties; time and frequency; etc.

(13) Information on each agency's legal authority is contained in NIST SP 808 - Directory of Federal Government Laboratory Accreditation/Designation Programs.

(14) Some organizations use other terms to refer to the process, such as product listing, product evaluation, product regulation, product approval, or the publication of research reports, but in this discussion, we will use the term "certification." The reader should be aware of the existence and use of other terms, however, to describe this activity.

(15) G. J. Varoufakis, Materials Testing in Classical Greece, Technical Specifications of the 4th Century BC, Hellenic Organization for Standardization, Athens, Greece, 1983.

(16) ANSI serves as the U.S. standards coordinator and the U.S. member body of ISO, IEC, and several other international private sector standards organizations.

(17) CASCO is acronym for the ISO Council Committee on Conformity Assessment. This committee works to promote mutual recognition and acceptance of national and regional conformity assessment systems, and the appropriate use of international standards for testing, inspection, certification, and quality system registration.

(18) The use of the term "listed" is sometimes preferred by some certifiers to the term "certified" for products which have been evaluated and shown to be in conformance with applicable standards.

(19) See NIST SP 903, Directory of U.S. Private Sector Product Certification Programs, for more information.

(20) See NIST SP 739 "Directory of Federal Government Certification Programs" for additional information on this topic.

(21) For more information on this topic, see NIST SP 903, "Directory of U.S. Private Sector Product Certification Programs," pp. A-3 to A-5. See also the U.S. Patent and Trademark Office web site at http://www.uspto.gov.

(22) W. Edwards Deming, Out of Crises, Massachusetts Institute of Technology, Cambridge, MA 1986.

(23) OSHA is responsible for the regulation of all electrical products used in the work place. For a list of the products under OSHA's jurisdiction which require certification by a Nationally Recognized Testing Laboratory (NRTL), see 29 CFR 1910.

(24) Note this definition is somewhat different from the ISO definitions. ISO Standard 9000-1987 defines quality system as: "the organization, structure, responsibilities, procedures, processes and resources for implementing quality management." The standard defines quality management as : "that aspect of the overall management function that determines and implements quality policy." The standard defines quality policy as: "the overall intentions and directions of an organization as regards quality, as formally expressed by top management." These ISO definitions also include several additional footnotes.

(25) CEN and CENELEC have issued a draft European standard, EN 46001 - Specific Requirements for the Application of EN 29001 to Medical Devices. Medical device manufacturers doing business in the EU will have to comply with the quality system requirements of EN 46001.

(26) "Quality system registration" and "quality system certification" are frequently used interchangeably; however, confusion arises when "quality system" is not placed in front of the term "certification." If these two words are missing, it is easy to confuse the assessment and approval of a manufacturer's quality system with product certification -- two entirely different activities! For that reason, the preferred term in the United States is "quality system registration."

(27) Quality System Update (QSU) and other related documents are published by Irwin Professional Publishing, a division of Richard D. Irwin, Inc. and a subsidiary of the Times Mirror Company.

(28) ISO has published a "Directory of Quality System Registration Bodies" which includes information on national accreditation bodies. Copies of this directory are available from ANSI. See Appendix D for information on contacting ANSI.

(29) For other examples of how recognition might be used within the United States, see the National Research Council report, Standards, Conformity Assessment, and Trade: Into the 21st Century, National Academy of Sciences, 1995.

(30) The International Accreditation Forum (IAF) is an organization of national and regional groupings of accreditation bodies which aims to achieve and maintain confidence in the accreditation programs operated by accreditation body members through the: exchange of information; participation in joint activities; harmonization of operating procedures; participation in regional grouping which maintain regional multilateral recognition agreements; and participation in evaluation programs based on peer review of accreditation body members leading to a worldwide multilateral recognition agreement. IAF will begin peer evaluations of accreditation bodies beginning in March 1997. The first peer review conducted will be of the ANSI-RAB NAP Program, which is expected to be followed by peer reviews of the accreditation programs of Australia-New Zealand, Canada, China, Japan, and Malaysia, and the European Accreditation of Certification (EAC) regional mutual recognition program. Among other responsibilities, members agree to operate accreditation programs in conformance with relevant ISO/IEC Guides and publicly available IAF application guide

(31) In such cases, the private sector body is usually recognized in some way by its national government as the authoritative body in that area.

(32) See Appendix I for a list of the titles of these guides.