Gas Metrology Group: CRM Document

APPENDIX A

Statistical Analysis of Data

Introduction

The data submitted to NIST by the producer and the auditor will be examined in detail to determine that the candidate lot of CRM's is of the concentration claimed and that the lot is stable and homogenous. The statistical treatment is illustrated in this appendix by means of an example based, for the most part, on real experimental data. However, it was necessary to synthe-size some data, particularly in describing the auditor results, to produce an example illustrative of the whole process. It should be noted that this example does not define the degree of measurement precision required of a CRM, and, in many cases, the precision shown may not necessarily be attainable, due to such factors as instrument sensitivity and the chemical and physical properties of the gases involved.

Illustrative Example and Statistical Analysis

Producer's Calibration Data:

The producer provided the following data on the calibration of his instrument:

Table 1

SRM No.	Concentration*	Signal or Sensitivity
1677	9.67 ± .09 ppm	11.51 mV (signal)
1678	44.9 ± .5 ppm	54.10 mV (signal)
1679	97.1 ± .9 ppm	114.86 mV (signal)

*The ± values are the total uncertainties given in the SRM certificates.

A linear regression of signal vs. concentration provides the equation:

signal = .464 + 1.181 (concentration)

The fit to the straight line is good, and is only minimally affected by making the intercept zero. This leads to the modified equation:

signal = 1.187 (concentration)

The fit provided by this equation is shown in the following table where the fitted values in the last column should be compared with the observed values of signal:

Table 2

Signal
Observed	Fitted
11.51	11.48
54.10	53.30
114.86	115.26

We conclude that the instrument gives essentially a linear response and is in good state of calibration. However, since the samples to be analyzed have a concentration close to that of SRM 1678 (about 45 ppm), a sensitivity value can be adopted that is based on a calibration line going through the origin and through the point whose abscissa is the certified value of SRM 1678 and whose ordinate is the measured value for this SRM. This gives the calibration equation which will be used for all further analyses:

signal = 1.205 (concentration)

Analysis of Internal Reference Standard

One of the cylinders was randomly selected as the lot "Internal Reference Standard." The measurements,* in mV are shown in the second column of Table 3. The third column shows the concentration values, in ppm, obtained by using the sensitivity 1.205 mV/ppm for the conversion.

The average concentration for the Internal Reference Standard is:

$X=\; 44.6139\; ppm$

with a standard deviation among single replicate measurements of

s_x = 0.018 ppm

The standard error of the average value for this cylinder is:

$s$_x = 0.018/SQRT(10) = 0.0057 ppm

Table 3

Measurement	Signal (mV)	Concentration (ppm)	Measurement No.	Signal (mV)	Concentration (ppm)
1	53.7623	44.616	6	53.7683	44.621
2	53.7382	44.596	7	53.7454	44.602
3	53.7454	44.602	8	53.7237	44.584
4	53.7948	44.643	9	53.7707	44.623
5	53.7719	44.627	10	53.7767	44.628

*Note: It is not required that either producer or auditor provide the actual measured value of the signal. However, the value of the concentration calculated from the signal should be expressed with sufficient digits to reflect the magnitude of the signal. In other words, don't round off the calculated concentration when submitting the data for evaluation.

Analysis of All Samples

The Internal Reference Standard is now used by the producer to analyze all the cylinders in the lot. This is generally accomplished by measuring the ratio of the signal for the sample to that for the internal standard. Table 4 shows the signal ratios, denoted as R, and the corresponding calculated con-centrations, denoted as C, using the value 44.6139 ppm for the internal standard. Thus, each concentration is obtained by the equation:

C = R (44.6139)

where

R = signal for sample/signal for internal standard

The last column of Table 4 is used for two purposes: a) to obtain an additional estimate of the standard deviation among replicates, and b) to test whether a significant systematic shift has occurred between the two sets ("first analysis" and "second analysis"). The estimate of the standard deviation, converted to a single measurement basis is 0.021 and is consistent with that obtained previously (s = 0.018) [see section 2, preceding]. As to a possible shift, there is no evidence for such an occurrence. A test of significance can be carried out as follows:

$t\; =\; -0.0048/(0.030/SQRT(10))\; =\; -0.50$

This value is not significant, when compared with the critical value of Student's t, for 10-1 = 9 degrees of freedom.

The standard deviations, 0.021 and 0.022, for the two sets of values in Table 4 are mutually consistent. Moreover, since they are of the same order of magnitude as the measurement error (as derived from replicate measurements on the same sample), it may be concluded that no measurable heterogeneity exists between the cylinders of this lot.

13

Table 4

					Calculated Concentration Difference Between Duplicates
	First Analysis		Second Analysis
Sample	R	C	R	C
5	1.000177	44.622	0.999601	44.596	0.026
6			1.000592	44.592
11			1.000592	44.640
1.000102	44.618	1.000406	44.632	-0.014
15			1.000392	44.631
21			0.999482	44.591
23	1.000904	44.654	0.999807	44.605	0.049
29			1.000438	44.633
30			1.000368	44.630
31	0.999906	44.610	1.000353	44.630	-0.020
33	0.999350	44.585	1.000307	44.628	-0.043
34	0.999520	44.592	1.000050	44.616	-0.024
35			1.000480	44.635
39	0.999563	44.594	1.000210	44.623	-0.029
45	0.999514	44.592	1.000050	44.616	-0.024
46			0.999415	44.588
48			0.999168	44.577
49			0.999800	44.605
50	0.999881	44.609	0.999304	44.583	0.026
51			0.999060	44.572
52	0.999452	44.589	0.999321	44.584	0.005
Average		44.6065		44.6099	-0.0048
Std. dev.		0.021		0.022	0.030*

*Since the numbers in this column are differences of two measurements, the standard deviation found for this column is SQRT(2) times that for single measurements. Consequently, the standard deviation for single measurements derived from the differences is (0.030)/SQRT(2) = 0.021. This values does not include possible variability between samples.

The best average value for the concentration of the lot is:

$((44.6065\; x\; 10)\; +\; (44.6099\; x\; 21))/(10\; +\; 21)\; =\; 44.6088$

The standard error of this overall average is:

$0.022/SQRT(31)\; =\; 0.0040$

This standard error does not include calibration error, or errors in the value of the SRM and in the value of internal reference standard.

Auditor's Calibration Data

Table 5 shows the auditor's calibration results.

SRM No.	Concentration*	Signal (counts)
1677	9.79 ± .09	19,688
1678	45.3 ± .5	91,642
1679	97.1 ± .8	197,990
1680	476. ± .4	979,130

*The ± values are the total uncertainties given in the SRM certificates.

The sensitivities (signal/concentration) are successively:

2011, 2023, 2039, and 2057 counts/ppm.

These values indicate a trend due, either to the presence of a blank or to curvature, or to both.

A regression analysis shows a slight amount of curvature, but otherwise the calibration data appear satisfactory. Since the samples are of the order of magnitude of SRM 1678, the latter will be used for conversion of signal to concentration, through the equation:

signal = 2023 (concentration).

Auditor's Sample Measurements

The auditor made 10 replicate analysis on each of two cylinders, no. 48 and no. 29. The results are shown in Table 6.

Table 6

Cylinder 48		Cylinder 29
Signal	Concentration	Signal	Concentration
89866	44.422	89781	44.380
90044	44.510	90121	44.548
89495	44.239	89874	44.426
89570	44.276	90050	44.270
90024	44.500	89558	44.270
89997	44.487	89874	44.426
89568	44.275	90226	44.600
89708	44.344	89888	44.433
89742	44.361	89987	44.482
89852	44.415	89655	44.318
Average	44.3829		44.4396
Std. dev.	0.099		0.101

Evaluation of Uncertainties and Intercomparison of Results

Internal Comparison of Auditor's Results

The standard deviation of a single measurement made by the auditor is 0.10. The results for cylinders 48 and 29 may be compared by Student's t-test:

$t\; =(44.3829\; -\; 44.4396)/(0.100\; SQRT(1/10\; +\; 1/10))\; =\; -1.27$

The t-value is not significant. There is, therefore, no evidence of heterogeneity between the two cylinders.

Total Uncertainty of Producer's Values

The value of each sample obtained by the producer is obtained by a procedure represented by the following diagram:

                   Calibration    Internal Reference
   SRM                Factor           Standard            Sample
    |__________________| |_______________| |_________________|
        Calibration              A                  B

The total error in the sample value is composed of four parts:

1. the uncertainty in the SRM value
2. the uncertainty in the calibration experiment
3. the uncertainty due to comparison A
4. the uncertainty due to comparison B

We use the rule (derived from the law of propagation of errors) that the square of the relative error of the final value is equal to the sum of the squares of the relative errors of the components.

More specifically, if $c$_i is the final concentration value obtained for a particular sample (denoted by the subscript i) in the lot, we have (from step B):

$c$_i = R_i C_Ref (1)

where R_i is the ratio of signals for sample i to the reference sample, and C_Ref is the concentration attached to the reference sample. But C_Ref is obtained in step A by averaging ten values obtained each as

Signal/k (2)

where k is the calibration value derived from the calibration experiment. In our case, k = 1.205. The average of the ten measurements may be described by

$C$_Ref = S/k (3)

where $S$ is the average of ten replicate signal values. Combining (1) and (3) gives

$c$_i = R_i (S/k) (4)

The value of k is obtained experimentally from a single signal value divided by the concentration of the SRM. Thus we may write:

k = S_o/C_o (5)

where C_o is the concentration attached to the SRM by the SRM certificate and S_o is the signal corresponding to it.

Combining (4) and (5), we obtain finally:

$c$_i = R_i S (C_o/S_o (6)

The law of propagation of errors gives:

$(sc$_i/c_i) = (^sR_i/R_i) + (^sS/S) + (S_sig S_o/S_o) + (^sC_o/C_o)² (7)

The first term of the right side represents step B; the second term, step A; and the third term the uncertainty of the calibration experiment itself. The last term represents the uncertainty of the SRM used for calibration.

We now estimate these four components.

1) ^sR_i/R_i (step B) is obtained from the last column of Table 4:

$sR$_i/R_i = 0.021/44.61 = 4.71 x 10^-4 (8)

2) S_s/S (step A) is obtained from the calculations derived from Table 3:

$S$_s/S = 0.0057/44.62 = 1.28 x 10^-4 (9)

3) ^sS_o/S_o (calibration experiment) has not been measured, but we can assume the same precision as in step A for a single measurement:

$sR$_i/R_i = 0.018/44.62 = 4.03 x 10^-4 (10)

4) ^sC_o/C_o (uncertainty of SRM) is derived from the uncertainty stated in the SRM certificate. This stated uncertainty is equal to two standard deviations. Thus:

$sC$_o/C_o = (1/2) * (0.5/44.9) = 55.68 x 10^-4 (11)

Adding the squares we have:

$(sC$_o/C_o)² = [(4.71)² + (1.28)² + (4.03)² + (55.68)²] x 10^-8 = 3140 x 10^-8

Hence:

$sc$_i/c_i = ±56 x 10^-4

Since all $c$_i values are approximately the same and equal to

$c$_i = 44.62 ppm, we obtain

$sc$_i = 56 x 10^-4 x 44.62 = ±0.25 ppm (12)

We see that the predominant component of uncertainty, in this case, is that of the SRM.

Comparison of Producer's and Auditor's Values

By a calculation similar to that above, we obtain, for the auditor's value for a particular cylinder, say c^*_i:

$(sc*$_i/c^*_i)² = (S_ct/ct)² + (S_ct/ct)² = (^SC^*_o/C^*_o)²

Step A is not present for the auditor's data. The symbol ct represents a count, and an average of ten counts; Co* represents the value given by the certificate for the SRM used in the calculation of the calibration factor. We have:

$(sc*$_i/c^*_i)² = ((0.10/SQRT(10))/44.4)² + (0.10/44.4)² + (0.25/45.3)²

= (7.12x10^-4)² + (22.52x10^-4)² + (55.19x10^-4)²

= 3604x10^-8 = (60x10^-4)²

Since c^*_i = 44.4 ppm for both samples analyzed by the auditor, we have:

$sc*$_i = 60x10-4x44.4 = ±0.27 ppm

We now obtain the following summary results (Table 7):

Table 7*

	Sample 48	Sample 29
Producer	44.58 ± .25	44.63 ± .25
Auditor	44.38 ± .27	44.44 ± .27

*The ± values represent standard errors in this table.

It is apparent that the result obtained by the auditor for each sample is not significantly different from that of the producer for the same sample. Thus, the auditor's values in this case, substantiate those provided by the producer.

In general, it may be assumed that there is no significant difference between the concentration claimed by the producer and that found by the auditor if the following expression is satisfied:

$|c$_i - c^*_i| <= 2 SQRT(^sc²_i + ^sc^2*_i)

goto Index

goto Home Page.