Teaching Students How to Evaluate Data - Analytical Chemistry (ACS

---

A N A L Y T I C A L CHLHISTRY

1140 ‘l’ulde

VI. Aurlyrir of Vuriuiioo fur Datu in l’ul)ln V Froetlom

Hum of Bquaren

Rquare

P

S.D.

I

0 MIB31i 1.50057

0 GOA3b U.16117

4 3 . 41; 11 03

... ...

0.77007 0 27305

O.OSSU(i 0 01370

8.26

0.803

B u t ~ w nconirioundr (over tlinory)

I3ut HRCll anaiysta Inrtwc*tion between and.~&and nonipounila I)rinlic*uti.c ~l,lBllIl -.”.

... ..

..

. .”..

Mean

L)spreei of Ibtll

__

14 $1

20 . 39 .. .

. . . .. .-_ 3.00085 .“

.

.

.

. ..

0 117

-

‘l’lie aiiirlysis of vurisnccr for these d ~ t ( Lis sliowii ia Table VI. Tlittre arc avniltttdc (8, 9, 6, R) nurncrous expositioiis on t.hs nic*t.htisof ciiiiiput~ttior!for the a1idysi8 of variance. The yerieralircd forniulrw for iwdysis of vurirme oornput,ationo will in this estl.nip1i~nifty be not bc given herc. ‘l’Ii(1 suiiis of ~c~utirrs obtaiiied by t.hc follorviiig strps: The sum of s uares for duplicates is equd to one half the sum of the squares o? tile 20 differcmces list.4 in Table IV and the iuni of squares for interaction is equal to the sum of t,he Rquarw of the ten deviations in the last column of Table V. The sum of the squares for compounds hss been computed to test whether the observed differeme in carbon content between the compounds departs significantly from the theoretical difference of 9.29%. The observed value is 9.534 or 0.244% more than theory. The value 0.244 is squared and multiplied by 10, since this is the average result obtained by ten aiialystfs. The s u m of squares for analystv is obtained by adding for each analyst the two percentages shown in Table V. The sum of the squares of the deviations of t,hese ten s u m from t,heir own average gives directly the value 1.36057. Any tcndwcy for an individual analyst to run consistentsly high or low will inflate t,his s u m of squares. The xiuiiierical gt(!ps described above are somewk:.t different from the general formulw described iu ststistical texts which are designed to eliminate computation errors due to rounding off of the entries in the columns of differences. The particular numerical operations performed above apply l o duplicate analyses on each of two compounds. The four entries in the column of nwan squares are of special interest. The entry opposite compounds would approxiiiiatc that for duplicates if the difference fouud experimentally between the carbon contents for the two compounds (after allowing for the theoretical difference) was no greater than would normally arise between two aver e5 each based on 20 analyses mad:+ with a predifference between du licates. The mean cision shown by 6 uare is over 40 times that for duplicates, &le the critical value o?F at the 5% level of probability is 4.35. The results therefore are not in keeping with what would have been expected if the cornpounds had really possessed the theoretical composition. Similarly the F ratio (nl = 9; nz = 20) for analystsis 11.03 whereas the 5 and 1% critical values are 2.40 and 3.45,.mdicating that ’r differences existed between the laboratories than would ave been anticipated if the only sources of variation were those that apply to duplicates conducted in the same laboratory. The

30

Ereat‘

intcwc!tioii tJuhwcoii t l i i r t l y ~ tarid ~ C O I I I ~ M J U i* I ~($6~ ~uivalr?nt tsw, ebtirwtc3 of precision bawd on the uoii&mncy olthe d f i m p u in carbon contctttt fourid in the tun lobaratorhi. There u a

nignificant disagreemc!nt in the two estimata (dup1icat.m and interaction) of precision, aa the coinputod F e x d a . 2.40, the 5% value. Thc precision estimate bluwd on interaatlon IS the lrroper one to use in evaluatin anal ses done in different lahrrrtories. Judged by this stan;fard t i e anslyste have not b w i proved to disagmc ainong tliemmlvetd, Rinw the ratio 0.161 17/ 0.08566 is lestl than.3.18, theS%value (n,= 9; nz 9). The standard devitrtion R i obtained .by taking the squtrre root of the nieRn tqunrc. It wims rcdintic to bast. the estimate of precision on th(h incan square for interaction-that iR, t l c q r w mont anioiig the ten difftmiren in carbon eoritont report4 by the tc?n hfioratories. This give8 a standard deviation CJf 9.2’3% or about 5 parts piir thounand. Thin is largcv thsri the estimate t hat Powtv obtained for the accuracy by throwiner all the rwulta into o~iogroup anti igrioririy ih! tlidiiction between a n a l y w from the mino laboratory and thoso from diffwent Iatmratorien. llin ehtimute is thereforc h c r m r d i a t r betwren that for duplicatm (0.117) and that for intcraction (0.2!)3). I f cacti laborstory liwl rcportrtl oiily otic urialysis for cwh (wiipouIii1, his coniputal ion would linvc yic!ltli~tl tq)proYimatcly 0,2‘3. If each laboratory had made several analyse* on each compound, tBe rrsult of his compubtion would hRve bmt closer to ttint given hcre for duplicatrs.

-

A viilid cstiriintc of the precision must not depciid on tlicb particular apportionment of analyscfi between laboratories and tiuplicates. The t echniyue of the analysis of variance clearly differentiates between the two estiniates of precision. It extracts from the data the precision based on interaction, axid that is the one which would more accurately predict the dispersion of reported differences between two compoundj if these were sent to a number of different laboratories for analysis and analyzed in duplicate. This approach also confirms, by the contrast in magnitude of the two estimates of precision, that the reproducihility of the conditions as achieved between different laboratories is much less satisfactory thau that ninintained for duplicatw run in one hhoratory. LWERATUHE CITEI)

Brownlee, K. A., “Industrial Experimentation,” 2nd ed., RrookIyn, N. Y., Chernicsl Publishing Co., 1947. (2) Fisher, R. A., “Statistical Methods for Research Workers.” 8th ed., London, Oliver & Boyd, 1941. (3) Fisher, R. A., “The Design of Experiments,” 2nd ed., London. Oliver & Boyd, 1935. (4) Finher, R.A., and Y ~ t e sFrank, , “Statistical Tables for Biologicd, .4gricultural and Medical Ile.setirch,” 2nd ed., London, Oliver & Boyd, 1943. (5) Power,P.W., IND. ENO.CHEI.,ANAL.En., 11, G W 7 . 7 (1839). (6) Smallwood, H. M., J. (:hem. Education, 23, 352-f1 (1940). (7) finedecor, 0.W.,“Statistical Methods Applied to Experimena in Agriculturc! and Biolow,” 4th ed., Amm, Iowa, Iowa State (1)

College Press, 1840. (8) Yates Y., “Design and Analysis of Factorial Experiments."

Techniral Cornrriunication 35, Hiirpctndon. England, Impend Bureau of Soil Science, 1837.

P ~ c r r v s oSeptanibor 10, 1048.

Teaching Students How to Evaluate Data T

ilE very idea of introducing niore subjects into ail alrcady overcrowded currioulum fills the teacher of chemist>ry with dread. However, the subject of statistics is so importtriit in industry and is becoming so increasingly important in the devclopment of analytical and h t i n g methods, that Bome introduction to the subject should be given in undergraduate courm. Furthermore, in graduate ooursea in sualytical chedxt,ry, Rufficient meterin1 on the application of ststiatics should be introduced 80 that prospective remarch chemisb will realise that they can ihorten their rasoarch work to an apprwiahle exhnt hy the

proper tlcaign of t!spcriniciitK aiid the proper USCof the dttb olr taiiid from their csperinients, Tho pre~entsutjhors do not ink

licve tlirtt sufficienl niatmial could ha bitroducod in thc usual courses in analytical chemistry to givo the atudenb suficicntj background to handlo NtatistricaItectiniqucw compatcntly, or c v w fully to understand statintird implications. IIowiwer, the students can be niado to rtxtlisct the irnportanro of the atatixtid approach and they rttn bo taught H few itenls of denientary niatistiral manipulation. At prosunt, int,rodurtory t o x t h k ~in quantitative nialysis

1141

V O L U M E 20, NO. 12, D E C E M B E R 1 9 4 8

l'he general topic of whwt cun he twught undcrgracliiates in the beginning WBUKW in quantitative unulyais is discussed PI well ar what can be taught graduate utudents corimrning the wncupta of preciaion and accuracy. I t ir believed vufficient to acquaint undergraduate student. with the conceptr of the arithmetical mean and the standard deviation, and the relationr derived regarding the lirnite of variation of an observed average aa well IIE the ooncepta of confidence limit8 and quality mintrctl charts. Althnirph

usually content themwlvt*s with little beyond the discuwiori of the term, accuracy and precision. Ia their >. laboratory work the students in the usual beginning quantitative analysis course merely calculate the average of two or three reuulb. This repreaents, ia many c w , the total introduction to the theory of errors and statistical methods given to Fbum 1 uadergraduatee. The quation is: what should be taught to these students? What is said below is merely por-onsl opinion concerning what might be taught to undergraduates in the limited time available. At beat, this material will have to be included in an hour or two of lecture, supplemented, perhaps, by the assignment of several exerciees to be done by the studenta in the application of statistica to problem.

2

irridergruduute etuctentr cannol be expwtwl irdlvidually to perform sufiicient expcritnentr tu apply etatintical conceptr with any degree of vdidity, i t & pomible to give them data on wbich they can bus atatiatioal operations,, in order to fwrniliarim thernnelves with the mechanic# involved. Strrvs should be placed on the student%acquiring win inlroduclian to the basic concepts which can be i i d to evaluate the validity of data whether ohtuined in the cheniisrl lahorntory or elsewhere.

siniilar prtions of thc m i i v subttiiiw to okttaiii rcpliwtv rturultr that do not coincide; the latter featurv perplexes studentr. Mast studonb have difirult y in realizing why prccituc! resultn arc not in themselves a guarantee of accuracy, or why resultx tliffering by Rcverd tenths of 1u$ may yield an wt*uraksveragv. One of the diffirultioil which pemist through the undwgraduete curriculum into the graduate training is a lack of uncierstanding of xignificant figures. It is not uncommon to nec the results CMICUlated by graduatch students in connection with problenis or with uxprinierital data exprtmstd to far inore significant figurcw than are warranted by either thv data available or lhv prwision and ~iwrrrcyctf the expt.rimentaI ~ i ~ c ~ a n ~ r ~ ~ m c ~ i i t s .

BASIC CONCEPTS

There is probably no rea1 argument concerning the indusion in the beginning course in quantitative analysis of an ample cousideration of the factors which influence the reliability of a determination, the proper uee of significant figures, and Bome understanding of the nature of deviatron from the most probable value or from the average value. Among the allied factors that might be considered are the general nature of experimental error, and the factors which will c a w one person using the ciame method 011

Figure 3

In their presentation of iiiakrid to students, the present authors have availed themselves of the published work of the American Society for Testing Materials (I), and of individual authors such as Mitchell (91, Moran (S), and Wernimont (4). While the material preaentod here is familiar to all with any statistical background, it niay k of value to othw t~achc*rsof snalytiral rhmixtry. MEAN AND S1'ANI)ARI) I)EVIA'I'ION

Figure

Tho Ytudenb should be introduccd to the nornial frequcncy distribution curve as shown in Figure 1. In thib: familsr h u m p shaped or bell-shaped curve, the number of itonis of a given value is plottoid against the valuu. A vivid illustration of the validity of the iiormal curve in practice and of the influence of experimental r w u h oil the &ape of the curve can be soen in the data obtained in thct snalynin of a ~amplcof ce~ncntfor ailica and calrium by 182 analysts. (Figurea 2 and 3 aw reproduced fmrn photographa given to the authors by the late Father Francis W. Powor of Fordham liiiivtmity. The data were apparently t a b

A N A L Y T I C A L CHEMISTRY

1142

-

cloviatioii; tlw rolv ol (n 1) w P (+INfrom a coolxwbtiw study of conant, antily~iih; I Iic ~iiil~lishod ing iiiodificd nt1~11(1ard rcctioii furtor is iriilicvitrtl. sourre of tlio data, if any, could not be lorahd.) Ipi~ure2 showri thu remlts obtained for silica. I l ( w the tlutu trrv in good nytcvnicnt, and rcsult in a tall, slondrr curve. Thc sttitistid npturc of tlic datrr coinliarid to thc noriiial i'urvil is well einph~uizt*dI)y thu rctsultcl tailing to ltms thtin atid to nioro than '25%. Incitiuntally, wch a curvc is it good i i i o ~ ~ l i ~ MANGE buildcr for students in indicating tliat cven cqit!rirnced rriinlysts lfaviii~ititrodwed the students to h two functiotrs of tlici clo not always get perfect results. Figure 3 hht)\\.S tho tlirtrr ohnican iiiid the s t a n d ~ r ddeviation, it will thon tx! IKMHiblu to tain(*d on calcium by the same group. Ilcrc the tlntrr foriii :I indicatc to then1 how thcau functions can IN u~cclto cst imab t l i c curve which is lcss t d l and less slrndcr. lwrcentage of thu total nunrber of a group of oxpurimontal VBIIJI*~ The grapliical armnqrnwnt H of I+'igures 3 :riitl 3 enablc tlic instructor to prcwrit to thv s t ~ ~ t l r ntho t s ronwpt of tlic nornid curvr and its changing shapc. in narordance with t h i s ii:iturc o f thc d a h obtained ax rc1g:ttrtIs accuracy ant1 precision. A fundanientsl introduction of tlic students t o *tutistiral ti)oIs 68 21. 95 15, 99 '3. 0 IO PO 30 40 50 60 10 80 90 95, can then be obtained by acquainting thum with two basic statis99 595 1 I I I [ 1 I / I l j I , , I , , tical concepts characteristic of tho distribution of the espcrii ' " i ' I . . ' , I " " , i2 i '2576' i 0 1 1 262' I645 I 960 ' 2 326 3 mental data: (1) a measure of the average or iwitral tendcncy of C b745' data such as the usual arithmetic niean or average, arid (2) u.11 index to the spread or dispersion of the values obtained about Figure 5 the central value mentioned such as the standard dcvintiori These values might be defined as follows: I

-

--

x..-

Arithmetic mean or aversge

XXi

n

Table I.

Silicu Content Culcu1rtiu;ie for Control Charts

Test No.

s,

XI

XI

Er

?u

H

Sample 1

where n is the number of values in the set and Xi is an individual value representing all values from XI to X.. The summation limits have been omitted, as tlicy are readily defined and explainrd by the instructor; their inclusion in the formulas, as in the one for average deviation subsrquently presented, often confusee beginning students. This treatment is ill agreement with the recommendations of the manual on prcsentation of data issued by the.Anlcrican Society for Testing Materials (1). In this manual it is rec0111mended that, "given a set of n observations of a single variable obtained under the same essential conditions," there be presented "as a minimum, the average, the standard deviation, and the number of observations."

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

10.06 9.90

10.09 10.01

10.05 10.13

30.20 30.04

10.07 10 01

0.01

!).so

9.81 10.12 10.28 10.11 10.32 9.97 10.06 10.08 9.56

9.90 10.32 10.36

60 30.40 30.90 20.33 30.95 80.17 30.21 29.93 28.73 29.77 30.30 30.22 29.94 30.55 30.53 30.28 29.84 30.43 29.88 30.39 31.37 31.21 30.33 29.87 29.98

9.87 10.13 10.30 10.16" 10.32 10.06 10.07 9.98 9.58 9.92 10.10 10.07 9.98 10.18 10.18

0.18 0.36 0.10

10.09

0.06

'3.90 10.26 10.22

10.31 9.88 10.03 9.83

9.56 9.74 10.20 10.05 0.82 10.10 10.0'3 10.06 9.86 10.09 9.96 10.11 10.37 10.36 10.08 9.50 9.67

!O.Ol

.0.01

10.07

1o.w

10.14 10.13 10.10 9.85 10.11 9.96 10.14 10.47 10.41 10.10 10.22 0.90

n = 80

ResiiltaI rejected

l0:32 10.82 10.12 10 10 9.59 10.02 10IO9 10.10 10.10 10.31 10.31 10.12 10.10 10.23 9.96 10.14 10.53 10.44 10.15 10.15 10.41 Sum Av.

%I)

--

9.95 10.14 9.96 10.13 10.4u 10.40 10.11 9 go 9.96

808.35 10 08

0.23

o:oi

0.44

0.09

0.27 0.03 0.28 0.19 0 05 0.28 0.21 0.22 0.25 0.14

0.00 0.03 0.16

0.08

0.07 0.72 0.74

5.23

Sample 2

1 2 3 4

9.04 IO.OO 10.08

9.90 10.14 10.08

0.97 0.32 10.08

29.87 30.42 30.17

9.96 10.14 10.06

0.03

5

9.34 9.45 9.98

10.17 10.03 9.9!l 10.06 10.13 10.02 10.20 10.10 10.09 10.10 10.07 10.14

10.26 10.23 10.02 10.08 10.10 10.11

29.77 29.71 29.99 30.20 30.60 29.89 30.24 30.52 30.18 30.44 30.21 30.44

'3.92 0.90 10.00 10.07 10 $0 9.!I6 10.08 10.17 10.011 10.15 10.07 10.15

0.92 0 78 0 03

4

10.00

0 10

Figure 4

12 13 14

10.37 9.70 10.04 10.0s 10.05 10.08

io

0.98

11

15

Because the average deviation,

10.06

n

Ay. of 2.

1-1

n M frequently found in chemical librature, it nhould bu cli4inutl. As students are usually interested in epplying stat int,ictLl row mptn to small nunikrs of data, they ere introducod to tho follow-

- '9-

-

45

Resul taI rejected

la.00

10.28 10.14 10.26 10.08 10.32

Sum Av.

-

0.16

0.05

O.Of

0.?7 0.35

0.2u

0.23 0.09 0.18 0.02 0.34

452.65 10.06

I'reliininary control limits for range0 of 3 urninn all drtr: 0.217: limitn = 0 to (2.574 X 0.217) -A to 0.66 h n a l control limits for rrngea of 3 after discarding all valucs abovo 0.7. 3 2L4 0.155; limits 0 to (2.574 X 0.151) 0 to 0.40 I7 Control litnits for avera ea* Baniplo 1. 10.07 * Cl.823'X 0.155) 10.07 0.16 9.01 to 10.23. Sample 2. 10.08 * (1.023 X 0,155) 10.00 0.16 0.01 to 10.22.

.

-

- -

-

--

V , O L U H E 20, NO. la, D E C E M B E R 1 9 4 8

1143 range within which the true objective avorage, X’, rimy b expccted to lie with n given statiotical probabililiy. Thia om b8 done using thc averagu and standard deviation, m d the following relationship: Computation of Limits for Objt?ctive Average, F: X*au in P, per cent of observetione n < 25, uw nomograph of 11, a, P, n > 25, u t / d n i For large numbers of measurements, the formula involving t may be used to calculate a; for snisll numbers of data down to fnur, tho nomograph shown in Figure 6 can be used. Thus, for four measurenicnts on a sample, deturniining one constituent, the objective avcrage will fall within the limits of tho average *1.9 t h c a the standard deviation 95% of the time. In preeenting such material to students it should be emphasized that the objective average, of the observed m w u r e nients may not be identical with the true or most probable value, .Y’T, of the quantity measured, owing to systematic or constant errors. Such errors include incorrect gravimetric factom, erroneous titrant standardizations, and certain apparatus faulta. After consideration of the topics mentioned, the subject of control charts can be d i s c u d . By this time, the students will have enough background to understand the meaning of control limits and a state of control.

x‘,

8

Figure 6 which will lie in ntry stated interval about the average or central value. This can be done by acquainting them with two nomographic charts which are giv’en in the A.S.T.M. manual. Using the nomograph shown in Figure 4, the minimum fraction or percentage of the results which lie within a given range can be easily calculated. This chart is based on Tchebycheffs theorem. Its use is based only on knowing the average and the standard deviation, and on selecting some value oft. For example, for four obeervations-i.e., n = b i t can be scen that all four observations will probably fall within the region of the average plus or minus twice the standard deviation-i.e., select t = 2. This follows froiii the chart where, for 1 = 2, less than 25% of the measurements will be outside the limitssp ified. The nomograph of Figure 4 allows %ne to state without reservation that, on the presentation of a, X,and u, more than (1 1l/t*) of the total number of observations lie within the range, X * b. Further, to calculate the approximate percentage of the total nuniber of observations within given limits, as contrasted to minimum percentages within them limits, use is made of the nomograph shown in Figure 5. This nomographic chart is b a d on the distribution law integral. By means of this chart, it is possible to calculate the percentage of the total number of observations lying within any given symmetrical range about the average value. For t = 2, 95% of the observed results will ~ J C within the range of the mean plus or minus twice the standard deviation. In addition to thc average, standsrd deviation, and the number of observations, use of this chart require8 that the data be obtained under controlled conditions. The latter condition is probably fulfilled under careful analytical practice. Thus, it would be F i b l e to have tho students take a group of values obtained under normal laboratory conditions for anslyzing a given sample and to calculate what percentage of the total number of ~ b s e r ~ awill t i fall ~ ~ within, let us say, plus or minus twice the average deviation from the mean. Furthermore, i t is then possible to indicate to students how they might Iw able to calculste the confidenco rango or thc

During the current semester at Purdue, an interestin experiyear ment was performed in the second semester part of the course in quantitative anal@. The group of approximately forty students was divided into two sections. The members of each section analyzed the same sample. The sample ueed was a mixture of sodium chloride, potassium chloride, and d i m . The determinations were (1) material insoluble in water, chloride by precipitation as silver chloride and weiehinkand calculation of the sodium and potassium contents m g t experimental data obtained. The results of the various determinations for each of the two samples were then plotted in the form of control charts. The data obtained for the silica content of the two samples, which had the same percenteke of silica but differkg p r c e n t a y s of sodium chloride and potaasiym chloride, are given inTable In the same course, the students are given theresulta obtained over a period of years for a sample used in the first semester of the quantitative analysis course. They are asked to calculate the arithmetical mean or avera the averap deviation, and the standard deviation. The pro%m of rejection of observed values -an important problem to students in “quant”-is discussed.

kt

{:{

.

In the basic course in analytical chemistry for graduate students, various indexes of precision used in the literature in addition to those mentioned are discussed. Sets of research data are exhibited and various items such as the mean, average and standard deviations, and variance are calculated. The meanings of the various t y p of reliability statements that can be made me analyeed. In the seminar, in which all graduate students majoring in analytical chemistry participate, six hour-sessions were. devoted to a discussion of the material presented in the A.S.T.M. manual (I) and Wernimont’s (4) paper on the use of control charta in the analytical laboratory. ACKNOWLEDGMENT

The authors wish to thank the American Society for Testing Materials for permission to reproduce in Figurea 4, 5, and 6 material froni (I). LITERATURE CITED (1) Am.800.Touting Materida, “A.S.T.M. Manud on the h n t a tion of Data,” Philadelphia, 1946. (2) Mitchell, J. A., ANAL.CXEX., IS,961-7 (1947). (3) Moran, J., INU.ENQ.CHEX.,ANAL.ED.,18,380-4 (1941%

(4) Wernimont, G.,ZW., 18,587-82 (1W). RECElVED

September 10, 1048.