Interpretation of Experimental Results

---

I/EC A

W

O

STATISTICAL R

K

B

O

O

K

DESIGN

F E A T U R E

by W. J. Youden, National Bureau of Standards

Interpretation of Experimental Results Statistical design of experiments includes a definite procedure for evaluation of the data to answer specific questions f \

GOOD STATISTICAL DESIGN IS l i k e a

blueprint for a construction j o b . Builders u n d e r t a k e construction with a particular function in m i n d rather than putting u p a structure a n d wondering w h a t to d o with it. I n a similar way an experimenter sets u p a program to get answers to certain specific questions. Along with the program should go a predetermined arithmetical procedure for ascertaining whether or not convincing answers have been obtained. Statistical Analysis of Data Collections of d a t a are sometimes brought to a statistician with a request that a statistical analysis be m a d e of the material. This implies a complete reversal of the traditional experimental approach. Presumably the statistician is expected to find some answers lurking in the data. O n c e in possession of these answers the questions c a n be formulated. A report on the d a t a is expected in effect to say, " T h e s e are the questions that your work has answered." T h e r e is a substantial penalty attached to this approach to data. Naturally the d a t a are b o u n d to support a question that was suggested by the data. Nothing remains of the classical notion that an experiment is a test of a n hypothesis or tentative guess. This is not to say that data, once obtained for the purpose of testing certain ideas or getting desired information, should not be carefully scrutinized. O u t of such scrutiny there sometimes come strong hints as to the proper questions to set for the next experiments. Furthermore, recent progress in statistical theory has provided a sound basis for framing questions after the data are in hand, but only at a price of requiring substantially more evidence than would have been needed if the questions h a d been

formulated data.

prior to obtaining

the

Factorial Experiments Statisticians did not invent the method of factorial experimentation. No statistician is needed to suggest to a ceramic worker t h a t h e might try three different particle sizes, three different compacting pressures, a n d three different firing temperatures. A thorough exploration of the effects of these factors on the properties of the product would lead to the preparation of 3 X 3 X 3 or 27 different ways of making the product. T h e results might be exhibited graphically by showing nine curves, each curve relating the observed property to, say, the firing temperature. T h e curves would naturally fall in three sets; a set for each particle size a n d within each set a curve for each pressure. Very often this graphical representation suffices a n d no statistical condensation of the d a t a is undertaken. Factorial experiments with m a n y factors a n d m a n y levels suffer from the rapid increase in the n u m b e r of combinations a n d consequent increase in the work involved. In addition, the graphical presentation soon becomes cumbersome. Statisticians have devised a novel way of transforming the d a t a obtained in factorial experiments into other n u m bers that bring out more clearly the roles of the various factors—both separately and in conjunction with other factors. Statisticians have also devised programs for fractional factorials—i.e., subsets of combinations carefully chosen from the large n u m b e r of possible combinations of the factors. T h e subsets are not, as a rule, a m e n a b l e to graphical presentation and consequently some sort of statistical reduction of the d a t a becomes necessary.

This reduction follows a standard procedure which converts the o b servations into quantities called m a i n effects a n d interactions. T h e n a t u r e of these quantities, which are always certain sums a n d differences taken a m o n g the observations, m a y be illustrated with a simple example, which was suggested by a chemist. Melting Points of Organic Compounds T h e suggestion was m a d e that it was easy to collect d a t a from the literature for a vivid example of the analysis of factorial data. Consequently the melting points for the following eight benzene derivatives were obtained and listed below the compounds:

,N0 2 /

M.p., ° C. - 4 5

N02 84

32

Br

Br

\N02

NOi 43

Position Br

Br ,N0 2

M.p.,°C. - 3 1

N02 127

43

/ \ N 0

2

NOi 75

T h e intent of the investigation is to ascertain the effect on the melting point of substitution of certain groups in various positions in the benzene ring. T h e three positions 1, 2, a n d 4 correspond to three factors. T h e r e is a choice of chlorine or bromine for position 1, and presence or absence of the nitro group for positions 2 a n d 4. Operations such as the following m a y be m a d e upon the data. T h e effect on replacing chlorine by bromine m a y be observed for four sets of circumstances. V O L 49, NO. 12

·

DECEMBER 1957

73

A

I/EC

STATISTICAL DESIGN Μ,Ρ.,Ό. Cl Br

Halogen benzene 1-Halogensnitrobenzene l-Halogen-4nitrobenzene l-Halogen-2,4nitrobenzene AT.

Temp. Change

—45

—31

+14

32

43

+11

84

127

+43

43

75

+32

28

53

25

A glance at the changes in tem­ perature shows that about the same change takes place when bromine replaces chlorine in benzene as when there is a nitro group adjacent to the halogen. A different change takes place if there is a nitro group p a r a to the chlorine. This change is about the same whether or not there is a nitro group in the ortho position. O n the average, for the four circumstances, the change in melting point for the bromine derivatives over the chlorine derivatives is 25° C. This is called the m a i n effect associated with this contrast. Chem­ ists, naturally enough, hesitate to make use of such an average as two quite different changes are being averaged together. No h a r m results from taking this average provided the next computations specified by the statistician are m a d e . These have to do with the questions that naturally follow. Is the temperature change, when bromine replaces chlo­ rine, independent of the presence or absence of the nitro group in the ortho position or in the p a r a posi­ tion? With no nitro group in the para position the average rise in the melting point upon substituting bromine for chlorine is 12° C . — quite different from the average rise of 38° C. when there is a p a r a nitro group present. T h e effect on the melting point of changing the occupant of position 1 therefore depends on whether position 4 is occupied. T h e two positions are said to interact with each other. T h e difference (26° C.) in these two changes is a measure of this inter­ action. T h e possibility of interaction be­ tween positions 1 and 2 is tested by taking the average of the results with position 2 vacant and com­ paring with the average result with position 2 occupied. These average changes are 28° and 22° C , respec­ tively. Apparently the effect of changing the chlorine to bromine is only slightly influenced by the 74 A

A Workbook Feature

presence of a nitro group in the ortho position. O n the other hand, there is a very pronounced inter­ action between positions 2 and 4. T h e presence of an interaction involving a factor means that the average effect for that factor should not be used, b u t rather should be computed separately for each in­ stance of the other factor with which it interacts. Furthermore, when a two-factor interaction is present, it should be examined to see if it persists for the separate instances of the third factor (and other factors if there are more than three). T h e interaction of positions 2 a n d 4 will now be examined separately for the chlorine and bromine com­ pounds (position 1). Effect of Ortho Nitro Chlorine Bromine compounds compounds No para nitro 32 - ( - 4 5 ) 43 - ( - 3 1 ) =77 =74 With para nitro 43 - 84 = 75 - 127 = -41 -52 Difference = interaction 118 126

Evidently the two-factor inter­ action of positions 2 a n d 4 is very m u c h the same for both the chlorine a n d bromine (position 1) com­ pounds. W e have, indeed, calcu­ lated w h a t is termed a three-factor interaction, which in this case (126 — 118) is negligible. Although the effect on the melting point of introducing an ortho nitro group in the chlorine compounds is altered greatly by the presence of a p a r a nitro group, this same alteration, both as to direction and magnitude, is observed from the bromine com­ pounds. T h e point to be m a d e is that visual inspection of the d a t a does not immediately suggest this information. Statistical design does provide a predetermined method of evaluation for the d a t a obtained in factorial experiments. T h e calcu­ lation of the main effects and inter­ actions directs attention to the important features in the data. T h e absence of interactions im­ plies a purely cumulative-additive effect on the property as various groups are a d d e d to the c o m p o u n d . Sometimes there are multiplicative effects a n d these give rise to inter­ actions. If logarithms are taken of the observations, the cumulative effects m a y then become additive

INDUSTRIAL AND ENGINEERING CHEMISTRY

a n d the interactions which com­ plicate the interpretation disappear. Experience with factorial ex­ periments has shown that inter­ actions involving more than three factors seldom reach disturbing mag­ nitudes. T h e use of fractional factorials rests u p o n the assumption that the higher order interactions are of little consequence. The deleted experimental combinations are those which sacrifice the oppor­ tunity to calculate the interactions that are deemed of little interest. In a very real sense the experimenter concentrates his efforts on the com­ parisons t h a t are of most importance to him. All three factors in the above example are of the type where discrete choices exist. A position is filled with either chlorine or bromine; it is either e m p t y or occupied with a nitro group. A choice a m o n g catalysts for a reaction is of the same type. O t h e r factors such as t e m p e r a t u r e or concentration are continuous in character. The experimenter then selects particular levels of such factors to provide the various experimental combina­ tions. W h e r e all the factors are continuous the experimenter m a y choose certain combinations of levels that facilitate the fitting of a m a t h ­ ematical function to the observations. Such a function is useful as a means of interpolation so t h a t the best operating levels for the factors m a y be chosen. T h e criterion m a y b e yield, purity, profit, or some com­ bination of requirements. Whatever the method of attack, the experimenter should be a r m e d in advance with a matching numeri­ cal procedure for evaluation of the data. Criteria can be set u p before­ hand forjudging whether or not any given factor is influencing the per­ formance of the process. Evaluation of the d a t a becomes objective a n d not a personal matter when the problem of interpretation is faced at the outset of the work.

Our authors like to hear from readers. If you have questions or comments, or both, send them via The Editor, l/EC, 1155 16th Street N.W., Washington 6, D. C. Letters will be forwarded and answered promptly.