Chapter 7 분산분석

Chapter 8 실험계획및분산분석 (Experimental Design & ANalysis Of VAariance, ANOVA) 2017/5/01

8.1 선형모형과분산분석 (Linear Model & Analysis of Variance) 선형모형 (linear model): 설명변수들의선형의선형결합의형태로반응변수를설명하고자함. (to explain the response variable by the linear combinations of the explanatory variables) 분산분석 (analysis of variance) : 전체변동을몇개의성분으로분할하는기법 (Divide total variation into several components) 전체변동에대해각각의변동요인의기여규모를파악 (contribution of particular components) 목적 (Aims) : 모분산의추정과가설검정 (estimation & testing for the variances) 모평균의추정과가설검정 (estimation & testing for the means)

* motivation 비교하고싶은그룹이두개이면 (comparisons of two groups) -> t-test 비교하고싶은그룹이두개이상이면 (more than two groups) -> 두개그룹씩뽑아서쌍을만든후에여러개의 t-test 를실시한다. (pairwise t-tests) 번거롭기도하고이론적으로틀린결론에도달할수있다. (cumbersome & theoretically wrong -> 다중비교의문제 (multiple-comparisons problems) 전체자료를사용하지않고자료의부분만을사용하므로효율이떨어진다. (efficiency problems due to the usage of partial data) 전체자료를이용하여서세그룹이상을비교하는분석 (more than 3 groups using whole data) -> 분산분석 ANOVA ( 종속변수는연속형, 독립변수는이산형 ) response var: conti, explanatory var: categorical

<Ex 8.1.1> 혈청콜레스테롤을낮추는세가지약 A, B, C 를비교. 실험에참여한사람들에게 A, B, C 중하나를처방한뒤에, 혈청콜레스테롤이줄어들었는지측정. 처방받은약에따라그룹을나눴을때, 각그룹별표본평균에변동과각그룹내의관측값의변동을생각. 이때전자는그룹간변동 (between group variation), 후자는그룹내변동 (within group variation). 그룹간변동은처방약의효과, 그룹내변동은식습관의차이, 유전적차이등다양한이유로인하여발생. 따라서그룹간변동이그룹내변동에비하여상대적으로작으면, 약의효과는동일하다고결론. Response variable: reduction of cholesterol level Explanatory variable: drugs A, B, C Within group variation: due to other factors like genetic or nutrition, etc Between group variation: drug effect

Cholesterol level within group variation Cholesterol level A B C Treatments between group variation A B C Between group variation: small Within group variation: large

8.2 일원배치분산분석 (One-way ANOVA) [ 완전확률화계획법 ] : (complete randomization) 처리변수 (treatment) 1 2 3 k x 11 x 12 x 13 x 1k x 21 x 22 x 23 (treatments) x 2k x 31 x 32 x 33 x 3k x n1 1 x n2 2 x n3 3 x nk k 합 (total) T.1 T.2 T.3 T.k T.. (total) 평균 (average) x.1 x.2 x.3 x.k x..

모형 (model) x ij j ij ij번째측정치 j처리의평균 ij번째오차 ij-th observation mean of j-th treatment group error of ij-th observation j ij k j j 1 k j j : 전체평균 Grand mean : j번째처리효과 Effect of j-th treatment group

x ij = μ j + ε ij = μ + τ j + ε ij 처리변수 (treatment) 1 2 3 k x 11 x 12 x 13 x 1k x 21 x 22 x 23 x 2k x 31 x 32 x 33 x 3k (treatments) x n1 1 x n2 2 x n3 3 x nk k 합 (total) T.1 T.2 T.3 T.k T.. 평균 (average) x.1 x.2 x.3 x.k x.. pop mean μ 1 μ 2 μ 3 μ k μ effect τ 1 = μ 1 μ τ k = μ k μ

모형의가정 (Assumptions of the model) (a) 독립확률표본 (independent random sample) (b) x i,j ~N(μ j, σ 2 j ), i = 1,2,, n j k번째 -> j번째 (c) σ 1 2 = σ 2 2 = = σ k 2 = σ 2 (d) μ j 의평균은 μ. 따라서 τ j = μ j μ 이라고하면, τ j = 0 (e) x ij 의평균은 μ j 이고 ε ij 의평균은 0 이다. (f) ε ij 와 x ij 의차이는상수, ε ij 의분산은 x ij 의분산과동일. 따라서오차항의분산은 σ 2 이다. (g) ε ij 는독립이며정규분포를따른다. (indep. & normally dist) => x ij = μ j + ε ij = μ + τ j + ε ij, ε ij ~N(0, σ 2 ) independent

모형의가설 (Hypothesis of the model) H 0 μ 1 = μ 2 = = μ k vs H A 적어도하나이상의 μ j 는다르다. 만약영가설이사실이면 ( 즉, 모평균이서로같다면 ) 처리효과는모두 0 이므로영가설과대립가설은다음과같이표현. H 0 τ j = 0, j = 1,2,, k vs H A 적어도하나이상의 τ j 은 0 이아니다. (More than one τ j s are not equal to 0.) 영가설이사실이고가정이만족될때, 모집단의분포 등분산과정규성의가정이만족되나영가설이사실이아닌경우모집단의분포 Under H 0 with normal and homogeneity assumptions Under H A with normal and homogeneity assumptions

총제곱합 (sum of squares, total) k SST ( x x ) k j 1 i 1 n j ( x x x x ) j 1 i 1 n j ij.. 2 ij. j. j.. 2 k n j k n j k n j 2 2 ( xij x. j ) 2 ( xij x. j )( x. j x.. ) ( x. j x.. ) j 1 i 1 j 1 i 1 j 1 i 1 k n j k 2 2 ( xij x. ) j n j ( x. j x.. ) j 1 i 1 j 1 Within-group SS Among(Between)-group SS

Within-group SS SST SSW SSA within among group 그룹내제곱합그룹간제곱합 그룹간평균제곱분산비 = 그룹내평균제곱 MSA variance ratio= MSW Among(Between)-group SS -> 분산비가커지면그룹간의 variation 이크다. 그룹간의성질이다르다. 그룹의효과가크다. ->larger VR -> larger between-group SS 0 -> groups are different -> bigger group effect!

σ 2 의불편추정량 MSW k n j j 1 i 1 k j 1 ( x x ) ij ( n 1) j j 2

ANOVA Table factor Sum of squares Degree of freedom Mean square Variance ratio Between group Within group total 요인제곱합자유도평균제곱합 F 집단간제곱합 SSA = j=1 집단내제곱합 SSW = j=1 총제곱합 k k k SST = j=1 n j n j i=1 n j i=1 x.j x ij x ij x.. 2 x.j 2 x.. 2 k 1 MSA = SSA/(k 1) N k N 1 MSW = SSW/(N k) MSA MSW

<Ex 8.2.3> 소의연령에따른육류의셀레늄 (selenium) 농도비교 (Age group of cows and selenium concentration of milk) (1) 데이터 표 8.2.3 연령에따른셀레늄함유량 (mg=100g) 나이그룹 A B C D 1820 1483 191 724 1020 1652 775 752 2588 1723 1098 613 805 1309 1393 804 2670 727 644 918 631 1002 533 1182 1022 1463 136 949 641 966 734 1243 1555 1777 1605 877 760 788 485 985 222 1129 1247 1368 1085 472 449 1295 1197 1529 1692 775 471 236 1676 1249 1422 697 1307 771 831 754 1520 445 849 344 869 698 937 489 990 1199 961 513 167 1022 2575 489 429 239 731 824 1073 1426 2408 798 944 1130 448 948 1846 1064 631 1096 1034 991 222 1088 629 1016 1261 590 721

요인 제곱합 자유도 평균제곱합 F 집단간제곱 5918736.75 3 1972912.25 9.33 집단내제곱합 23038971.22 109 211366.71 총제곱합 28957707.96 112 Critical value = 3.95, alpha=0.01 > qf(0.99,3,109) [1] 3.966509 > 1-pf(9.353,3,109) [1] 1.48582e-05 >

SAS program data cele; input value group $ @@; cards; 1820 A 1483 A 2588 A 1723 A 2670 A 727 A 1022 A 1463 A 1555 A 1777 A 222 A 1129 A 1197 A 1249 A 1520 A 489 A 2575 A 1426 A 1846 A 1088 A 912 A 1383 A 191 B 1098 B 644 B 136 B 1605 B 1247 B 1529 B 1422 B 445 B 990 B 489 B 2408 B 1064 B 629 B 724 C 1020 C 613 C 805 C 918 C 631 C 949 C 641 C 877 B 760 C 1368 C 1085 C 1692 C 775 C 697 C 1307 C 849 C 344 C 1199 C 961 C 429 C 239 C 798 C 944 C 631 C 1096 C 1016 C 1025 C 948 C 1652 D 775 D 752 D 1309 D 1393 D 804 D 1002 D 533 D 1182 D 966 D 734 D 1243 D 788 D 485 D 985 D 472 D 449 D 1295 D 471 D 236 D 1676 D 771 D 831 D 754 D 869 D 698 D 937 D 513 D 167 D 1022 D 731 D 824 D 1073 D 1130 D 448 D 948 D 1034 D 991 D 222 D 1261 D 590 D 721 D 42 D 994 D 375 D 767 D 1781 D 1187 D proc anova data=cele; class group; model value=group; means group/tukey; * means group/bon; run;

* Multiple Comparisons ( 다중비교 ) ex) significance level = for a test Let H : 0 p( do not reject H H is true) 1 01 1 01 01 H : 0 p( do not reject H H is true) 1 02 2 02 02 then p( do not reject H H ) where H H and H 0 0 0 01 02 p( do not reject H01 and do not reject H02 H0 (1- ) (1- ) (1- ) In general, if we want to test, then k (1 ) (1 ) 2 1 2 3 0 k 4 1 0.1855 0.8145 (.95).95 overall is 0.1855, not 0.05 -> inflated type I error!! )

* Bonferroni Correction : Set individual significance the overall significance level is about for m multiple tests. m m=4 4 0.05 1 0.95 1 0.05 4 example) When we have 10 hypotheses, Individual p=0.05 -> multiple comparisons problem (too many false findings) Individual p= 0.05 10 0.005 This is often called Bonferroni corrected p-value.

[ 처리그룹쌍별두모평균차이의검정 ] Detecting pairwise differences After rejecting H0 : 1 2 5 have larger differences?, which pairs 1. LSD (least significant difference, 최소유의차검정법 ) 2. Duncan s new multiple range test Duncan 의새로운다중범위검정법 3. Tukey s HSD Liberal Conservative Duncan LSD SNK Tukey HSD Scheffe

[Tukey 의 HSD (honestly significance difference) 검정 ] MSE HSD = q n, k, N k j n j MSE HSD q n * *, k, N k * j n j 's are the same : sample size of smaller cell ymax ymin q, k, N k : dist of, S 2/ n : significance level, k : number of gropus, N k: df [Bonferroni 방법 ] 계산된 p-value에가능한모든방법의수 k 2 를곱함. Bonferroni corrected p-value: multiply # of all possible k methods to the p-value 2

데이터의각처리그룹별표본평균의차이 Differences of means between pair of groups A B C D A - 463.43 574.65 596.63 B - 111.22 133.20 C - 21.99 D - α=0.05. k = 4, N k = 109 > qtukey(0.05, nmeans= 4, df=109, lower=f)= 3.689, MSE= 211366.71

Tukey 의 HSD 결과 개별영가설 HSD* 검정결과 H 0 : μ A = μ B HSD = 3.689 211366.71 2 1 22 + 1 14 = 410.0038 463.43 > 410.0038 이므로 H 0 을기각함. H 0 : μ A = μ C HSD = 3.689 211366.71 2 1 22 + 1 29 = 339.0679 574.65 > 339.0531 이므로 H 0 을기각함. H 0 : μ A = μ D HSD = 3.689 211366.71 2 1 22 + 1 48 = 308.7657 596.63 > 308.7522 이므로 H 0 을기각함. H 0 : μ B = μ C HSD = 3.689 211366.71 2 1 14 + 1 29 = 390.2862 111.22 < 390.2691 이므로 H 0 을기각하지못함. H 0 : μ B = μ D HSD = 3.689 211366.71 2 1 14 + 1 48 = 364.2698 113.20 < 364.2539 이므로 H 0 을기각하지못함. H 0 : μ C = μ D HSD = 3.689 211366.71 2 1 29 + 1 48 = 282.0575 21.99 < 282.0452 이므로 H 0 을기각하지못함.

8.3 완전확률화완전블록계획법과이원배치분산분석 (Randomized complete block design & Two-way ANOVA) R.A.Fisher (1925) : to compare the yields of certain species 땅을블록 (block=land) 으로나누고블록안에서 Randomize (other factors) in a block 하는것이다. 처리 블록 1 2 3 k 합 total 1 x 11 x 12 x 13 x 1k T 1. 2 x 21 x 22 x 23 x 2k T 2. 평균 average x 1. x 2. 3 x 31 x 32 x 33 x 3k T 3. x 3. n x n1 x n2 x n3 x nk T n. 합 Total T.1 T.2 T.3 T.k T.. 평균 Average x.1 x.2 x.3 x.k x n. x..

모형 x ij = μ + β i + τ j + ε ij i = 1,2,, n; j = 1,2,, k x ij : 각실험단위로부터얻은관측값 (Observation) μ: 미지의상수로서전체평균. (grand mean) β i : i 번째블록의블록효과 block effect for i-th obs τ j : j 번째처리의처리효과. trt effect for j-th obs ε ij : 그외효과들의총합. random error 모형의가정 a x ij : 랜덤독립표본 (random and independent sample) (b) x ij ~independent N μ ij, σ 2 ε ij ~independent N 0, σ 2 k (c) j=1 τ j = n i=1 β i = 0 블록효과와처리효과는가법적이다. Block effect & trt effect are additive.

가설 (hypothesis) H : 0 j 1,2,, k H 0 j :All 0 is not true. Some 0. A j j * k SST ( x x ) n j 1 i 1 ij.. 2 k n k n j k n 2 2 2 ( xi. x.. ) ( x. j x.. ) ( xij xi. x. j x.. ) j 1 i 1 j 1 i 1 j 1 i 1 SST SSBl SSTr SSE df : nk 1 ( n 1) ( k 1) ( n 1)( k 1)

ANOVA table 요인 (factor) 제곱합 (sum of square) 자유도 (df) 평균제곱 (mean square) F 처리 (trt) SSTr (k 1) MSTr = SSTr/(k 1) MSTr MSE 블록 (block) SSBl (n 1) MSBl = SSBl/(n 1) 잔차 (residual) SSE (n 1)(k 1) MSE = SSE/(n 1)(k 1) 합 (sum) SST kn 1

Ex. 8.3.1 약의종류와나이에따라치료까지걸린시간 Trt time by drug and age group 약의종류 (drug) 나이그룹 (age group) A B C 합 total 평균 Age average < 20 11* 8 10 29 9.7 20-29 6 5 11 22 7.3 30-39 7 10 13 30 10 40-49 9 12 13 34 11.3 50-10 17 15 42 14 합 total 43 52 62 151 평균 average 8.6 10.4 12.4 10.5

response<-c(11, 6, 7, 9, 10, 8, 5, 10, 12, 17, 10, 11, 13, 13, 15) drug<-factor(c(rep('a',5),rep('b',5),rep('c',5))) age<-factor(rep(1:5)) dat<-data.frame(response=response,drug=drug,age=age) anova(lm(response~drug+age,data=dat)) Analysis of Variance Table Response: response Df Sum Sq Mean Sq F value Pr(>F) drug 2 36.133 18.0667 3.4522 0.08300. age 4 71.733 17.9333 3.4268 0.06505. Residuals 8 41.867 5.2333 --- Signif. codes: 0 *** 0.001 ** 0.01 * 0.05. 0.1

reduction of response time 8.4 요인실험과이원배치분산분석 (Factorial Experiment and two-way ANOVA) 반응시간 (reduction of response time ) = 약품수준 ( 소량, 중간, 다량 )* 연령층 ( 중년, 노년 ) drug level (min, med, max)*age(mid, old) 교호작용이없을때 (Without interaction) 요인 B 약품용량 (Factor-B, drug level) 요인 A 연령 Factor A-age j=1 j=2 j=3 중년층 (Mid) i=1 5 10 20 노년층 (old) i=2 10 15 25 age Drug level Drug dosage age

교호작용이있을때 (With interaction) 요인B 약품용량 요인A j=1 j=2 j=3 j=2-1 j=3-2 - 연령 중년층 (i=1) 5 10 20 5 10 노년층 (i=2) 15 10 5-5 -5

요인 (factor) A 1 요인 (factor) B 1 2 b 합 total 평균 average x 111 x 121 x 1b1 x 11n x 12n x 1bn T 1.. x 1.. 2 x 211 x 221 x 2b1 x 21n x 22n x 2bn T 2.. x 2.. a x a11 x a21 x ab1 x a1n x a2n x abn T a.. x a.. 합 total 평균 average T.1. T.2. T.b. T... x.1. x.2. x.b. x...

모형 x ijk = μ + α i + β j + (αβ) ij +ε ijk, i = 1, 2,, a; j = 1, 2,, b; k = 1, 2,, n x ijk : 관측값 (observation) μ: 전체평균 (grand mean), α i : 요인 A의효과 (effect of factor A), β j : 요인 B의효과 (effect of factor B), (αβ) ij : 요인 A와요인 B의교호작용 (interaction), ε ijk : 실험오차 (random error).

모형의가정 (Assumptions of the model) i. 각칸의관측값들은두요인의수준들의특정조합으로정의된모집단에서뽑은 n 개의독립표본으로구성되어있다. (independent sample) ii. ab 개의모집단은각각정규분포를따른다. Normal distribution iii. 모든모집단은동일한분산을가진다. Same variances x ijk = μ + α i + β j + (αβ) ij +ε ijk, ε ijk ~iid N(0, σ 2 ), i = 1, 2,, a; j = 1, 2,, b; k = 1, 2,, n

Hypotheses( 가설 ) H : 0 i 1,, a 0 H : Not H 0 for some i. A 0 0 i 0 0 H : 0 j 1,, b j H : Not H 0 for some j. A 0 i j H :( ) 0 i 1,, a j 1,, b ij H :Not H ( ) 0 for some i, j. A ij SST=SSA+SSB+SSAB+SSE

이요인완전확률화설계의분산분석표 ( 고정효과모형 ) ANOVA table for two-way complete randomized design 요인 factor 제곱합 SS 자유도 df 평균제곱 MS F A SSA a 1 MSA = SSA/(a 1) B SSB b 1 MSB = SSB/(b 1) AB SSAB (a 1)(b 1) MSAB = SSAB/(a 1)(b 1) MSA MSE MSB MSE MSAB MSE 처리 trt SSTr ab 1 잔차 residual SSE ab(n 1) MSE = SSE/ab(n 1) SST = SSTr + SSE SSTr = SSA + SSB + SSAB

<Ex. 8.4.2> 간호사의가정방문시간 (time of staying home for a nurse) = 간호사의연령, 환자의질환 (age of the nurse, disease of the patient) 모형 (Model) x ijk i j ( ) ij ijk i 1,, a j 1,, b k 1,, n

* miscellaneous ( 기타 ) Log transformation: when normal assumption is violated. Normality is still problematic even after the variable transformation. Sample size is too small to check normality -> Nonparametric approach e.g. income, concentration

Type of Sum of Squares * Type Ⅰ:sequential (if we know the relative importance of the variables) Type Ⅱ: partial without interaction terms **TypeⅢ:partial with interactions (If we don t know the relative importance of the variables) TypeⅣ: There are missing cells (if none, same as TypeⅢ) *, ** : defaults One way ANOVA model : Y Ai ij