Chapter 7 분산분석

Size: px

Start display at page:

Download "Chapter 7 분산분석"

괴옥 국
5 years ago
Views:

1 Chapter 8 실험계획및분산분석 (Experimental Design & ANalysis Of VAariance, ANOVA) 2017/5/01

2 8.1 선형모형과분산분석 (Linear Model & Analysis of Variance) 선형모형 (linear model): 설명변수들의선형의선형결합의형태로반응변수를설명하고자함. (to explain the response variable by the linear combinations of the explanatory variables) 분산분석 (analysis of variance) : 전체변동을몇개의성분으로분할하는기법 (Divide total variation into several components) 전체변동에대해각각의변동요인의기여규모를파악 (contribution of particular components) 목적 (Aims) : 모분산의추정과가설검정 (estimation & testing for the variances) 모평균의추정과가설검정 (estimation & testing for the means)

3 * motivation 비교하고싶은그룹이두개이면 (comparisons of two groups) -> t-test 비교하고싶은그룹이두개이상이면 (more than two groups) -> 두개그룹씩뽑아서쌍을만든후에여러개의 t-test 를실시한다. (pairwise t-tests) 번거롭기도하고이론적으로틀린결론에도달할수있다. (cumbersome & theoretically wrong -> 다중비교의문제 (multiple-comparisons problems) 전체자료를사용하지않고자료의부분만을사용하므로효율이떨어진다. (efficiency problems due to the usage of partial data) 전체자료를이용하여서세그룹이상을비교하는분석 (more than 3 groups using whole data) -> 분산분석 ANOVA ( 종속변수는연속형, 독립변수는이산형 ) response var: conti, explanatory var: categorical

4 <Ex 8.1.1> 혈청콜레스테롤을낮추는세가지약 A, B, C 를비교. 실험에참여한사람들에게 A, B, C 중하나를처방한뒤에, 혈청콜레스테롤이줄어들었는지측정. 처방받은약에따라그룹을나눴을때, 각그룹별표본평균에변동과각그룹내의관측값의변동을생각. 이때전자는그룹간변동 (between group variation), 후자는그룹내변동 (within group variation). 그룹간변동은처방약의효과, 그룹내변동은식습관의차이, 유전적차이등다양한이유로인하여발생. 따라서그룹간변동이그룹내변동에비하여상대적으로작으면, 약의효과는동일하다고결론. Response variable: reduction of cholesterol level Explanatory variable: drugs A, B, C Within group variation: due to other factors like genetic or nutrition, etc Between group variation: drug effect

5 Cholesterol level within group variation Cholesterol level A B C Treatments between group variation A B C Between group variation: small Within group variation: large

6 8.2 일원배치분산분석 (One-way ANOVA) [ 완전확률화계획법 ] : (complete randomization) 처리변수 (treatment) k x 11 x 12 x 13 x 1k x 21 x 22 x 23 (treatments) x 2k x 31 x 32 x 33 x 3k x n1 1 x n2 2 x n3 3 x nk k 합 (total) T.1 T.2 T.3 T.k T.. (total) 평균 (average) x.1 x.2 x.3 x.k x..

7 모형 (model) x ij j ij ij번째측정치 j처리의평균 ij번째오차 ij-th observation mean of j-th treatment group error of ij-th observation j ij k j j 1 k j j : 전체평균 Grand mean : j번째처리효과 Effect of j-th treatment group

8 x ij = μ j + ε ij = μ + τ j + ε ij 처리변수 (treatment) k x 11 x 12 x 13 x 1k x 21 x 22 x 23 x 2k x 31 x 32 x 33 x 3k (treatments) x n1 1 x n2 2 x n3 3 x nk k 합 (total) T.1 T.2 T.3 T.k T.. 평균 (average) x.1 x.2 x.3 x.k x.. pop mean μ 1 μ 2 μ 3 μ k μ effect τ 1 = μ 1 μ τ k = μ k μ

9 모형의가정 (Assumptions of the model) (a) 독립확률표본 (independent random sample) (b) x i,j ~N(μ j, σ 2 j ), i = 1,2,, n j k번째 -> j번째 (c) σ 1 2 = σ 2 2 = = σ k 2 = σ 2 (d) μ j 의평균은 μ. 따라서 τ j = μ j μ 이라고하면, τ j = 0 (e) x ij 의평균은 μ j 이고 ε ij 의평균은 0 이다. (f) ε ij 와 x ij 의차이는상수, ε ij 의분산은 x ij 의분산과동일. 따라서오차항의분산은 σ 2 이다. (g) ε ij 는독립이며정규분포를따른다. (indep. & normally dist) => x ij = μ j + ε ij = μ + τ j + ε ij, ε ij ~N(0, σ 2 ) independent

H 0 τ j = 0, j = 1,2,, k vs H A 적어도하나이상의 τ j 은 0 이아니다. (More than one τ j s are not equal to 0.

10 모형의가설 (Hypothesis of the model) H 0 μ 1 = μ 2 = = μ k vs H A 적어도하나이상의 μ j 는다르다. 만약영가설이사실이면 ( 즉, 모평균이서로같다면 ) 처리효과는모두 0 이므로영가설과대립가설은다음과같이표현. H 0 τ j = 0, j = 1,2,, k vs H A 적어도하나이상의 τ j 은 0 이아니다. (More than one τ j s are not equal to 0.) 영가설이사실이고가정이만족될때, 모집단의분포 등분산과정규성의가정이만족되나영가설이사실이아닌경우모집단의분포 Under H 0 with normal and homogeneity assumptions Under H A with normal and homogeneity assumptions

11 총제곱합 (sum of squares, total) k SST ( x x ) k j 1 i 1 n j ( x x x x ) j 1 i 1 n j ij.. 2 ij. j. j.. 2 k n j k n j k n j 2 2 ( xij x. j ) 2 ( xij x. j )( x. j x.. ) ( x. j x.. ) j 1 i 1 j 1 i 1 j 1 i 1 k n j k 2 2 ( xij x. ) j n j ( x. j x.. ) j 1 i 1 j 1 Within-group SS Among(Between)-group SS

12 Within-group SS SST SSW SSA within among group 그룹내제곱합그룹간제곱합 그룹간평균제곱분산비 = 그룹내평균제곱 MSA variance ratio= MSW Among(Between)-group SS -> 분산비가커지면그룹간의 variation 이크다. 그룹간의성질이다르다. 그룹의효과가크다. ->larger VR -> larger between-group SS 0 -> groups are different -> bigger group effect!

13 σ 2 의불편추정량 MSW k n j j 1 i 1 k j 1 ( x x ) ij ( n 1) j j 2

14 ANOVA Table factor Sum of squares Degree of freedom Mean square Variance ratio Between group Within group total 요인제곱합자유도평균제곱합 F 집단간제곱합 SSA = j=1 집단내제곱합 SSW = j=1 총제곱합 k k k SST = j=1 n j n j i=1 n j i=1 x.j x ij x ij x.. 2 x.j 2 x.. 2 k 1 MSA = SSA/(k 1) N k N 1 MSW = SSW/(N k) MSA MSW

15 <Ex 8.2.3> 소의연령에따른육류의셀레늄 (selenium) 농도비교 (Age group of cows and selenium concentration of milk) (1) 데이터 표 연령에따른셀레늄함유량 (mg=100g) 나이그룹 A B C D

17 요인 제곱합 자유도 평균제곱합 F 집단간제곱 집단내제곱합 총제곱합 Critical value = 3.95, alpha=0.01 > qf(0.99,3,109) [1] > 1-pf(9.353,3,109) [1] e-05 >

18 SAS program data cele; input value group cards; 1820 A 1483 A 2588 A 1723 A 2670 A 727 A 1022 A 1463 A 1555 A 1777 A 222 A 1129 A 1197 A 1249 A 1520 A 489 A 2575 A 1426 A 1846 A 1088 A 912 A 1383 A 191 B 1098 B 644 B 136 B 1605 B 1247 B 1529 B 1422 B 445 B 990 B 489 B 2408 B 1064 B 629 B 724 C 1020 C 613 C 805 C 918 C 631 C 949 C 641 C 877 B 760 C 1368 C 1085 C 1692 C 775 C 697 C 1307 C 849 C 344 C 1199 C 961 C 429 C 239 C 798 C 944 C 631 C 1096 C 1016 C 1025 C 948 C 1652 D 775 D 752 D 1309 D 1393 D 804 D 1002 D 533 D 1182 D 966 D 734 D 1243 D 788 D 485 D 985 D 472 D 449 D 1295 D 471 D 236 D 1676 D 771 D 831 D 754 D 869 D 698 D 937 D 513 D 167 D 1022 D 731 D 824 D 1073 D 1130 D 448 D 948 D 1034 D 991 D 222 D 1261 D 590 D 721 D 42 D 994 D 375 D 767 D 1781 D 1187 D proc anova data=cele; class group; model value=group; means group/tukey; * means group/bon; run;

20 * Multiple Comparisons ( 다중비교 ) ex) significance level = for a test Let H : 0 p( do not reject H H is true) H : 0 p( do not reject H H is true) then p( do not reject H H ) where H H and H p( do not reject H01 and do not reject H02 H0 (1- ) (1- ) (1- ) In general, if we want to test, then k (1 ) (1 ) k (.95).95 overall is , not > inflated type I error!! )

21 * Bonferroni Correction : Set individual significance the overall significance level is about for m multiple tests. m m= example) When we have 10 hypotheses, Individual p=0.05 -> multiple comparisons problem (too many false findings) Individual p= This is often called Bonferroni corrected p-value.

22 [ 처리그룹쌍별두모평균차이의검정 ] Detecting pairwise differences After rejecting H0 : have larger differences?, which pairs 1. LSD (least significant difference, 최소유의차검정법 ) 2. Duncan s new multiple range test Duncan 의새로운다중범위검정법 3. Tukey s HSD Liberal Conservative Duncan LSD SNK Tukey HSD Scheffe

23 [Tukey 의 HSD (honestly significance difference) 검정 ] MSE HSD = q n, k, N k j n j MSE HSD q n * *, k, N k * j n j 's are the same : sample size of smaller cell ymax ymin q, k, N k : dist of, S 2/ n : significance level, k : number of gropus, N k: df [Bonferroni 방법 ] 계산된 p-value에가능한모든방법의수 k 2 를곱함. Bonferroni corrected p-value: multiply # of all possible k methods to the p-value 2

24 데이터의각처리그룹별표본평균의차이 Differences of means between pair of groups A B C D A B C D - α=0.05. k = 4, N k = 109 > qtukey(0.05, nmeans= 4, df=109, lower=f)= 3.689, MSE=

25 Tukey 의 HSD 결과 개별영가설 HSD* 검정결과 H 0 : μ A = μ B HSD = = > 이므로 H 0 을기각함. H 0 : μ A = μ C HSD = = > 이므로 H 0 을기각함. H 0 : μ A = μ D HSD = = > 이므로 H 0 을기각함. H 0 : μ B = μ C HSD = = < 이므로 H 0 을기각하지못함. H 0 : μ B = μ D HSD = = < 이므로 H 0 을기각하지못함. H 0 : μ C = μ D HSD = = < 이므로 H 0 을기각하지못함.

28 8.3 완전확률화완전블록계획법과이원배치분산분석 (Randomized complete block design & Two-way ANOVA) R.A.Fisher (1925) : to compare the yields of certain species 땅을블록 (block=land) 으로나누고블록안에서 Randomize (other factors) in a block 하는것이다. 처리 블록 k 합 total 1 x 11 x 12 x 13 x 1k T 1. 2 x 21 x 22 x 23 x 2k T 2. 평균 average x 1. x 2. 3 x 31 x 32 x 33 x 3k T 3. x 3. n x n1 x n2 x n3 x nk T n. 합 Total T.1 T.2 T.3 T.k T.. 평균 Average x.1 x.2 x.3 x.k x n. x..

29 모형 x ij = μ + β i + τ j + ε ij i = 1,2,, n; j = 1,2,, k x ij : 각실험단위로부터얻은관측값 (Observation) μ: 미지의상수로서전체평균. (grand mean) β i : i 번째블록의블록효과 block effect for i-th obs τ j : j 번째처리의처리효과. trt effect for j-th obs ε ij : 그외효과들의총합. random error 모형의가정 a x ij : 랜덤독립표본 (random and independent sample) (b) x ij ~independent N μ ij, σ 2 ε ij ~independent N 0, σ 2 k (c) j=1 τ j = n i=1 β i = 0 블록효과와처리효과는가법적이다. Block effect & trt effect are additive.

30 가설 (hypothesis) H : 0 j 1,2,, k H 0 j :All 0 is not true. Some 0. A j j * k SST ( x x ) n j 1 i 1 ij.. 2 k n k n j k n ( xi. x.. ) ( x. j x.. ) ( xij xi. x. j x.. ) j 1 i 1 j 1 i 1 j 1 i 1 SST SSBl SSTr SSE df : nk 1 ( n 1) ( k 1) ( n 1)( k 1)

31 ANOVA table 요인 (factor) 제곱합 (sum of square) 자유도 (df) 평균제곱 (mean square) F 처리 (trt) SSTr (k 1) MSTr = SSTr/(k 1) MSTr MSE 블록 (block) SSBl (n 1) MSBl = SSBl/(n 1) 잔차 (residual) SSE (n 1)(k 1) MSE = SSE/(n 1)(k 1) 합 (sum) SST kn 1

32 Ex 약의종류와나이에따라치료까지걸린시간 Trt time by drug and age group 약의종류 (drug) 나이그룹 (age group) A B C 합 total 평균 Age average < 20 11* 합 total 평균 average

33 response<-c(11, 6, 7, 9, 10, 8, 5, 10, 12, 17, 10, 11, 13, 13, 15) drug<-factor(c(rep('a',5),rep('b',5),rep('c',5))) age<-factor(rep(1:5)) dat<-data.frame(response=response,drug=drug,age=age) anova(lm(response~drug+age,data=dat)) Analysis of Variance Table Response: response Df Sum Sq Mean Sq F value Pr(>F) drug age Residuals Signif. codes: 0 *** ** 0.01 *

35 reduction of response time 8.4 요인실험과이원배치분산분석 (Factorial Experiment and two-way ANOVA) 반응시간 (reduction of response time ) = 약품수준 ( 소량, 중간, 다량 )* 연령층 ( 중년, 노년 ) drug level (min, med, max)*age(mid, old) 교호작용이없을때 (Without interaction) 요인 B 약품용량 (Factor-B, drug level) 요인 A 연령 Factor A-age j=1 j=2 j=3 중년층 (Mid) i= 노년층 (old) i= age Drug level Drug dosage age

36 교호작용이있을때 (With interaction) 요인B 약품용량 요인A j=1 j=2 j=3 j=2-1 j=3-2 - 연령 중년층 (i=1) 노년층 (i=2)

37 요인 (factor) A 1 요인 (factor) B 1 2 b 합 total 평균 average x 111 x 121 x 1b1 x 11n x 12n x 1bn T 1.. x x 211 x 221 x 2b1 x 21n x 22n x 2bn T 2.. x 2.. a x a11 x a21 x ab1 x a1n x a2n x abn T a.. x a.. 합 total 평균 average T.1. T.2. T.b. T... x.1. x.2. x.b. x...

38 모형 x ijk = μ + α i + β j + (αβ) ij +ε ijk, i = 1, 2,, a; j = 1, 2,, b; k = 1, 2,, n x ijk : 관측값 (observation) μ: 전체평균 (grand mean), α i : 요인 A의효과 (effect of factor A), β j : 요인 B의효과 (effect of factor B), (αβ) ij : 요인 A와요인 B의교호작용 (interaction), ε ijk : 실험오차 (random error).

39 모형의가정 (Assumptions of the model) i. 각칸의관측값들은두요인의수준들의특정조합으로정의된모집단에서뽑은 n 개의독립표본으로구성되어있다. (independent sample) ii. ab 개의모집단은각각정규분포를따른다. Normal distribution iii. 모든모집단은동일한분산을가진다. Same variances x ijk = μ + α i + β j + (αβ) ij +ε ijk, ε ijk ~iid N(0, σ 2 ), i = 1, 2,, a; j = 1, 2,, b; k = 1, 2,, n

40 Hypotheses( 가설 ) H : 0 i 1,, a 0 H : Not H 0 for some i. A 0 0 i 0 0 H : 0 j 1,, b j H : Not H 0 for some j. A 0 i j H :( ) 0 i 1,, a j 1,, b ij H :Not H ( ) 0 for some i, j. A ij SST=SSA+SSB+SSAB+SSE

41 이요인완전확률화설계의분산분석표 ( 고정효과모형 ) ANOVA table for two-way complete randomized design 요인 factor 제곱합 SS 자유도 df 평균제곱 MS F A SSA a 1 MSA = SSA/(a 1) B SSB b 1 MSB = SSB/(b 1) AB SSAB (a 1)(b 1) MSAB = SSAB/(a 1)(b 1) MSA MSE MSB MSE MSAB MSE 처리 trt SSTr ab 1 잔차 residual SSE ab(n 1) MSE = SSE/ab(n 1) SST = SSTr + SSE SSTr = SSA + SSB + SSAB

42 <Ex > 간호사의가정방문시간 (time of staying home for a nurse) = 간호사의연령, 환자의질환 (age of the nurse, disease of the patient) 모형 (Model) x ijk i j ( ) ij ijk i 1,, a j 1,, b k 1,, n

44 * miscellaneous ( 기타 ) Log transformation: when normal assumption is violated. Normality is still problematic even after the variable transformation. Sample size is too small to check normality -> Nonparametric approach e.g. income, concentration

45 Type of Sum of Squares * Type Ⅰ:sequential (if we know the relative importance of the variables) Type Ⅱ: partial without interaction terms **TypeⅢ:partial with interactions (If we don t know the relative importance of the variables) TypeⅣ: There are missing cells (if none, same as TypeⅢ) *, ** : defaults One way ANOVA model : Y Ai ij

Chapter 7 분산분석

Chapter 7 분산분석 Chapter 8 실험계획및분산분석 (ANalysis Of VAariance, ANOVA) Updated 2018/4/30 7.1 머리말 (Introduction) 분산분석 (analysis of variance) : 전체변동을몇개의성분으로분할하는기법 (Divide total variation into several components) 전체변동에대해각각의변동요인의기여규모를파악