Microsoft Word - multiple

Size: px

Start display at page:

Download "Microsoft Word - multiple"

보검 도
5 years ago
Views:

1 Chapter 3. Multiple Liear Regressio Data structure ad the model yi 0 1xi1 pxip i, i1,, (Y X ),,, : idepedet with E( ) 0 ad 1 : ukow 0, 1,, p, 0 1 i var( i ) X (1, x,, xp), rak( X) p1, X : give where xj ( x1 j,, xj) 1

2 Least squares estimates miimize ( yi 0 1xi1 pxip) w.r.t. 0,, p i1 ormal equatio : e ˆ ˆ ˆ i yi ( 0 1xi1 pxip) yi yˆ i (p.57) e 0 e 0 ˆ y ˆ x ˆ x xi1ei 0 ( xi1 x1) ei 0 S ˆ ˆ 111 S1 p p S y1 x 0 ( ) 0 ˆ ˆ ipe i xip xp e i S p11 S pp p S yp i i p p Sij ( xai xi )( xaj xj ) a1 where. Syj ( ya y)( xaj xj ) a1 least squares regressio fit: y ˆ ˆ ˆ 0 1x1 x ˆ p p estimate (ubiased) of : ˆ 1 1 y y SSE p1 p1 ( i ˆi) i1

ad i var( i ) Least square estimate: ˆ argmi( y X)( y X) ( y X )( y X) yy yx Xy XX

3 Matrix approach For y ( y1,, y) x ( x,, x ) ( j 1,, p), j 1 j j X (1, x,, x p ),, 1 ( 0, 1,, p ), ad ( 1,, ), Model: y X Assumptios : 1,, are idepedet with E( ) 0 ad i var( i ) Least square estimate: ˆ argmi( y X)( y X) ( y X )( y X) yy yx Xy XX ( Xy ) Xy ( y X)( y X) XX Xy Xy 0 Recall that ( cx ) ( xc ) c x x ( yay ) ( A A) y y 3

4 ( y X)( y X) XX Xy Xy 0 ˆ ( XX ) 1 Xy ( ) rak( X X ) rak( XX ) rak( X ) rak( X ) p 1 ˆ 1 1 yˆ X X( X X) X y Py ( P X( X X) X ) I case of p 1, ( x j)( yj) ( xj)( xjyj) 1 ˆ0 j y j ˆ x xj ( xj) ˆ x 1 j xj x j y j xjyj ( xj)( yj) xj ( xj) This is coicidet with the result of simple liear regressio. 4

5 Method of iferece (1) Properties of estimates Recall that E y X y I yˆ X ˆ X XX Xy P P X XX X 1 1 ( ), var( ) var( ), ( ) y ( ( ) ) E( ˆ ), var( ˆ ) ( X X) ˆ 1 1 E( ) E ( XX ) Xy ( XX ) XE( y) ˆ var( ) ( XX ) Xvar( y) X( XX ) ( XX ) 1 i. ii. E( ˆ ), E( e) 0, var( e) I P e y yˆ I P y E() e I PE( y) I PX X PX X X 0 X var( e) I Pvar( y) I P I PI I P( I P) ( I P) IP 5

6 () Iferece uder additioal ormality assumptio Let i. 1 C ( XX) ( c ij ) 0 i, j p ˆ i i se..( ˆ ) i ˆ 1/ ~ t ( p1); se..( i) cii ( i1,, p) Pr ˆ t ( p1; ) se..( ˆ ) 1 i i i ˆ Reject H v.s. 0 0 : i i H 0 1 : i i ˆ 0 i i iff t ( p 1, ) se..( ˆ ) i p-value: ii. ˆ 0 0 1/ ~ t ( p1); se..( 0) c00 ˆ 0 se..( ) ˆ ˆ Similar to ˆi 6

7 E( Y x ) x where x ( x, x,, x ) with x 1 ˆ / ~ t ( p1), se..( ˆ 0) ˆ x0( XX ) x0 se..( ˆ ) iii p 00 0 ˆ x ˆ var( ˆ ) x var( ˆ ) x C.I. ad Test iv. Predictio for y0 x0 0 ( 0 1,, ) yˆ ˆ 0 x0 ( ˆ 0) y yˆ se..( y ) / ~ t ( p1); se..( y0 y0) (1 x0( XX) x0) 0 yˆ 0 ˆ ˆ 7

8 Example (Supervisor Performace Data) data (=30, p=6) 8

9 scatter plot eed to be doe to see the validity of liearity assumptio model settig (eq. (3.3)): Y 0 1X1X 6X6 estimated LS fit (eq. (3.5)) LSE s with s.e. s (Table 3.5): idividually x 1 & x 3 the oly sigificat variables. PROC REG DATA=p054; MODEL y = x1 x x3 x4 x5 x6; RUN; 9

10 Aalysis of Variace Source DF Sum of Mea Squares Square F Value Pr > F Model <.0001 Error Corrected Total Root MSE R-Square Depedet Mea Adj R-Sq Coeff Var Parameter Estimates Variable DF Parameter Stadard Estimate Error t Value Pr > t Itercept X X X X X X

11 Measurig the quality of fit ( 3.7) i. Decompositio of sum of squares : ( yi y) ( yi yˆi) ( yˆi y) SST SSE SSR ( 1) ( p1) ( p) 1 1 Recall, for e y yˆ, e 0, x1e 0,, x e 0; i i i i i i ip i ( x x ) e 0,, ( x x ) e 0 i i ip p i ( y yˆ )( yˆ y) e ˆ ( x x ) ˆ ( x x ) 0 i i i i 1 i1 1 p ip p i1 i1 ii. Multiple correlatio coefficiet (MCC) & Adjusted MCC R SSR SSE 1 SST SST ; 0 R 1 R 1 meas that determiatio of y by liear combiatio of x becomes larger or 11

12 proportio of variatio of y explaied by x 1, x, p 설명변수의개수가증가하면항상 R ; SSE 가무조건감소! 제약 (costrait) 이있으면 최소값은커지고 ˆ ˆ ˆ i ˆi i 0 1 i1 p ip SSE ( y y ) ( y x x ) i1 i1 mi ( y x x ) ( 0, 1,, p ) i1 i 0 1 i1 p ip 최대값은작아진다. 1

13 mi ( y x x ) p1 p1 1, 0 1 i 0 1 i1 p ip ( SSE of Model : y x x ) mi ( y x x ) 1 i i 0 1 i1 p 0 1 i1 p ip i ( SSE of Model: y x x ) i 0 i1 p ip ip i SSE( reduced model) SSE( full model) R R ( reduced model) ( full model) R 는설명변수의개수가서로다른모형간의적합도비교에는부적절함. 따라서, 다음의 adjusted R 를고려함. SSE ( p 1) Ra 1 SST ( 1) 설명변수개수가서로다른모형의적합도비교에사용 13

14 Example (Supervisor Performace Data) Full model: y0 1x1 6x6 (SS) (df) SSR SSE SST R 0.73, R a Simpler (Reduced) model : y 0 1x13x3 (SS) (df) SSR SSE SST R 0.708, R a

15 Hypotheses testig i liear regressio model ( 3.9) i. Reduced Model ( 축소모형 ) v.s. Full Model ( 완전모형 ) H 0 : reduced model (RM) v.s. H 1: full model (FM) where RMFM (RM: (q +1) regressio parameters, FM:( p +1) regressio parameters q p r) (example) (FM) y 0 1x16x 6 ( i1,,30) i i i i (RM) (a) H0: y i 0 i v.s. H 1 : full model H0: v.s. H 1 : ot H 0 (b) H0: y0 0 1xi1 3xi3 i v.s. H 1 : full model H0: v.s. H 1 : ot H 1 15

16 ii. sums-of-squares i RM & FM df p1 FM FM SSE( FM ) ( yi yˆ i ), yˆ i : l.s.fit uder FM (# of parameters p 1) i1 RM RM SSE( RM ) ( yi yˆi ), yˆi : l.s.fit uder RM (# of parameters q 1) i1 df q1 df p SSR( FM ) SST SSE( FM ) SSR( RM ) SST SSE( RM ) df q, SST ( y y) SSE( FM ) SSE( RM ); SSR( FM ) SSR( RM ) i 0 1 i1 p ip p1 1 mi ( yi 0 1xi1 pxip) SSE( RM) restrictios 1 wrt.. SSE( FM ) mi ( y x x ) SSE( RM ) SSE( FM ): i1 i Degree of freedom ( df ) SSE : (# of parameters) SSR :(# of parameters) 1 SST : 1 reductio i residual s.s. by itroducig p SSR( FM ) SSR( RM ): added amout of explaatio due to the p 16 q more parameters (variables) to RM q more parameters (variables) to RM

17 iii. F-test statistic for RM vs FM : F SSR( FM ) SSR( RM ) #( FM ) #( RM ) SSE( FM ) #( FM ) ( Rp Rq) {( p1) ( q1)} (1 Rp ) ( p1) SSE( RM ) SSE( FM ) #( FM ) #( RM ) SSE( FM ) #( FM ) F ~ F( pq, p 1) uder H 0 Reject RM vs FM if F F( pq, p 1; ) p-value: p ESSRFM ( ) SSEFM ( ) ( i ), i p1 p1 i1 i1 17

18 Example: (Supervisor Performace Data) (FM) iid i 0 1 i1 6 i6 i, i ~ (0, ) y x x N H : 0 v.s. H 1 : ot H PROC REG DATA=p054; MODEL y = x1 x x3 x4 x5 x6; TEST x=x4=x5=x6=0; RUN; i.e., RM is yi 0 1xi13xi3 i. SSE(FM)=1149 (df=3) SSE(RM)= (df=7) F ( ) (6 ) 0.58; F F(4,3;0.05) ( or, F ( RFM RRM ) (6 ) ( ) (6 ) (1 RFM ) (30 7) ( ) (30 7) ) Do ot reject H 0 at 5% level! 18

19 iv. Iferece after adaptig a reduced model : - Test more reduced model v.s. the reduced model ew reduced model v.s. ew full model Example (Supervisor Performace Data) New full model: iid i 0 1 i13 i3 i, i ~ (0, ) y x x N PROC REG DATA=p054; MODEL y = x1 x3; RUN; 19

20 1 Sigificace of x 1 ad x 3 <Table 3.8> (FM) H1: yi 0 1xi1 3xi3 i v.s. (RM) H0: yi 0 i H0: v.s. H 1 : ot H 0 0

21 F ( SSE( RM ) SSE( FM )) / (3 1) ( SST SSE( FM )) / (3 1) SSE( FM )/( 3) SSE( FM )/( 3) ; highly sigificat SSR( FM ) 3.7 F(, 7;0.05) SSE( FM ) 7 <ANOVA Table> Source S.S d.f. Mea square F-test Regressio SSR p MSR = SSR / p F = MSR / MSE Residual SSE -p-1 MSE = SSE / (-p-1) Total SST -1 Sigificace of x 1 : H0: 1 0 v.s. H 1 : ot H 0 - (RM) H0: yi 0 3xi3 i v.s. (FM) H1: yi 0 1xi13xi3 i - Either F-test or t-test 1

22 - t-test : t ˆ se..( ˆ ) ; t(7;0.05) 와비교또는 p -value< H : v.s. H 1 : ot H H 0 : yi 0 1 ( xi1xi3) i v.s. H 1 : yi 0 1xi13xi3 i F ( RFM RRM ) ( 1) ( ) (1 RFM ) ( 3) ( ) 7 F F(1,7;0.05) 4.1 Do ot reject H 0 at 5% level PROC REG DATA=p054; MODEL y = x1 x3; TEST x1=x3; RUN;

23 Iterpretatios of regressio coefficiets ( 3.5) yi 0 1xi1 pxip i, i1,, i. 0(costat coef.) : the value of y whe x 1 x x p 0 ii. j (regressio coef.) : the chage of y correspodig to a uit chage i x ( j 1,, p ) j whe x i s (i j) are hold costat (fixed) iii. also called partial regressio coef. e.g.: Yˆ X X Yˆ X eyx Xˆ X ex X 1 1 eˆ e 3 YX 1 XX1 j ( j ): the cotributio of X j ( X ) to the respose variable Y after both variables have bee liearly adjusted for the other predictor variables ( X 1). 3

24 Graphical relatioship betwee full ad reduced model t Let 1 t P , ad what is a graphical meaig of 1 t χ I P x, χ t t of χχχχy PY? χ Let ( 1 ) t P χχχχ, X (1, x) t ( I1 11 1) x( IP 1 ) x? t Px ad t, ad PX t X XX X. What is a graphical meaig Fid a relatioship amog PYPY, χ, 1 ad PY X. ˆ 0 y y y1 x x x1 ˆ 1 y1 4 1

eda_ch7.doc

eda_ch7.doc ( ) (, ) (X, Y) Y Y = 1 88 + 0 16 X =0601 Y = a + bx + cx X (nonlinea) ( ) X Y X Y b(016) ( ) log Y = log a + b log X = e Y = b ax 71 X (explanatoy va :independent ), Y (dependent : esponse) X, Y Sehyug