UNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences

UNIVERSITY OF TORONTO SCARBOROUGH
Department of Computer and Mathematical Sciences
Midterm Test, Fall - 2014
STAC67H3: Regression Analysis
Duration: One hour and fifty minutes (110 minutes)
FIRST NAME:
LAST NAME:
STUDENT NUMBER:
SIGNATURE:
Aids Allowed:
• A handwritten cheat-sheet, covering both sides of an A4/letter sized paper. You need
to submit your cheat-sheet with your answer-sheet after the test.
• A calculator (No phone calculators are allowed)
No other aids are allowed.
All your work must be presented clearly in order to get credit. Answer alone (even though
correct) will only qualify for ZERO credit. Please show your work in the space provided; you
may use the back of the pages, if necessary, but you MUST remain organized. Show your
work and answer in the space provided, in ink. Pencil may be used, but then any re-grading
will NOT be allowed.
There are 11 pages including this page. Please check to see you have all the
pages.
Good Luck!
Question 1
2
3
4
Total
Points
31
15
15
123
62
Score
1
1. In a certain Federal Power Commission hearing some years ago, witnesses presented the
following data (here rounded). The variables are: (i) predictor X: percentage of liquids
i:
1
2
3
4
5
6
7
Xi :
13.3
16.9
19.9
23.2
26.3
30.1 42.6
Yi :
3.5
5.1
4.8
6.7
6.0
9.5
8.1
in gas production output, and (ii) response Y : unit cost in cents of processing output.
Assume that the following normal error regression function Yi = β0 + β1 Xi + i , i =
1, 2, · · · , 7 is appropriate. [62 Points]
(a) Write down the assumptions of your regression model. [4 Points]
Answer: The assumptions of the normal error regression models are:
• The mean of the error term is zero
E(i ) = 0.
• The variance of the error term is σ 2 , a fixed constant
Var(i ) = σ 2 .
• The errors are independent to each other
Cov(i , j ) = 0 ∀ i 6= j.
• The errors are normally distributed, in short
i ∼ N (0, σ 2 ).
(b) Compute [5 Points]
7
X
Xi = 172.3
i=1
7
X
Yi = 43.7
i=1
7
X
Xi2 = 4809.21
i=1
7
X
Yi2 = 298.05
i=1
7
X
Xi Yi = 1172.51
i=1
2
(c) Obtain the least squares estimate of β1 . Interpret the result. [5 + 3 = 8 Points]
Answer: The least squares estimate of β1 is
P
¯ Y¯
Xi Yi − nX
96.86571
= 0.1704876
b1 = P 2
=
2
¯
568.1686
Xi − nX
Interpretation: With an 1% percent increase of liquids in gas production output,
the average unit cost of processing output increases to 0.1704876 cents.
(d) Obtain the least squares estimate of β0 . Interpret the result. [5 + 3 = 8 Points]
Answer: The least squares estimate of β0 is
¯=
b0 = Y¯ − b1 X
43.7
172.3
− 0.1704876
= 2.046426
7
7
Interpretation: When the gas production output consists NO liquids, the average unit cost of processing output becomes 2.046426 cents.
(e) Obtain an unbiased estimate of the error variance σ 2 . [1 + 3 + 2 + 2 = 8 Points]
Answer: The total sum of squares is
7
7
X
X
2
¯
SST =
(Yi − Y ) =
Yi2 − nY¯ 2 = 25.23714
i=1
i=1
The regression sum of squares is
7
7
X
X
2
2
ˆ
¯
¯ 2 = 16.51441
SSR =
( Y i − Y ) = b1
(Xi − X)
i=1
i=1
The error sum of squares is
SSE = SST − SSR = 25.23714 − 16.51441 = 8.722737
Hence, the unbiased estimate of σ 2 is
M SE =
8.722737
SSE
=
= 1.744547
n−2
7−2
3
(f) Estimate the variances of b1 and b0 . [3 + 3 = 6 Points]
Answer: The estimated variance of b1 is
M SE
1.744547
=
= 0.003070475
2
¯
568.1686
i=1 (Xi − X)
s2 {b1 } = Pn
The estimated variance of b0 is
¯2
X
1
+ Pn
s {b0 } = M SE
¯ 2 = 2.109508
n
i=1 (Xi − X)
2
(g) Estimate the 95% confidence interval of β1 . Interpret the result. [3 + 2 = 5
Points] Answer: The 95% confidence interval of β1 is
b1 ± t(1 − 0.05/2; n − 2) × s{b1 } = (0.0280469, 0.3129284)
Interpretation: If we keep drawing samples of size fixed size n, approximately
95% of them will provide b1 in the interval (0.0280469, 0.3129284).
(h) Estimate the 95% confidence interval of β0 . Interpret the result. [3 + 2 = 5
Points]
Answer: The 95% confidence interval of β0 is
b0 ± t(1 − 0.05/2; n − 2) × s{b0 } = (−1.687125, 5.779977)
Interpretation: Assumption: the scope of the regression model includes 0. If
we keep drawing samples of fixed size n, approximately 95% of them will provide
b0 in the interval (−1.687125, 5.779977).
4
(i) Test the null hypothesis H0 : β0 = 0 against the alternative hypothesis that
HA : β0 6= 0. Specify the level of significance α at 0.05. [3 Points]
Answer: In (h) we see that the 95% confidence interval of β0 includes 0 in it.
Hence, the null hypothesis is accepted at the 5% level of significance. We write:
there is no enough evidence to reject the null hypothesis.
(j) Obtain the 95% Bonferroni joint confidence intervals of β0 and β1 . Compare your
results with (g) and (h). [3 + 2 = 5 Points]
Answer: The 95% Bonferroni joint confidence intervals of β0 and β1 are
b0 ± t(1 − 0.05/4; n − 2) × s{b0 } = (−2.548116, 6.640967)
b1 ± t(1 − 0.05/4; n − 2) × s{b1 } = (−0.004801236, 0.345776500)
Comparison: The confidence intervals obtained in (j) are wider than the confidence intervals obtained in (g) and (h).
(k) Compute the coefficient of determination. Interpret the number. [3 + 2 = 5
Points]
Answer: The estimated coefficient of determination for this regression problem
is
R2 =
SSR
16.51441
=
= 0.6543691
SST
25.23714
Interpretation: The fitted regression model explains approximately 65.4% variability of the total variation in the response Y .
5
2. In a manufacturing study, the production times for 111 recent production runs were
obtained. The table below lists for each run the production time in hours (Y ) and the
production lot size (X).
i:
1
2
3
···
109
110
111
Xi :
15
9
7
···
12
9
15
Yi :
14.28
8.80
12.49
···
16.37
11.45
15.78
Consider that the first order linear regression model (Yi = β0 +β1 Xi +i ; i = 1, 2, · · · n)
is appropriate. We further assume that the model error terms are independent with
i ∼ N (0, σ 2 ). [31 Points]
(a) We compute the following sum of products of X and Y for this data
P111
i=1 (Yi
−
¯ = 2608.704. Furthermore, the estimated least squares regression
Y¯ )(Xi − X)
coefficients are b0 = 6.863487 and b1 = 0.5332749. Complete the following analysis
of variance (ANOVA) table. [16 Points]
Answer:
Table 1: ANOVA Table for Simple Linear Regression.
Source of
Variation
df
MS
F
Regression
SS
P ˆ
(Yi − Y¯ )2 = 1391.156
1
1391.156
310.1974
Error
P
(Yi − Yˆi )2 = 488.8373
109
4.484746
Total
P
(Yi − Y¯ )2 = 1979.99
110
(b) Perform the test that the production lot size has no effect on production time.
That is, test the null hypothesis that H0 : β1 = 0 against the alternative that
HA : β1 6= 0. Use the level of significance α = 0.01. [5 Points]
Answer: Here, the test statistic is F = 310.1974 which follows, under H0 , an
F distribution with numerator and denominator degrees of freedom 1 and 109,
respectively.
The tabulated value of F is F (0.99; 1, 109) = 6.87325. Since F = 310.1974 ≥
F (0.99; 1, 109) = 6.87325, we reject the null hypothesis at α = 0.01 level of
significance.
6
(c) Consider, from a residual plot you think the constant variance assumption of the
fitted model might be violated. Hence, you want to perform a statistical test to
check this assumption. For this purpose you fit the following model loge σi2 =
γ0 + γ1 Xi using e2i against Xi and obtain regression sum of squares as 4.90996.
Perform an appropriate test with level of significance α = 0.01. [10 Points]
Answer: Here, residual sum of squares of the first model is SSE = 488.837. The
regression sum of squares for the second model is SSR∗ = 4.90996. Hence, the
Breusch-Pagan test statistic is
χ2BP
SSR∗
=
/
2
SSE
n
2
= 0.1265802
Under H0 and when n is large, χ2BP follows χ2 distribution with 1 degrees of
freedom. At α = 0.01 level of significance χ2 (0.99; 1) = 6.634897. Since χ2BP =
0.1265802 < χ2 (0.99; 1) = 6.634897, we conclude H0 , that the error variance is
constant.
7
3. Consider the regression model Yi = β0 + i , i = 1, 2, · · · , n. Assume that the error
term i are independent N (0, σ 2 ). Construct the likelihood function, and obtain the
maximum likelihood estimator of β0 . [10 + 5 = 15 Points]
Answer:
From the stated assumptions, we write
Yi ∼ N (β0 , σ 2 )
Hence,
1
1
2
f (Yi |β0 , σ ) = √
exp − 2 (Yi − β0 )
2σ
2πσ
2
From the independence assumption, we write our likelihood function as following
L(β0 , σ 2 |Y1 , · · · , Yn ) = f (Y1 , · · · , Yn |β0 , σ 2 )
n
Y
=
f (Yi |β0 , σ 2 )
i=1
(
n
1 X
−n/2
2 −n/2
(Yi − β0 )2
= (2π)
(σ )
exp − 2
2σ i=1
)
The log likelihood function is written as
n
n
1
log L(β0 , σ 2 |Y1 , · · · , Yn ) = − log(2π) − log(σ 2 ) − 2
2
2
2σ
( n
X
)
(Yi − β0 )2
i=1
Differentiating this log likelihood function with respect to β0 and equating to 0, we get
n
1 X
(Yi − β0 ) = 0
σ 2 i=1
Solving this equation for β0 , we get the maximum likelihood estimator of β0
Pn
Yi
ˆ
β0 = i=1 = Y¯ .
n
8
4. The height of soap suds in the dishpan is of importance to soap manufacturers. An
experiment was performed by varying the amount of soap and measuring the height
of the suds in a standard dishpan after a given amount of agitation. The data are as
follows:
Grams of Product,
Suds Height,
X
Y
4.0
33
4.5
42
5.0
45
5.5
51
6.0
53
6.5
61
7.0
62
Assume that a simple linear regression model of the form Y = Xβ + is reasonable.
[15 Points]
(a) Write down the vectors Y, β, and the matrix X. [4 Points]
Answer: The response vector is:
 
33
 
 
42
 
 
45
 
 
Y = 51
7×1
 
 
53
 
 
61
 
62
The vector of regression coefficients is
 
β0
β = 
2×1
β1
The error vector is
 
 1
 
2 
 
 
3 
 
 
= 4 
7×1
 
 
5 
 
 
6 
 
7
9
The regression matrix is

1


1


1


X = 1
7×2


1


1

1

4.0


4.5


5.0


5.5


6.0


6.5

7.0
(b) Obtain the matrices XT X and XT Y. [6 Points]
The matrix XT X is


7.00 38.50

XT X = 
2×2
38.50 218.75
The matrix XT Y is


347

XT Y = 
2×1
1975
(c) Find the least squares estimate of β. [5 Points]
Answer: The least squares estimate of β is

b = XT X
2×1
−1
2×2
10

−2.678571

XT Y = 
2×1
9.500000
Appendix
1. When α = 0.05 and n = 7
t(1 − α/2; n − 2) = 2.570582.
2. When α = 0.05 and n = 7
t(1 − α/4; n − 2) = 3.163381.
3. When α = 0.01 and n = 111
F (1 − α; 1, n − 2) = 6.87325.
4. When α = 0.01 and n = 111
t(1 − α/2; n − 2) = 2.621688.
5. When α = 0.01 and n = 1
χ2 (1 − α; n) = 6.634897.
11