Notes to supplement Handout 11

Introductory Applied Econometrics
EEP/IAS 118
Spring 2015
1
Steven Buck
Discussion Guide for Handout #11
Introducing the Concept of Statistical Inference
Statistical inference is a process which helps us draw conclusions about parameters of interest in a population
(e.g. income per capita in Nicaragua) based on probability calculations. To understand what is meant by
this let’s re-visit our example from Handout #11.
Example. Suppose I think the average income per capita in 1998 for Nicaragua is $5,000 cordobas. Some
researchers collect a random sample with 2234 people (n=2234) in Nicaragua and have them complete a
survey, which asks about annual income. We use the data they collect to calculate the following:
. sum i_income urban indigenous
Variable |
Obs
Mean
Std. Dev.
Min
Max
-------------+-------------------------------------------------------i_income |
2234
5369.052
7533.916
5.409909
168552.9
For now we will focus on i income, which is the variable income. The Stata sum command outputs:
ˆ Sample mean: y¯ = 5369 and Sample variance: s2 = σ
ˆy2 = 75342
We can use this information to estimate an interval for which we have a level of C confidence that the
interval contains the true population mean µy . For example, if C = 95%, then this means that there is a 0.95
probability that the estimated confidence interval contains µy . We would not say that there is an estimated
0.95 probability that µy = 5369. There is no uncertainty in µy . The uncertainty relates to the
confidence interval, which varies with each random sample. I marked this last sentence in bold so
read it again.
Once again, the formula to estimate a C% confidence interval is:
s
s
[¯
y−σ
ˆy¯ · c, y¯ + σ
ˆy¯ · c] = [¯
y − √ · c, y¯ + √ · c]
n
n
where c is the critical [t] value associated with the Student’s t distribution, which defines the distance |c|
from zero in which C% of the Student’s t distribution lies. We use the Student’s t-distribution instead of
the standard Normal distribution because we do not observe σy2¯ (= var(¯
y )) and instead estimate σ
ˆy2¯ for
estimating the confidence interval.
We calculate a few confidence intervals (for n=2234, we know from reading a Student’s t-table that c0.90, n→∞ =
1.645, c0.95, n→∞ = 1.96, c0.99, n→∞ = 2.575):
ˆ σ
ˆy2¯ =
σ
ˆy2
n
=
75342
2234
= 159.42
ˆ 90% CI: [5369 − 159 · 1.645, 5369 + 159 · 1.645]
ˆ 95% CI: [5057, 5680]
ˆ 99% CI: [4960, 5777]
We observe that our estimated 95% CI does not contain 5000. What does this mean? When we interpret our
95% confidence interval, we specify that there’s a 95% chance that the interval covers the true value. There
is only a 5% chance that it does not contain the true value–so after estimating a 95% confidence interval,
you should be skeptical to hear that µ is equal to something that lies outside your estimated interval.
In fact, the empirical evidence might make you think that µy 6= 5000. Said differently, you’ve used data
and some assumptions to generate a probability statement that leads you to infer that the average income
per capita in Nicaragua is something different than 5000. This last bit of logic is the essence of hypothesis
testing so read it again!
While confidence intervals and hypothesis testing are closely linked, we can distinguish there utility in
statistical inference as follows:
ˆ Confidence intervals are appropriate when our goal is to estimate a population parameter and we want
to qualify our estimate with a probability statement about proximity to the true mean.
ˆ Hypothesis testing (aka, tests of significance) are appropriate when our goal is to assess the evidence
provided by the data in favor of some claim about the population. So when you think of hypothesis
test, think of a horse race between some base belief (the null hypothesis) and some alternative belief
(the alternative hypothesis).
2
Hypothesis Testing about the mean of a population
For our example from households in Nicaragua, is there a high probability that a random sample would yield
an y¯ = 5000? Based on the confidence intervals we just calculated, the evidence suggests not. But a more
conventional way to address this specific question is to conduct a test of significance (hypothesis testing),
which is the subject of Handout #11.
Let’s walk through the two-sided hypothesis test at the bottom of the first page of Handout #11.
Step 1: Define hypotheses
H0 : µy = 5000
H1 (Ha ): µy 6= 5000
ˆ If we don’t have a good reason to think the average income per capita should be higher or lower than
5000, our alternative hypothesis will result in a two-sided test.
ˆ Note: The null hypothesis is presumed correct until we find statistical evidence against it.
Step 2: Compute test statistic
σy2¯), and the sample size n:
To compute the t-stat, we need the sample mean y¯, the sample variance s2 (ˆ
t2233 =
x
¯−µ
σ
ˆy¯
=
5369−5000
159.4
= 2.31
You can think of the t-stat as the sample mean (i.e. y¯ = 5369) converted to different units, where the units
are measured in terms of standard deviations from the null hypothesis (the null hypothesis is naturally zero
standard deviations from itself!).
2
Step 3: Choose a significance level (α) and find the critical value (cα )
ˆ We are performing a two-sided test and we have more than 120 degrees of freedom, which we need to
know to identify the correct cα . From a Student’s t table we can find the following critical values for
two-sided tests:
ˆ cα=0.10 n→∞ = 1.645
cα=0.05 n→∞ = 1.96
cα=0.01 n→∞ = 2.58
The significance level α is the probability of finding a value as far or farther from the true value as c. In
other words, if we are drawing from a standardized Student’s t distribution with more than 120 degrees of
freedom, then there is a 10% chance of drawing a random t-stat whose magnitude/size (|t-stat|) is greater
than 1.645; there is a 5% chance of drawing a random t-stat whose magnitude/size (|t-stat|) is greater than
1.96; and there is a 1% chance of drawing a random t-stat whose magnitude/size (|t-stat|) is greater than
2.58.
Step 4: Reject the null hypothesis or fail to reject it
ˆ If |t| > |cα |, then we reject the null hypothesis.
ˆ If |t| < |cα |, then we fail to reject the null hypothesis.
ˆ Note: We never really “accept” the alternative or null hypothesis, we simply do or do not find evidence
against the null.
In our example at what levels α do we reject and fail to reject the null hypothesis?
ˆ We reject H0 at the 10% significance level because t = 2.31 > 1.645.
ˆ We reject H0 at the 5% significance level because t = 2.31 > 196.
ˆ We fail to reject H0 at the 1% significance level because t = 2.31 < 2.58.
Step 5: Interpretation/Conclusion
ˆ We reject the null hypothesis using a two-sided test of significance with α = 0.05: There is statistical
evidence at the 5% level that the average income per capita in Nicaragua is different from 5000
cordobas. Based on a 5% significance test, it’s unlikely that the original assertion that µy = 5000 is
correct.
ˆ We fail to reject the null hypothesis using a two-sided test of significance with α = 0.01: There is no
statistical evidence at the 1% level that the average income per capita in Nicaragua is different from
5000 cordobas. The scientific community would say that we don’t have very strong evidence against
the original assertion. However, this doesn’t mean that H0 is correct, there are many other values of
µy for which we would fail to reject H0 .
Comment on interpreting the significance level:
ˆ We can also think of the significance level as the probability of rejecting the null hypothesis when, in
fact, it is true–this is called a Type I Error.
ˆ A Type II error is when you fail to reject the null hypothesis when, in fact, you should reject the
null hypothesis.
3