TWO-SAMPLE T TEST VARIANCE TEST INTRO TO JMP Two-Sample t-Test

9/22/2010
TWO-SAMPLE T TEST
VARIANCE TEST
INTRO TO JMP
Dr. Yan Liu
Department of Biomedical, Industrial and Human Factors Engineering
Wright State University
Two-Sample t-Test
Used to determine whether two population means are equal
Variations
Paired samples vs. unpaired samples
In paired samples, there is a one-to-one correspondence between the values in the
two samples. That is, for two random samples X1 = {x11, x12 , …, x1n} and X2 = {x21,
x22 , …, x2n} x1i corresponds to x2i (i=1,2,…,n). In this case, the difference, x1i– x2i is
usually tested. If we can define Y = X1 – X2 , then it becomes a one-sample t-test for Y
in which µ0 = 0
In unpaired samples, there is no one-to-one correspondence between the values in the
two samples
Equal variances vs. unequal variances of samples
The variances of the two samples may be assumed to be equal or unequal
Equal variances yield a simpler formula than unequal variances
2
1
9/22/2010
Two-Sample t-Test (Cont’d)
Assumption
Null Hypothesis
X1 and X2 are two random samples of independent observations, each from an
underlying normal distribution N(µi,σi2), where i =1, 2
H0: µ1 = µ2
Alternative Hypothesis
H1: µ1 ≠ µ2, or
H1: µ1 < µ2 , or
H1: µ1 > µ2
3
Two-Sample t-Test (Cont’d)
The formula for the unpaired two-sample t-test is
X1 − X 2
t=
S12
df =
/ n1 + S 22 / n2
If t is “near” zero, then it is consistent with
(1) t ~ t ( df )
the null hypothesis; otherwise, it is not.
Standard error (SE)
( S12 / n1 + S 22 / n2 ) 2
( S12 / n1 ) 2 /( n1 −1) + ( S 22 / n2 ) 2 /( n2 −1)
(2)
S1 , S 2 are the sample standard deviations of X1 and X2 , respectively
n1 , n 2 are the sample sizes of X1 and X2 , respectively
If equal variances are assumed, then (1) reduces to
t=
X1 − X 2
S p 1 / n1 +1 / n2
(3), where
Sp =
( n1 −1) S12 + ( n2 −1) S 22
n1 + n2 − 2
If n1 = n2 = n, then S =
p
S12 + S 22
2
If equal variance are assumed, then (2) reduces to
df = n1 + n2 − 2 (4)
4
2
9/22/2010
Search Interface Example I. You want to compare the effectiveness of two search
interfaces – A vs. B. You have recruited 8 students for the experiment, each of
whom perform 5 predetermined search tasks on each of the two interfaces. The
following table shows the students’ task performance scores in the experiment
Interface
S1
S2
S3
Student
S4 S5
S6
S7
S8
A
5
4
2
3
2
1
5
4
B
4
3
2
2
2
1
3
3
This is a Paired Two-Sample Two-Sided T Test
5
Let X = (a student’s performance score in interface A) − (his performance score in
interface B ).
Student
Let µ1 denote the mean of X
S1 S2 S3 S4 S5 S6 S7 S8
H 0: µ 1 = µ 0 = 0
H1: µ1 ≠ µ0 = 0
Interface
A
B
X
5
4
1
4
3
1
2
2
0
3
2
1
2
2
0
1
1
0
5
3
2
4
3
1
Set α = 0.05, then the critical values are t0.025(7) = 2.365 and -t0.025(7) = -2.365
and the critical region is t ≥ 2.365 or t ≤ -2.365
X = 0.75, S X = S X
n
= 0.707
8
= 0.25
t = XS−Xµ0 = 0.075.25−0 = 3 > 2.365
So we can reject the null hypothesis that interfaces A and B have the same
effectiveness, i.e., there is evidence to support that interfaces A and B differ in
their effectiveness
We can also estimate the p-value using the t-distribution table, p ≈ 0.02
6
3
9/22/2010
Search Interface Example II. You want to compare the effectiveness of two
search interfaces – A vs. B. You have recruited 16 students for the experiment, each
of whom perform 5 predetermined search tasks on only one of the two interfaces. In
particular, 8 students perform the tasks on A, the other 8 on B. The following table
shows the students’ task performance scores in the experiment
Interface
A
B
5
4
4
3
2
2
3
2
2
2
1
1
5
3
4
3
This is an (Unpaired) Two-Sample Two-Sided T Test
7
Let X1 and X2 represent the performance scores of the students from groups A and B,
respectively.
Let µ1 denote the mean of X1, and µ2 denote the mean of X2
H0: µ1 = µ2
H1: µ1 ≠ µ2
X 1 = 3.75, S1 = 1.488, X 2 = 2.5, S 2 = 0.926
Assuming the equal variances, we can calculate df = n1 + n2 – 2 = 8+8-2 = 14
Set α = 0.05, then the critical value t0.025(14) = 2.145. In other words, the critical
region is t ≤ -2.145 or t ≥ 2.145
S12 +S22
2
Sp =
t=S
=
X1 − X 2
p
1/ n1 +1/ n2
(1.488)2 +(0.926)2
2
=1.239
= 1.2393.⋅751−/28.+51/ 8 = 2.018 < 2.145
So you cannot reject the null hypothesis that the two search interfaces have the same
effectiveness
We can also estimate the p-value using the t-distribution table, 0.05<p<0.1
8
4
9/22/2010
Advantages of Paired Comparison Design
Test is more sensitive when using paired comparison design compared
to using unpaired comparison design
Removing the extraneous variables due to variations across different
experimental units
Search Interface Example . The effect of the variations across different students
is removed in the paired design (example I) but not in the unpaired design
(example (II)
For the same sample size (the number of replicates in each experiment
condition), paired comparison design requires fewer experimental
units
Search Interface Example . 8 students are used in the paired design, but 16
students are used in the unpaired design.
9
Chi-Square One-Sample Variance Test
Test whether the variance of a normal distribution is equal to some specific value
H0: σ 2 = σ 02
H1: σ 2 ≠ σ 02 or
σ 2 < σ 02 or
σ 2 > σ 02
Chi-square test for testing the hypotheses is
χ 2 = σSS = (n−σ1) S
2
0
2
0
2
χ 2 ~ χ 2 (n −1)
If the χ2 statistics calculated from the data falls within the critical region, then reject
H0; otherwise, cannot reject H0 and favor H1
10
5
9/22/2010
Table 2.7
α
χ2 Distribution
11
Equipment Variance Test Example I. A chemical engineer wants to examine
whether the variance of a piece of test equipment is different from the average
variance of other similar test equipment in the industry (which is 15). For the
study, a random sample of 10 observations of the test equipment is taken, and
the sample variance of the observations S2 = 14.5
H0: σ 2 = σ 02 = 15
H1: σ 2 ≠ σ 02
Set α = 0.05, then the critical values are χ20.025(9) = 19.02 and χ20.975(9) = 2.70 . In
other words, the critical region is χ2 < 2.70 or χ2 > 19.02
2
.5
χ 2 = ( n−σ1) S = 9⋅14
= 8.7
15
2
0
Because the χ2 statistics calculated from the data does not fall within the critical
region, we cannot reject H0
12
6
9/22/2010
Equipment Variance Test Example II. A chemical engineer wants to examine
whether the variance of a piece of test equipment is less than the average
variance of other similar test equipment in the industry (which is 15). For the
study, a random sample of 10 observations of the test equipment is taken, and
the sample variance of the observations S2 = 14.5
H0: σ 2 = σ 02 = 15
H1: σ 2 < σ 02
Set α = 0.05, then the critical value is χ20.95(9) = 3.33. In other words, the critical
region is χ2 < 3.33
2
.5
χ 2 = ( n−σ1) S = 9⋅14
= 8.7 > 3.33
15
2
0
Because the χ2 statistics calculated from the data does not fall within the critical
region, we cannot reject H0
13
Two-Sample Variance Test
Test whether the variances of two normal distributions are equal
H0: σ 12 = σ 22
H1: σ 12 ≠ σ 22 or
σ 12 < σ 22 or
σ 2 > σ 22
1
F test for testing the hypotheses is
S12
F = S 2 F ~ F (n1 −1, n2 −1)
2
If the F statistics calculated from the data falls within the critical region, then reject
H0; otherwise, cannot reject H0 and favor H1
14
7
9/22/2010
Table 2.7 (Cont’d)
F1−α (v1 , v2 ) = 1 / Fα (v2 , v1 )
F Distribution
15
Two Equipment Variances Test Example I. A chemical engineer wants to test
whether the variances of two types of test equipment are the same. For the test,
a random samples of 10 observations is taken for each of the two types of
equipment, and the sample variances of the two samples are S12 = 14.5 and S22 =
10.8
H0: σ12 = σ 22
H1: σ12 ≠ σ 22
Set α = 0.05, then the critical values are F0.025(9,9) = 4.03 and F0.975(9,9) = 1/4.03
= 0.248. In other words, the critical region is F < 0.248 or F > 4.03
S12
F = S 2 = 14.5 / 10.8 = 1.34
2
Because the F statistics calculated from the data does not fall within the critical
region, we cannot reject H0
16
8
9/22/2010
JMP 8 Quick References
17
Normality Check
Search Interface Example
18
9