6/25/2013 Three Types of Mixed Methods Designs I can…

6/25/2013
I can…
Three Types of
Mixed Methods Designs
Created by: John W. Creswell
jcreswell1@unl.edu
University of Nebraska-Lincoln
…describe the three major types of mixed methods designs.
…design an outline of a study within the South African context for
each of the three designs.
Adapted by: Carolina Bustamante
bustamantemc@huskers.unl.edu
University of Nebraska-Lincoln
Convergent Parallel Design
Quantitative
Data Collection
and Analysis
A Convergent Parallel Design
Compare
or relate
Interpretation
Qualitative
Data Collection
and Analysis
Qualitative
Data Collection
and Analysis
Quantitative
Data Collection
and Analysis
Explanatory Sequential Design
Quantitative
Data Collection
and Analysis
Follow up
with
Qualitative
Data Collection
and Analysis
Interpretation
Merge
Exploratory Sequential Design
Interpretation
Qualitative
Data Collection
and Analysis
Builds to
Quantitative
Data Collection
and Analysis
Interpretation
Steps in Conducting a Convergent
Design
Wittink, M. N., Barg, F. K., & Gallo, J. J. (2006). Unwritten rules of talking to doctors about depression:
Integrating qualitative and quantitative methods. Annals of Family Medicine, 4(4), 302-309.
Purpose: Develop a better understanding of concordance and discordance between patients’ and
physicians’ assessments of older patients’ depression status.
Sample for QUAN & QUAL: 48 patients who self-identified as depressed.
1.
Collect the quantitative and qualitative data at roughly the same time an
independently (single step approach)
2.
Independently analyze the quantitative and qualitative data. Possibly
transform the qualitative data into “counts” by counting codes, themes.
3.
Compare the results from the quantitative and the qualitative strands (or
the transformed qualitative “counts”).
4.
Discuss the comparison of those results. Indicate areas of convergence
or divergence between the quantitative and qualitative results.
5.
For areas of divergence, provide explanations for the divergence, such
as collect more data, reexamine the quantitative and qualitative results, or
point out limitations in one of the databases or the other.
QUAN Data Collection
Three measures of participants’ depression status:
physician’s rating, patient’s self-rating, standardized
measure of depressive symptoms. Demographic data.
QUAL Data Collection
Semi-structured interviews on patient’s
perceptions.
QUAN Data Analysis
Identify agreement or disagreement between
physician’s and patient’s ratings. Group comparisons for
significant differences in terms of other variables of
interest.
QUAL Data Analysis
Transcription of interviews, development
of four themes.
Merge Results
Comparison of QUAL findings according to four themes, together with QUAN
results according to agreement and disagreement between physician and patient,
and other measures.
Interpretation
Linking themes regarding how patients talk to physicians with demographic data
and measures of depression provided a better understanding of the topic.
1
6/25/2013
Procedures and Challenges in a Convergent Design
7
Data Analysis Issues in Convergent Design
Subdivided into
Subdivided into
Ask parallel questions
Addressed
through
Sampling: Use equal or unequal sizes;
Consider same sample or different
sample
Addressed
through
Analyzed by
Analyzed by
Report separate results
Compared with
QUAL
Compared with
QUAN
Merge results:
 Text comparisons
 Joint display
 Data transformation
• Analyze each database separately
• Merge the databases:
• Side-by-side comparison
• Joint display
• Data transformation
Explain
Convergence/
Divergence
Side-By-Side Results Comparison
Joint Display
Classen et al. (2007)
Side-by-side Comparison in Results
Data Transformation
Barkaoul (2007) – Scoring an English foreign language exam
2
6/25/2013
Activity 1
When Results Diverge,
Conflict, or are Inconsistent
Handout: Three Types of Mixed Methods Designs | Convergent Parallel
Discuss with one or two other people possible ideas for a convergent parallel mixed
methods study within the context of South Africa. Fill the table with the group’s ideas.
–
–
–
–
–
Purpose of study:
Side with one form of data or the other
Gather more data and make further comparisons
Reexamine each data base for rigor
Reevaluate whether parallel questions asked
State limitation
Sample:
QUAN Data Collection:
QUAL Data Collection:
QUAN Data Analysis:
QUAL Data Analysis:
Merge of results (Side-by-side comparison, joint display, data transformation):
Interpretation:
Steps in Conducting an Explanatory
Sequential Design
An Explanatory Sequential Design
Quantitative
Data Collection
and Analysis
Explained
by
Qualitative
Data Collection
and Analysis
1.
Collect the quantitative data (phase 1)
2.
Analyze the quantitative data.
3.
Determine what quantitative results need to be further
explained. Determine what participants can help with
this explanation.
4.
Collect the qualitative data (phase 2)
5.
Analyze the qualitative data.
6.
Explain how the qualitative data helps to explain the
quantitative results
Interpretation
Ivankova, N. V., & Stick, S. L. (2007). Students’ persistence in a Distributed Doctoral Program in
Educational Leadership in Higher Education: A mixed methods study. Research in Higher Education,
48(1), 93-135.
Procedures and Challenges in Conducting
an Explanatory Sequential Design
Purpose: Identify factors that contribute to students’ persistence in the program and explore
participants’ views on these factors.
QUAN Sample
207 students enrolled in program
QUAN Data Collection
Online survey that measured nine predictor variables based on theories of student persistence.
QUAN Data Analysis
Divide sample into four groups according to level of persistence.
Mixing
Identify individuals within each group that had average scores.
Identify significant factors from the QUAN results that need to be further explored.
qual Sample
One typical individual from each group.
qual Data Collection
One-on-one interviews on significant factors from QUAN results.
qual Data Analysis
Theme development.
Interpretation
Discuss major QUAN results and how the qual data helped explaining statistical results more in depth.
Leads to
Analyzed by
Tested by
Phase 1
Followed up by
Explained by
1- Identify results
to follow up
2- Identify participants
for follow up
Choose qualitative
sample from quantitative
sample
Report results
separately
Phase 2
Addressed by
Analyzed by
Interpreted by
In discussion, explain how
qualitative results help to
explain quantitative results
3
6/25/2013
Sampling Issues in Explanatory
Sequential Design
Order of Interpretation in Explanatory Sequential Design
• Choose follow up qualitative sample from members
of the quantitative sample
•
Sample size will differ between two samples
•
Follow up with qualitative sample to:
• Explore surprising, confusing findings
• Explore outlier, extreme cases
• Explore significant predictors
• Explore significant demographic groups
1.
2.
3.
4.
QUAN results
How results are used to build into qual
qual findings
How qual results help to explain QUAN results
• Recognize that qualitative sample cannot be identified
until after quantitative results completed
Activity 2
An Exploratory Sequential Design
Handout: Three Types of Mixed Methods Designs | Explanatory Sequential
Discuss with the same group possible ideas on how to convert your previous convergent parallel
study into an explanatory sequential mixed methods study. Fill the table with the group’s ideas.
Purpose of study:
QUAN Sample:
QUAN Data Collection:
QUAN Data Analysis:
Qualitative
Data Collection
and Analysis
Builds into
Quantitative
Data Collection
and Analysis
Interpretation
Mixing:
qual Sample:
qual Data Collection:
qual Data Analysis:
Interpretation:
Steps in Conducting an Exploratory
Sequential Design
Myers, K. K., & Oetzel, J. G. (2003). Exploring the dimensions of organizational assimilation: Creating
and validating a measure. Communication Quarterly, 51(4), 438-457.
Purpose: Describe and measure the assimilation of new employees within organizational settings.
qual Sample
13 participants selected from different types of organizations.
1.
Collect the qualitative data.
2.
Analyze the qualitative data.
3.
Design the quantitative strand based on what is learned from the
qualitative findings:
- develop a new instrument
- modify an existing instrument
- develop a typology or taxonomy
- design an intervention
4.
Collect the quantitative data.
5.
Analyze the quantitative data.
6.
Explain how quantitative results help to generalize the qualitative
data, provide a new instrument, identify new variables, help to form
a new typology, etc.
qual Data Collection
One-on-one semi-structured interviews.
Field notes.
qual Data Analysis
Theme development: 6 dimensions of organizational assimilation.
Mixing
Development of instrument based on 6 dimensions: Organizational Assimilation Index (OAI)
QUAN Sample
342 employees across diverse industries.
QUAN Data Collection
Administration of OAI survey.
QUAN Data Analysis
Scale reliability, subscale validation, construct validity (correlational hypothesis testing).
Interpretation
Need of both types of data to explore topic, and create and test instrument.
4
6/25/2013
Procedures and Challenges in an Exploratory Sequential Design
Sampling Issues in Exploratory
Sequential Design
Leads to
Explored by
Analyzed by
Phase 1
Determine what
qualitative results
to follow up
Leads to
Use rigorous
procedures
Phase 2
Report results
separately
Samples unequal;
Quantitative sample
different
Leads to
Phase 3
Leads to
Tested by
Analyzed by
• What purposeful qualitative sample to select to explore
• What quantitative sample to select based on qualitative
results
• Whether to draw the qualitative and quantitative samples
from the same population
Resulting in
Generalize qualitative themes
to large sample; Develop measures
or instruments that are context
sensitive
Steps in Designing a Good
Psychometrically-Sound Instrument
– How to transition from qualitative results to survey
development
• Quotes = items
• Codes = variables
• Themes = scales
– How to build new variables from qualitative results
– What qualitative results to use in developing intervention
strategies
1. Determine items, scales, constructs from qualitative results.
2. Develop initial survey.
3. Pilot-test initial survey and revise.
4. Field-test revised instrument.
5. Analyze field-test data (reliability, validity, item characteristics).
6. Establish supplemental materials, administrative procedures, and
publications for instrument.
Data Interpretation Issues in Exploratory
Sequential Design
Activity 3
Handout: Three Types of Mixed Methods Designs | Exploratory Sequential
– State first qual findings
– Next indicate how the qual findings build into
quan phase (e.g., how a good psychometric
instrument is designed)
– Next present the quan phase (test of the
instrument, typology, etc.)
– Finally indicate how quan results show
importance (generalizability) of the qual
contextualized themes
Discuss with the same group possible ideas on how to convert your previous explanatory sequential
study into an exploratory sequential mixed methods study. Fill the table with the group’s ideas.
Purpose of study:
qual Sample:
qual Data Collection:
qual Data Analysis:
Mixing:
QUAN Sample:
QUAN Data Collection:
QUAN Data Analysis:
Interpretation:
5
6/25/2013
I can…
…describe the three major types of mixed methods designs.
…design an outline of a study within the South African context for
each of the three designs.
Three Types of
Mixed Methods Designs
Created by: John W. Creswell
jcreswell1@unl.edu
University of Nebraska-Lincoln
Adapted by: Carolina Bustamante
bustamantemc@huskers.unl.edu
University of Nebraska-Lincoln
6