Theory – practice gap in CAQDAS users` experience in Poland

Transcription

Theory – practice gap in CAQDAS
users’ experience in Poland
KAROL HARATYK, UNIVERSITY OF WARSAW
KAROLHARATYK@GMAIL.COM
ANNA KORDASIEWICZ, UNIVERSITY OF WARSAW,
GOLDSMITHS COLLEGE (VISITING SCHOLAR)
A.KORDASIEWICZ@UW.EDU.PL
CAQDAS 2014 CONFERENCE: PAST, PRESENT & FUTURE: 25 YEARS
OF CAQDAS: THE ROLE OF METHODS TEACHING IN LEARNING TO
USE QUALITATIVE SOFTWARE
HORSLEY, 1-3 MAY 2014
The CAQDA vicious triangle in Poland…
2
CAQDA's
image in the
literature
users
opinions on
CAQDA
CAQDA
practice
THEORY – PRACTICE GAP
Karol Haratyk , Anna Kordasiewicz
Presentation plan
3
Description of a research project on qualitative
analysis in Poland
2. The CAQDA viscious triangle:
1.
1.
2.
3.
CAQDAS’ image – state-of-the-art
CAQDAS – users’ opinions
CAQDAS – users’ practice
3. Theory-practice gap
1.
Grounded theory applied? – An illustration
4. Conclusions
1. Description of
the research idea and project
4
a. IDEA OF THE PROJECT
b. DESCRIPTION OF THE RESEARCH PROJECT
Point of departure/some assumptions
5
The critics of the qualitative research
Lack of clarity of qualitative analysis procedures (i.a.
Silverman 2004, 2007, Lofland et al. 2010).
Explicit accounts of analytical proceeding is a rarity „anegdotalism” (Silverman 2004) or qualitative data
serving only illustrative purposes.
Few systematic methodological studies on the practice
and theoretical background of qualitative analysis
The theory – technique connection
6
Some but still few systematic methodological studies
on the practice and theoretical background of
qualitative analysis
Examples pertain mostly to the computer assisted analysis,
like Franzosi et al. (2012), MacMillan (2005), Koenig (no
data), Saillard (2011), Schönfelder (2011), Bielinski, Iwińska,
Kordasiewicz (2007), KWALON experiment (FQS 2011),
Jones, Diment (2010),
Theoretical background of analysis outside the context of
CAQDA has been presented in Frost et al. (2010), Stubbe et al.
(2003).
BUT: „Technique is theory” - Mauthner&Doucet (2003),
Ruppert et al. (2013)
Stubbe et al.
2003
Frost et al. 2010
7
Lonkila 1995,
MacMillan 2005
Bringer et al. 2006
Jones & Diment 2010
Some papers based on
KWALON experiment,
FQS 2011
Franzosi et al. 2012
Niedbalski&Slezak 2012
Theory
Technique
Bielinski et al. 2007
Results
KWALON
experiment, FQS 2011
The idea of the metamethodological project
8
The comprehension of the process of qualitative
analysis is vital for the enhancement of the quality of
qualitative research
We may find inspiration in the field of quantitative
methodological studies, e.g. experiments on the question
phrasing in survey research
It is necessary to investigate how we conduct our
analysis and what its connection with the
theoretical background is
We focus on computer assisted QDA but we include
and compare paper-based with the CAQDA
Research project consisted of:
9
CAQDA's
image in the
literature
users’ opinions
on CAQDA
CAQDA
practice
Literature review (1) CAQDA’s image in
the literature
Online survey on qualitative research
CAQDA users’ (2) opinions and (3) practice
Open-ended and closed-ended questions
201 academic and non-academic Polish
researchers
Conducted between 1.10.2012-31.12.2012.
Online survey on CAQDA users’ opinions and
practice
10
Young and academic researchers tied to major
academic centres in Poland and women were overrepresented
CAQDAS users - 66 people declared use of
CAQDAS (35% of the sample, overrepresentation)
CAQDAS used were mostly paid software:
Atlas.ti used by 55% of CAQDAS users
MaxQDA 42%
Nvivo 27%
QDAminer 22%
And some freeware:
Weft QDA 9%
Open code 6%
Who are the CAQDAS users?
11
43/65 aged 25-34 CAQDASUs younger then total
48/65 Ph.D. students (25) and Ph.D. (23) bigger
share of Ph.D. among CAQDASUs
55/65 connected to academia, but 28/65 combine
academic and extra-academic jobs bigger share
than in total
38/65 CAQDASUs declared that research activities were
most important in their professional life in last 3 years
CAQDASUs more focused on research than total
38 women, 26 men/65 women slightly
underepresented among CAQDASUs
Research project - continued
12
Qualitative research in progress
Practice: own analytical projects, software
workshops, teaching and training
discourse analysis,
autoethnography,
IDI and observation of analytical practice
2. The CAQDA viscious traingle
13
CAQDA's
image in the
literature
users
opinions on
CAQDA
CAQDA
practice
The CAQDA's image in the literature
14
users
opinions
on
CAQDA
CAQDA's
image in the
literature
CAQDA
practice
2. CAQDA’s image in the literature
15
a. FOCUS ON LEGITIMIZING CAQDA NOT ON CRITICAL
ASSESMENT
b. LEGITIMIZNG CAQDA THROUGH:
a.
b.
c.
Manual-like approach
Focus on CAQDA advantages
Relatively superficial references to grounded
theory strategy
Manual-like approach
16
Uniformely positive CAQDA image
17
Superficial references to grounded theory strategy
18
Papers on CAQDA:
Tie CAQDA to the grounded theory approach
irrespective of the actual software potential
Use references to grounded theory more as a
general qualitative analysis legitimization
strategy (conf. Kelle 1997) than to
systematically confront GT procedures with
software functionalities
CAQDA’s image in the literature - summary
19
Papers on CAQDA in Poland focus on
Introducting CAQDA
Legitimizing CAQDA
And so they have an introductory character which
impedes critical methodological studies of CAQDA
Result: lack of critical assessment of CAQDA
The users opinions on CAQDA
20
CAQDA's
image in
the
literature
users
opinions
on CAQDA
CAQDA
practice
3. CAQDAS – users’ opinions
21
CAQDAS ARE O.K., BUT…
a. ADVANTAGES OF THE CAQDA
b. DISADVANTAGES OF THE CAQDA
c. SUMMARY
CAQDAS – users’ opinions
22
Users (participants of the web survey) seem to share
part of the literature’s enthusiasm in that they are
able to list a number of advantages of CAQDA
BUT
Users also acknowledge many disadvantages of the
CAQDA not discussed in the literature
Among the advantages of the CAQDAS…
23
advantages
Speeding up of analysis (25)
Working with big stock of
data and efficient data
archiving (19)
Ease of information retrieval (6)
Systematicity (7)
Team work (4)
Visualisations (3)
Among the disadvantages of the CAQDAS…
24
disadvantages
Alienation from research
material
Adjustment of data choice to
software capabilities
Prices
(Atlas.ti):
505 euro
Basic Ph.D.
salary at the
University of educational
Warsaw is
1725 euro
560 euro /
commercial
month
Lack of holistic approach due to
focus on coding
Lack of coding standardized
procedures
Difficulties in mastering the
software
Costs
Screen dimensions
Users’ opinions – summary (1)
25
Among the good sides of CAQDA, technical
issues (39 users) prevailed over methodological
impact (15 users)
Among bad sides two main groups of arguments
received equal attention: negative
methodological effects (19 users), extramethodological constraints of the CAQDA (20
users)
Users’ opinions – summary
26
Slightly more users listed advantages (43 users) than
disadvantages (36), BUT
Few listed only good (10 users) or only bad (3 users)
sides of the CAQDAS
So the opinion of users is much more ambivalent
than the image produced in the literature
According to our users, the aim of qualitative analysis
is to deepen understanding but apparently CAQDAS
are not the tool helpful in achieving this aim…
Is it a result of users’ more critical approach or of
imperfect practice?
The CAQDA practice
27
CAQDA's
image in
the
literature
users
opinions on
CAQDA
CAQDA
practice
Practice
28
N=201; n=61
Coding of texts
Coding of images
Coding of recordings
Automatic coding
Retrieval of fragments coded with one code
Retrieval of fragments coded with combination of
two or more codes
97%
28%
23%
51%
69%
58%
Looking at the number of uses of a code
Looking at the size of fragments coded with a code
Visualizations
Reliability tests
51%
23%
33%
23%
CAQDAS learning process
29
Participants learned to use the software:
On their own (34/59)
On an academic course (11/59)
On a commercial training (8/59)
From a friend (6/59)
Impact of the learning mode on analytical practice
30
Those who learned on their own used more advanced
options (e.g. retrieval of a combination of codes,
visualisation – valid also for the academic courses)
Those who learned by themselves and on an academic
course analysed non-textual data more often.
Those who learned on commercial training relied more
heavily on automated or quantitative options, like
„automatic coding”, analysis of number of uses and the
scope of codes.
“…teaching the use of the programs to novice researchers has to be
embedded in a pedagogy which has a sense of the exemplars of
qualitative analysis, rather then as skills and techniques to be
mechanically aplied”, Lee&Fielding 1991
3. The theory – practice gap
31
CAQDA's
image in
the
literature
users
opinions on
CAQDA
CAQDA
practice
THEORY –
PRACTICE GAP
Methodological and theoretical pluralism
32
In our survey domination but not monopoly of
Grounded theory (45/201 mentioned spontanously,
123/201 confirmed when prompted)
•
•
•
•
•
•
•
Interactionism (13),
Anthropology (12),
Discourse analysis (11) and critical discourse analysis (6)
interpretative sociology (11),
phenomenology (11),
triangulation (11),
visual sociology (10)
We will focus on GT as most widely represented
CAQDAS and grounded theory approach
33
CAQDAS and grounded theory approach - good old marriage
or dangerous liaisons?
Not obvious nor necessary a connection.
GT commonly associated with CAQDA (Lonkila (1995),
Niedbalski, Slezak (2012), Bringer (2006)
The very design and „language” of the GT present in
CAQDAS
Procedures commonly analysed as corresponding:
Coding procedure
Memoing
Linking
Kołtun (2011) – reception of the grounded theory in Poland
– also through CAQDA university courses
CAQDAS and grounded theory approach
34
GT commonly associated with CAQDA
BUT:
CAQDAS’ use does not have to follow GT strategy
„purely rhetorical” references to GT (Lonkila
1995, Kelle 1997)
„There is a danger that the choice of techniques
available (computer programs like ATLAS/ti or
NUD.IST may also suggest the choice of method
(grounded theory)” (Lonkila 1995).
Grounded theory and CAQDAS use
35
N=201; n=181
In the last three years the strategy of grounded theory...
...has not constituted for me Count
important reference point
in qualitative research
...has constituted for me one Count
of the reference points in
qualitative research
...has constituted for me the Count
most important reference
point in qualitative research
Total
Count
Do you use specialized software for
qualitative analysis at the stage of the
analysis of the textual data?
Yes
No
14
45
Total
59
22%
38%
33%
38
55
93
60%
47%
51%
11
18
18%
15%
63
100%
118
100%
29
29,0
16%
181
100%
Coding as a key GT practice
36
GT is a rich and heterogeneous tradition, although
commonly coding is considered as one of the key
elements. It is understood not as a sheer technical
operation of a label atribution, but as analysis itself
(Lonkila 1995, Richrds & Richards 1991), through:
Open, axial and selective coding
connected with:
Categories and their properties and dimensions identification
Which is achieved through
a process of iterative analysis and confrontation (comparison)
of the data and generating the analytical categories from data
(Strauss 1987, Charmaz 2009)
What is the GT analytical style/coding practice?
37
Questions on practice testing GT or
non-GT analytical style:
Coding with a ready-made list or a code list
elaborated alongside analysis?
Coding data fragments once or more?
Indexing or theoretical character of codes?
Analysis after coding phase or
simultaneosly?
Usually in the last three years
(please, indicate which sentence describe better your practice in the scale 1 to 4)
Does the strategy of grounded
theory constituted a reference
po int in qualitative research in
the last three years?
No
X
1 – firstly I have created codes and next I have coded data rather without changing
the code list or changing it slightly
38
4 – code list I have created only during the process of coding data [31%]
Yes
N
X
Total
N
X
N
2,21 14 2,88 49 2,73
63
1 – I have coded data only once (rather I have not returned to fragments coded
2,00 14 2,84 49 2,65
previously in order to code them again)
4 – I have coded data for many times (I have returned to fragments coded previously
in order to code them again) [29%]
63
1 – mainly I have used codes in order to find fragments interesting for me
1,93 14 2,63 49 2,48
4 – codes have constituted for me mainly a tool to generate generalizations: after
coding data I have created subsequent, mo re general codes which I have used to code
again data or fragments o f data [22%]
63
1 – I have analyzed data already during the process of coding: I have browsed data, I 2,50 14 2,51 49 2,51
have made crosstabs or I have analyzed co-occurrence of codes etc. (not including
reliability tests) [14%]
4 – I have analyzed data only after coding them: : I have browsed data, I have made
crosstabs or I have analyzed co-occurrence of cod es etc. (not including reliability
tests)
63
Grounded theory applied?
39
There are no big differences in coding styles between
users who refer to GT and those who do not
Generally, the results indicate a slight propensity for
GT coding style, so maybe it is a sign of GT as a
paradigm for CAQDA
BUT
Not even GT proponents get close to the „ideal type”
of GT coding style, so the paradigm is weak and
applied rather loosely.
This results have to be confronted with an analistparticipant perspective in a qualitative study.
Inspiration from other studies versus GT approach
in the analytical pracitce
40
Table. Inspiration from other studies versus GT approach in the analytical pracitce
COMPARISON WITH OTHER THOERIES IN THE
COURSE OF ANALYSIS
In the course of the last three years…
…I aimed to compare my results with the state-of-the-art in
the course of the analysis
... I aimed not to comp my results with the state-of-the-art in
the course of the analysis
Total
Has the grounded theory approach constituted
a reference point in your research for last three
years?
NO
YES
39%
73%
62%
27%
13
100%
48
100%
Total
40
66%
21
34%
61
100%
4. Conclusions
41
1. Limited use of possibilities offered by CAQDAS.
2. Users’ ambivalent attitude towards CAQDAS.
3. Methodological texts: legitimization of CAQDA.
4. Domination of grounded theory in methodological texts as
well as users declarations.
5. Weak application of grounded theory as far as coding style
is concerned – theory – practice gap.
strategy of legitimization of CAQDA
in texts -> insufficient support
users’ ambivalent
attitude
limited use of
the possibilities
(Grounded) theory –
practice gap
42
43
Karol Haratyk, University of Warsaw
karolharatyk@gmail.com
Anna Kordasiewicz, University of Warsaw
a.kordasiewicz@uw.edu.pl
Authors
44
Karol Haratyk, Ph.D. student at the Institute of Sociology,
Univeristy of Warsaw, uses CAQDAS (MAXqda, ATLAS.ti) in
his research projects, conducts academic courses on
qualitative data analysis;
Anna Kordasiewicz, assistant professor at the Centre of
Migration Research, University of Warsaw, uses CAQDAS
(mainly ATLAS.ti) in her research projects, carries out
academic cources and commercial trainings in qualitative
data analysis (also using freeware, opencode);
We initiated an informal network of CAQDAS users in
Poland, that involves analysts from the University of Łódź,
Jagiellonian University in Cracow and University of Gdańsk;
Inspiration to take up the research subject was our
analytical and training experience, as well as our interest in
the theoretical background of qualitative analysis

Theory – practice gap in CAQDAS users` experience in Poland

Transcription

Similar documents

Press Kit - buddemusic.de

Generational Attributes

PDF

[Press Kit]Anna Farrow 2016-ANNA.EN

Art, Science Technology of QDA

May / June 2015 - Texas Scottish Rite Hospital for Children

Dirty looks, dirty bathrooms.Every credible medical organization

SEP.06

Anna Marie Periquet Profile JL

World War II Fighter Pilot Chances Death in Japanese Skies and

new - 1 Million 4 Anna