Investigating readers' mental maps of references in an online system

Computers & Education 53 (2009) 799–808
Contents lists available at ScienceDirect
Computers & Education
journal homepage: www.elsevier.com/locate/compedu
Investigating readers’ mental maps of references in an online system
Yu-Fen Yang a,*, Wing-Kwong Wong b, Hui-Chin Yeh a
a
b
Graduate School of Applied Foreign Languages, National Yunlin University of Science and Technology, 123 University Road Section 3, Douliu, Yunlin, Taiwan, ROC
Department of Electronic Engineering, National Yunlin University of Science and Technology, 123 University Road Section 3, Douliu, Yunlin, Taiwan, ROC
a r t i c l e
i n f o
Article history:
Received 16 December 2006
Received in revised form 24 April 2009
Accepted 27 April 2009
Keywords:
Evaluation of CAL systems
Reading strategy
Mental map
Reference
Interactive learning environments
a b s t r a c t
Referential identification and resolution are considered the keys to help readers grasp the main idea of a
text and solve lexical ambiguities. The goal of this study is to design a computer system for helping college students who learn English as a Foreign Language (EFL) develop mental maps of referential identification and resolution in reading. Four modules, Natural Language Processing (NLP), User Interface,
Recording, and Feedback Tool, are implemented in the system. Results of this study showed that the
more-proficient EFL readers were able to identify and resolve most of the references to form a coherent
mental map from different parts of a text. The less-proficient readers commonly resolved references by
relying on grammatical rules instead of semantic contextual clues. They often referred references to
incorrect objects. To overcome the difficulties in figuring out the relationship between two words, the
less-proficient readers usually asked for more feedbacks. As students progressed in reading, they
requested fewer feedbacks in the online system. Some recommendations for future studies are discussed.
Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction
Successful reading comprehension depends on whether a reader is able to integrate and interpret textual information appropriately
(Grabe & Stoller, 2002; Huang, Chern, & Lin, 2009). To achieve this, a reader should have the ability to identify and resolve cohesive ties
for they are claimed to best serve as the contextual clues for connecting information presented in a text (Al-Jarf, 2001; Halliday & Hasan,
1976; Yuill & Oakhill, 1988). Five cohesive ties are considered to help a reader integrate textual meaning; they are reference, substitution,
ellipsis, conjunction, and lexical cohesion (Halliday & Hasan, 1976; Pritchard & Nasr, 2004).
Among these five cohesive ties, referential identification and resolution are the keys to successful reading comprehension (Chung, 2000;
Oakhill & Yuill, 1986). Nunan (1993) states that it is a prerequisite for a reader to identify the referential relationship among sentences in
comprehending a text. The reader can further resolve the referential relationship based on his reading proficiency and linguistic competence (Al-Jarf, 2001). Eventually, he can construct a well-structured memory representation of the text. The mental representation allows
the reader to have a deeper understanding of the text and enables him to answer questions about the text, to recall, or to summarize it
(Potelle & Rouet, 2003).
According to Chen and Dai’s study (2003), most Taiwanese EFL college students fail to identify pronouns and relative pronouns in texts.
Huang’s study (1993) also indicates that most EFL college students have difficulties in identifying cohesive ties for they lack instruction and
practice in recognizing and resolving cohesive ties in their past learning experiences. It is clear that most EFL students in Taiwan lack skills
in referential identification and resolution necessary for reading comprehension. They are more likely to encounter difficulties and failures
in the process of reading.
The difficulties that EFL students encounter in processing cohesive ties have not been closely studied yet since traditional experiments
fail to assess the complicated reading processes of referential ties (Al-Jarf, 2001; Chen, 2001). The major methods used to document students’ reading processes are mainly naturalistic observation, interviews, or think-aloud protocols (Schacter, Herl, Chung, Dennis, & O’Neil,
1999). Researchers in traditional experiments most often implement think-aloud task or recall to understand a reader’s comprehension by
asking him to self report his reading process simultaneously or retrospectively. However, these methods are usually time consuming and
labor intensive. The reader might be poor at verbally expressing his own reading process. As a result, the verbal report may lead to a shallow investigation.
* Corresponding author. Tel.: +886 5 534 2601x3136; fax: +886 5 5312058.
E-mail address: yangy@yuntech.edu.tw (Y.-F. Yang).
0360-1315/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compedu.2009.04.016
800
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
Online learning systems are proposed to better investigate students’ learning process recently (Demetriadis, Papadopoulos, Stamelos, &
Fischer, 2008; Ding, 2009; Martindale, Pearson, Curda, & Pilcher, 2005) since they can support higher-order learning, teach problem-solving
skills to students who are struggling with learning difficulties, and have a positive impact on learning outcomes (Martindale et al., 2005;
Mercier & Frederiksen, 2008). However, very few computer systems have been designed to examine the roles of referential identification
and resolution in reading. How students encounter difficulties in resolving references, how they read and reread the sentences to select and
reselect references, and how individual students are supported by feedback in a system are seldom investigated. Particularly, how to manage the process and the product of referential identification and resolution in reading has become a big challenge for both EFL students and
teachers due to an average class size of more than 40 in Taiwan.
This study aims to design a computer system for helping EFL college students develop mental maps of referential identification and resolution in reading. Referential resolution, in this study, is defined as a reading strategy applied by a reader to accurately interpret references
in texts. It occurs when the reader identifies persons and objects in different parts of a text pointing to the same entity (Walsh & JohnsonLaird, 2004). Mental maps of references are represented as nodes and connected by lines to indicate how a reader figures out the relationships among references. Taking the following sentences as examples, ‘‘Sigmund Freud was a doctor of psychology in Vienna, Austria at the
end of the nineteenth century. He treated many patients with nervous problems through his cure talk.” ‘‘He” in the second sentence refers
to ‘‘Sigmund Freud” in the first sentence. That is, referential resolution can be used to establish the relationships among text elements that
have the same meaning. Based on our research purpose, three research questions are addressed: (1) How is the system developed to support students in referential identification and resolution? (2) How do students present their mental maps of referential identification and
resolution in the system? (3) How do students progress in their referential identification and resolutions in the system?
2. System development
In the system, students are required to follow two steps in reading a text: (1) to identify references generated by the system, (2) to
establish the relationships among the references to form mental maps. Key functions are provided to offer students apprenticeship in a
reading environment; they are system-guided instruction, warm-up practice, referential interface, and feedback. System-guided instruction informs students the goal of the system and prepares them for the following practices. Warm-up practice provides students initial
attempts to develop their skills in referential identification and resolution. The practice is also a guide for shaping and supporting students’
referential strategy development. Feedback serves as a scaffold to help students guess from textual clues and reflect on their incorrect answers of referential identification and resolution in order to make correct choices.
The system built for this study includes four modules, natural language processing (NLP), user interface, recording, and feedback. Fig. 1
shows the system architecture of this study. The teacher designs the course, selects texts which students have to read, and types in the
texts to the NLP module through the teacher interface (Fig. 1). The NLP module picks the referential devices from each text and segments
the text into sentences. The selected referential devices and the split sentences are then saved in the database.
The recording module traces students’ reading process and behavior while they are constructing mental maps in referential identification
and resolution. These traced data are then studied by the teacher to identify the difficulties students encounter and the performances
among different reading proficiency groups. The feedback module compares students’ initial maps with that of an expert while students
are constructing their mental maps. It then provides three candidate references for each device that needs correction to students whenever
they encounter difficulties figuring out the relationship between two words.
2.1. NLP module
In the system, a reader is asked to find out three types of references, personal, demonstrative, and locative pronouns since they appear
more frequently in texts (e.g., Fortanet, 2004; Kennison, 2003). Personal references refer to individuals or objects by specifying their functions or roles in the speech situation (Halliday & Hasan, 1976), such as ‘‘I,” ‘‘me,” and ‘‘you.” Demonstrative references act as forms of verbal
location, such as ‘‘this,” ‘‘these,” and ‘‘that.” The speaker figures out the reference by means of location on a scale of proximity. Locative
Natural Language
Processing Module
User Interface
Teacher
Text
POS Tagger
Learner
Chunker
Referring Interface
Referential
Devices
Referential
Device Selector
Sentences
Sentence
Detector
Database
Recording
Module
Feedback
Module
Expert’s
Map
Fig. 1. System architecture for referential identification and resolution.
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
801
Sigmund Freud was a doctor …at the end of the nineteenth century.
1) He treated many patients with nervous problems through his "talk cure."
2) For this type of treatment, …about anything that was bothering them.
3) While treating his patients, …person's actions in her or his present life.
4) Freud called the place where past … were hidden the unconscious mind.
5) Images from the unconscious … dreams or through the person's actions.
6) Freud wrote a book about …unconscious mind and dreaming in 1899.
7) The title of the book was The Interpretation of Dreams.
Fig. 2. Results of sentence detection.
Sigmund/NNP Freud/NNP was/VBD a/DT doctor/NN of/IN psychology/NN in/IN
Vienna,/NNP Austria/NNP at/IN the/DT end/NN of/IN the/DT nineteenth/JJ
century./NN He/PRP treated/VBD many/JJ patients/NNS with/IN nervous/JJ
problems/NNS through/IN his/PRP$ "/'' talk/NN cure."/NN
Fig. 3. The result of tagging two sentences with POS tags.
references are used to indicate location, such as ‘‘here” and ‘‘there.” In addition to these three types of references, definite noun phrases are
also included. The system provides tools to help the reader to reach the goal of referential identification and resolution. These tools include
POS (Part-of-speech) tagging, phrase chunking, referential device selection, and sentence detection. Most of the tools are provided by the
open source system OpenNLP (http://opennlp.sourceforge.net).
The system first tries to pick out all referential devices in a text automatically. In doing this, the system needs to do POS tagging of each
word in the text. With the help of the POS tags, the system chunks the text into noun phrases, verb phrases, prepositional phrases, and
adjectival phrases, etc. Not all noun phrases serve as referential devices, so the system must pick out the real referential devices from
all noun phrases. When the reader tries to pick out the references of a referential device, the system helps the reader to focus on a sentence
by highlighting the sentence at which the reader clicks the mouse. Sentence detection is automatically done by the system.
2.1.1. Sentence detection
When reading a text in the system, a reader can click the mouse at a word in the text in order to highlight the sentence he is reading. The
system does this by detecting the sentence boundary surrounding the word. Moreover, the system records the sentence selection as one of
the reader’s actions in using the system.
A sentence usually ends with the punctuation of period, question mark, or exclamation mark. However, the occurrence of a period does
not guarantee the end of a sentence. For example, ‘‘Mr.”, ‘‘Ms.”, and ‘‘76.5%” each contains a period that does not indicate the end of a sentence. Our system uses the sentence detector of OpenNLP. The result of sentence detection in the text ‘‘Sigmund Freud” is shown in Fig. 2.
2.1.2. Part-of-speech tagging
When given a text, the system automatically picks out a list of referential devices in the text so that the reader can try to identify these
devices. The first step of automatic selection of referential devices is to tag the words of the text with POS tags. That is, the part-of-speech
tag of each word must be determined. The result of tagging the first two sentences in the Sigmund Freud text is shown in Fig. 3. The POS
tags used by the system are given in Table 1.
2.1.3. Phrase chunking
After POS tagging, the system uses grammar rules to do syntactic analysis of each sentence. Some common grammar rules are shown as
follows.
NP ? modifier + baseNP
baseNP ? modifier + noun
modifier ? article | number | present particle | past particle | adjective | noun
Table 1
Some POS tags used in the system.
NNP
VBD
DT
NN
IN
JJ
PRP
NNS
PRP$
Proper noun
Verb
Determiner
Single noun
Preposition
Adjective
Pronoun
Plural noun
Possessive pronoun
802
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
VP ? verb + preposition
PP ? basePP + baseNP
basePP ? preposition
After syntactic analysis, each sentence is segmented into a number of phrases, including noun phrases, verb phrases, prepositional
phrases, etc. The result of chunking the first sentence in the Freud text is shown in Fig. 4.
2.1.4. Referential device detection
After chunking a text, the system picks out the noun phrases (Fig. 5) from the text and checks whether they are referential devices or not
(Fig. 5). The true referential devices in the first two sentences of the text are [NP He] and [PRP his]. The remaining NPs serve as the referents
of the referential devices. Therefore, the problem of referential device detection is to screen out those noun phrases that do not have any
referential function.
The noun phrases are divided into two groups: those without referents and those with referents. Thirty four of the noun phrases have
the pattern [NP X1] [PP of] [NP X2], e.g., ‘‘the end of the nineteenth century”. Among the 34 noun phrases, only three are referential devices.
Therefore, the pattern [NP X1] [PP of] [NP X2] is used to filter out the noun phrases (e.g., ‘‘the end of the nineteenth century”) that are
unlikely to be referential devices. A simple algorithm for detecting referential devices is developed (Fig. 6).
In one case, the chunking result of a noun phrase is ignored, e.g., [NP his talk cure], and the possessive pronoun is considered a true
referential device, [NP his]. A complete list of the result of selecting referential devices in the text is given in Fig. 7.
2.2. User interface
With the teacher interface, the teacher manages course data, provides texts which students should read, and observes students’ reading
process and behavior data. With the student interface, a student draws a map indicating the relationships among references. For the online
practice of referential identification and resolution, a referring interface, a recording module, and a feedback module are implemented
(Fig. 8). They will be described in detail as follows.
[NP Sigmund Freud] [VP was] [NP a doctor] [PP of] [NP
psychology] [PP in] [NP Vienna, Austria] [PP at] [NP the end] [PP
of] [NP the nineteenth century]. [NP He] [VP treated] [NP many
patients] [PP with] [NP nervous problems] [PP through] [NP his
"talk cure"]
Fig. 4. The result of chunking two sentences.
[NP Sigmund Freud]
[NP the nineteenth century]
[NP a doctor]
[NP psychology]
[NP He]
[NP many patients]
[NP Vienna, Austria]
[NP the end]
[NP nervous problems]
[NP his "talk cure"]
Fig. 5. The noun phrases from Fig. 4 that are candidates of referential devices.
Begin
All noun phrases are candidates of referential devices.
For each noun phrase,
If the phrase is a definite noun phrase (which begins with “the”, “this”, etc.)
and the phrase does not match the pattern [NP X1] [PP of] [NP X2],
then the noun phrase is considered a referential device.
End
Fig. 6. Algorithm of referential device detection.
He
his
this
his
him
them
his
he
there
she or he
these
the person's
her or his
the place
the unconscious mind
the unconscious mind
the person's
his
the unconscious mind
Fig. 7. Referential devices automatically detected in the text.
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
803
Fig. 8. The interface and modules in referential resolution practice.
A. Toolbar. This includes a number of graphic tools. Connection tool can connect referential devices. Feedback tool compares students’
initial graph and the expert’s. It then informs students their incorrect answers and provides them with three candidate references
for each device that needs correction. Other tools managing the canvas are cut, copy, paste, erase, group, ungroup, zoom in, zoom out,
undo, and redo.
B. Text field. This area is used to show the text. Students can select a word or a sentence as a text element and then drag into the canvas
directly when they comprehend the relations among the references. The sentence will be highlighted when students select a
sentence.
C. Referential device list. All referential devices are listed in this area. Students have to understand what these referential devices refer
to, then drag and drop them to the canvas. When a referential device is selected, the referential device will also be highlighted in the
text field.
D. Canvas. Students add links to indicate the relationships among references on this canvas. They can add, erase, drag and drop elements
on the canvas. Fig. 9 is an example of a student’s mental map.
E. Feedback frame. This will inform students the correctness of their answers in the referring practice. In addition, the feedback will
provide three options when the student has difficulties finding out the links between two words.
2.3. Recording module
The system uses a recording module to trace students’ reading process. From the data, the teacher can observe and identify the difficulties the students encounter and the difference in performance among various reading proficiency groups. The findings will be helpful for
the teacher to modify his/her instruction according to students’ strengths and weaknesses. The module uses some predicates to record
Fig. 9. An example of a student’s mental map.
804
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
Table 2
Some of the recording predicates.
Predicates
Description
Read a sentence: [T]
Add a cell: [X]
Erase a cell: [X]
Referring: [X][Y]
Erase referring: [X][Y]
Get feedback count: [X]
Get feedback of: [X]
Select a sentence T which the student is reading in the text field
Add an element X in the canvas
Erase an element X in the canvas
Add an meaningful relation between X and Y
Erase a relation between X and Y
Request the feedback for X times
Request the feedback for the reference X
students’ behavior data. Table 2 shows some of the recording predicates. For example, the recording module traces a student’s reading process in Fig. 10.
2.4. Feedback module
After students have constructed their initial graphs, they can request a feedback. The feedback module compares the initial graph with
the expert’s. The results of comparison will inform students which references are incorrect and offer them three possible referents for each
incorrect device.
A student’s answer is compared to the expert’s referential map. First, the module transforms the expert’s map and the student’s map
into predicates. Second, the module finds all references from the student’s predicates and then compares them with the expert’s. If the
student’s references do not match the expert’s, they are incorrect. Then the module will provide one correct reference and two distractors
as clues back to the student. For example, Table 3 compares a student’s map with the correct one and finds out that the referential device A
is incorrect in the student’s map. The feedback module will then offer the correct reference B and two distractors D and E back to the
student.
3. Method
3.1. Participants
A total of 90 junior and senior college students were recruited from two reading classes in a technological university in central Taiwan.
Their language proficiency levels were identified by their reading scores in a simulated online exam Testing of English for International Communication (TOEIC) with a reliability of 0.87. The full score in the online exam was 200.
The frequency distribution of all the participants’ score was used to divide the participants into three groups of readers. It was found
that the highest frequencies fell in two score intervals, 81–90 (8 students) and 131–140 (8 students). These two intervals were used to
2:
Read a sentence: [Sigmund Freud … of the nineteenth century.]
3:
Read a sentence: [He (3) treated … his (4) "talk cure."]
4:
Add a cell: [Sigmund Freud]
5:
Add a cell: [He (3)]
6:
Add a cell: [his (4)]
7:
Add a connection: [He (3)] connects to [Sigmund Freud]
…...
38:
Read a sentence: [Freud wrote a book … dreaming in 1899.]
39:
Add a cell: [his (22)]
40: Add a connection: [his (22)] connects to [Sigmund Freud]
Fig. 10. A student’s reading process.
Table 3
An illustration of comparing the student’s graph with the correct one.
Expert
Map
Student
Predicates
Add a cell: [A]
Add a cell: [B]
Referring: [A][B]
Map
Predicates
Add a cell: [A]
Add a cell: [C]
Referring: [A][C]
805
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
Frequency Distrbution of Readers' Scores in the Simulated Online TOEIC Exam
Frequency
10
8
6
4
2
0
31-40
41-50
51-60
61-70
71-80
81-90 91-100 101-110 111-120 121-130 131-140 141-150 151-160 161-170 171-180 181-190 191-200
more-proficient readers
average readers
less-proficient readers
Fig. 11. The frequency distribution of the participants’ score.
identify a less proficient reader group (31–90) and a more proficient reader group (131–200). Participants with scores in the range 91–130
were excluded from the study. The frequency distribution of the participants recruited in the current study is shown in Fig. 11.
Thus, 38 more-proficient readers and 37 less-proficient readers were identified in this study. The mean score of the more-proficient
readers was 157.89 with a standard deviation of 18.37. The mean score of the less-proficient readers was 57.84 with a standard deviation
of 14.70.
3.2. Material
The online practice of referential identification and resolution used three texts to examine the participants’ reading comprehension. The
three texts were selected from College Reading Workshop (Malarcher, 2005) based on the following four criteria: abundance of references for
reading practice, similar length, similar readibility level, texts written for EFL college students. The three texts were presented in sequence
in the textbook and they are text 1—Freud and the Meaning of Dreams (number of words: 708; number of referring phrases: 38), text 2—The
Tragedy of Echo and Narcissus (number of words: 692; number of referring phrases: 62), and text 3—Commerce through the Internet (number
of words: 651; number of referring phrases: 21).
Table 4 shows the readability of the three texts. According to the Flesch reading ease test (Farr, Jenkins, & Paterson, 1951), higher scores
indicate material that is easier to read and lower scores indicate harder-to-read passages. Texts with scores of 90–100 are understandable
to native speakers of English at the average 11-year old students and texts with scores of 60–70 are understandable to 13- to 15-year old
students. Passages with results of 0–30 are understandable to college graduates (Flesch, 1948; Kincaid, Fishburne, Rogers, & Chissom,
1975). Among the three texts, text 1 has the most words, 708. Text 2 has the most sentences, 46, and paragraphs, 8. Text 3 is the most
difficult with Flesch–Kincaid grade level at 11.5.
3.3. Procedures of data collection
The present study was conducted from April 24th, 2006 to June 10th, 2006. The 90 college students were asked to do the online practice
of referential identification and resolution. The online system recorded the participants’ reading behavior and performance. For each text,
the participants were required to do referring practice by selecting referential words and drawing the relationships among these words on
the canvas. The online system grades the participants’ referring practice by giving one point to each correct connection between two referential words.
3.4. Procedures of data analysis
The participants’ raw data were categorized into two types: reading process and reading product. Reading process refers to the participants’ construction of their mental maps and the records of their reading behavior. Reading product includes the participants’ scores of
referring and frequencies of feedbacks. In referring practice, three results were differentiated: correct, incorrect, and missed. These were
further analyzed with the Statistical Package for Social Science (SPSS) version 12.0.
Table 4
The readability of text 1–3.
Number of words
Number of sentences
Number of paragraphs
Flesch reading ease
Flesch–Kincaid grade level
Text 1
Text 2
Text 3
708
35
7
53.6
10.6
692
46
8
66
7.6
651
35
6
44.5
11.5
806
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
Table 5
The mean scores and standard deviation of referring practice.
Text 1
Referring
Feedback
Text 2
Text 3
Mean
SD
Mean
SD
Mean
SD
20.71/38 (54.5%)
17.90
8.91
17.31
43.42/62 (70%)
13.61
16.27
13.02
14.47/21 (69%)
7.67
6.11
7.93
4. Result
The 90 participants’ mean scores and standard deviations of referring practice and feedback frequency were shown in Table 5. The total
points of referring practice in the text 1 are 38; that of text 2 are 62; and that of text 3 are 21. Students made progress in referential resolution from 54.5% in text 1 to 69% in text 3. They also asked for fewer feedbacks as they progressed in reading as the mean of feedback
frequency decreases from 17.90 in text 1 to 7.67 in text 3.
A t-test was also conducted to examine whether there were significant differences between the more-proficient and the less-proficient
readers in practice of referential identification and resolution and frequency of feedbacks. The result of t-test showed that there was a significant difference (t = 5.29, p < .05) between the more- and less-proficient readers. This result was similar to that of feedback frequency.
That is, the more- and less-proficient readers’ mean scores in the referring practice and frequency of feedbacks were significantly different.
Regarding the differences between the more- and less-proficient readers in drawing mental maps of referential resolution, some examples were found. The more proficient readers were able to integrate personal references in different parts of the text to form a coherent
network (graph (a) of Fig. 12). In contrast, the less-proficient readers missed many personal pronouns which refer to ‘‘Sigmund Freud.”
Although some of the less-proficient readers tried hard to complete the referring practice, many of their answers were still incorrect.
For instance, graph (b) of Fig. 12 shows one of the less-proficient readers referred plural pronouns to a single subject. Some of the less-proficient readers referred a pronoun to an incorrect word (graph (c) of Fig. 12).
Pearson product-moment correlation coefficient was also conducted to find out the correlation among the frequency of feedbacks, errors, and missed rate in the practice of referential identification and resolution. Table 6 presents the correlation between the number of
errors and feedback frequency and that between the number of missed references and feedback frequency.
a
his (92)
he (3)
his (93)
his (88)
his (4)
his (6)
him (7)
Sigmund Freud
his (86)
his (73)
his (9)
his (26)
his (22)
he (10)
b
these (65)
them (53)
the impression (50)
they (54)
them (67)
c
their (33)
that (29)
Fig. 12. Students’ mental maps of referential resolution.
Table 6
The correlation between the number of errors and feedback frequency and that between missed referents and feedback frequency.
Readers
Text 1
Error/feedback
Moreproficient readers
Less-proficient readers
-.43
.19
Text 2
Missed/feedback
.38
.81
Error/feedback
.87
.14
Text 3
Missed/feedback
.76
.62
Error/feedback
.50
.36
Missed/feedback
.86
.82
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
807
Table 6 shows that the correlation between the number of errors and feedback frequency is negative for the more-proficient readers.
That is, when they asked for more feedbacks, they made fewer errors. This is also true for the relationship between the number of missed
references and feedback frequency. In contrast, the correlation between the number of errors and feedback frequency is positive for the
less-proficient readers. They did not actively ask for feedback and did not select the correct answer when the three options were provided
as feedback. However, the correlation between the number of missed references and feedback frequency is negative for the less-proficient
readers. That is, the more feedbacks they asked for, the fewer references they missed.
5. Discussion
From the results of this study, some findings are explored. First, the less-proficient readers had some common difficulties in referential
identification and resolution. They misinterpreted the relationship between two words as attribution instead of equivalence in the practice
of referential identification and resolution. For example, ‘‘The title of the book was The Interpretation of Dreams.” Because of the phrase ‘‘of
the book,” ‘‘the title” becomes definite and it refers to ‘‘The Interpretation of Dreams.” The relationship between ‘‘the title” and ‘‘of the book”
is attribution rather than equivalence.
The less-proficient readers also misperceived a reference when it syntactically played the role as dummy subject or relative pronoun.
For instance, ‘‘A few researchers, however, said that dreams had a more useful function.” ‘‘That” is a relative pronouns to connect the following clause rather than serving as a cohesive tie to refer to other text elements. Another example is ‘‘It may be true that almost every
company has a website, but many of these sites are for information rather than for commerce.” ‘‘It” here is a dummy subject used to substitute ‘‘almost every company has a website, but many of these sites are for information rather than for commerce” for the sake of a common sentence structure. It is known that changes of the syntactic roles call for different methods for interpreting the cohesive ties.
Second, the online system in several ways scaffolds students in referential identification and resolution. The system picks out and highlights all referential devices in the text automatically. This would reduce students’ cognitive overload to look for all possible references in a
text. The system can also automatically trace and record students’ reading behavior and process with the trace result illustrating students’
performances in referential identification and resolution. That is, students’ difficulties in different kinds of references, personal, demonstrate, and locative, can be examined. Based on this examination, the reading teacher can design follow-up lesson plans and classroom
activities to assist the struggling students in overcoming their difficulties and compensating their weaknesses. In other words, the trace
result serves as a guide for the reading teacher to design and plan subsequent remedial courses.
Moreover, in overcoming the difficulties in referential identification and resolution, the feedback with candidate references provided by
the system serves as a scaffold. However, this scaffold does not appear automatically in the system. Only when the students requested scaffolding did the system provide it to help students overcome the difficulties. It was found that when a student asked for more feedbacks, his
incorrect and missed references decreased.
Finally, it is suggested that the system is applicable to students from other backgrounds. The system could be employed as a tool for
teachers to develop students’ reading strategies in terms of referential identification and resolution. It can also assist teachers to evaluate
students’ mental maps based on the criteria of referential resolution. Different from other systems which compare students’ mental maps
with those of experts but do not reveal the criteria for the comparison, our system shows students’ mental maps composed by referential
resolution so that the criteria of comparing experts’ mental maps with those of students become standardized and possible. Regarding suggestions for improvements, some new help functions could be added.
1. After the participants had completed the practice of referential identification and resolution, the system should not only provide the
results of grading but also the expert’s mental maps. This will help the participants compare their initial mental maps with those of
the experts.
2. The percentage of similarities between students’ and the expert’s mental maps should also be provided, so that the participants’ mental
maps in the practice of referential identification and resolution can be further analyzed.
3. Readers’ self-assessment could be taken into consideration for the next implementation of the system. The participants could be given
more freedom in terms of the type and the number of texts they intend to read based on their individual needs. A reader’s portfolio
could also be adopted in the system that allows each reader to set up his reading plan for a whole semester.
4. In the next version of the system, more guided instruction on referential devices and examples will be provided. Furthermore, although
the system used an audio tutorial to assist students in a trial section, a tutorial video will be developed to show the readers how to draw
a mental map.
Acknowledgements
This study was supported by National Science Council in the Republic of China, Taiwan (NSC 96-2411-H-224-014 & NSC 95-2520-S-224001-MY3). These research grants made the continuation of this study possible.
References
Al-Jarf, R. S. (2001). Processing of cohesive ties by EFL Arab College Students. Foreign Language Annals, 32(2), 141–151.
Chen, I. (2001). A constructivist computer-assisted language learning environment for Pennsylvania State University. DAI, 62, No. 12A, p. 4031.
Chen, S. Z., & Dai, H. T. (2003). An analysis toward Taiwanese students’ cohesive ties in reading comprehension. In Proceedings of the 20th international english teaching and
learning (pp. 235–258). Taiwan.
Chung, S. L. J. (2000). Signals and reading comprehension – Theory and practice. System, 28, 247–259.
Demetriadis, S. N., Papadopoulos, P. M., Stamelos, I. G., & Fischer, F. (2008). The effect of scaffolding students’ context-generating cognitive activity in technology-enhanced
case-based learning. Computers and Education, 51(2), 939–954.
Ding, N. (2009). Visualizing the sequential process of knowledge elaboration in computer-supported collaborative problem solving. Computers and Education, 52(2), 509–519.
Farr, J. N., Jenkins, J. J., & Paterson, D. G. (1951). Simplification of Flesch reading ease formula. Journal of Applied Psychology, 35(5), 333–337.
Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.
808
Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808
Fortanet, I. (2004). The use of ‘‘we” in university lectures: reference and function. English for Specific Purposes, 23(1), 45–66.
Grabe, W., & Stoller, F. L. (2002). Teaching and researching reading. Malaysia: Pearson Education.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman.
Huang, C. T. (1993). Topic-prominence and morphosyntactic parameter in Chinese. MI: University Microfilm Inc.
Huang, H. C., Chern, C. L., & Lin, C. C. (2009). EFL learners’ use of online reading strategies and comprehension of texts: An exploratory study. Computers and Education, 52(1),
13–26.
Kennison, S. M. (2003). Comprehending the pronouns her, him, and his: Implications for theories of referential processing. Journal of Memory and Language, 49, 335–352.
Kincaid, J. P., Fishburne, R. P., Jr., Rogers, R. L., & Chissom, B. S. (1975). Deviation of new readability formulas for Navy enlisted personnel, Research Branch Report 8–75,
Millington, TN: Naval Technical Training, US Naval Air Station, Memphis, TN.
Malarcher, C. (2005). College reading workshop (2nd ed.). Compass Publishing Inc.
Martindale, T., Pearson, C., Curda, L. K., & Pilcher, J. (2005). Effects of an online instructional application on reading and mathematics standardized test scores. Journal of
Research on Technology in Education, 37(4), 349–360.
Mercier, J., & Frederiksen, C. (2008). The structure of the help-seeking process in collaboratively using a computer coach in problem-based learning. Computers and Education,
51(1), 17–33.
Nunan, D. (1993). Challenges in EFL classrooms. TESOL Matters, August/September, 7.
Oakhill, J., & Yuill, N. (1986). Pronoun resolution in skilled and less skilled comprehenders: Effects of memory load and inferential complexity. Language and Speech, 29, 25–36.
Potelle, H., & Rouet, J.-F. (2003). Effects of content representation and readers’ prior knowledge on the comprehension of hypertext. International Journal of Human Computer
Studies, 58(3), 327–345.
Pritchard, R. M. O., & Nasr, A. (2004). Improving reading performance among Egyptian Engineering Students. Journal of English for Specific Purposes, 23(4), 425–445.
Schacter, J., Herl, H. E., Chung, G. K. W. K., Dennis, R. A., & O’Neil, H. F. Jr., (1999). Computer-based performance assessments: A solution to the narrow measurement and
reporting of problem-solving. Computer in Human Behaviors, 403, 418.
Walsh, C. R., & Johnson-Laird, P. N. (2004). Co-reference and reasoning. Memory and Cognition, 32(1), 96–106.
Yuill, N., & Oakhill, J. (1988). Effects of inference awareness training on poor reading comprehension. Journal of Applied Cognitive Psychology, 2, 33–45.