CHAPTER TWO LITERATURE REVIEW
The literature reviewed in this chapter includes the models of reading comprehension, and the theoretical basis of this study: Dual Coding Theory and Schema Theory. Moreover, related empirical studies reporting on the effects of visual aids on reading comprehension and different sequencings of visual aids presentation are also reviewed.
Reading Comprehension
Reading comprehension is a complicated cognitive process in human mind. It is
“not just understanding words, sentences, or even texts, but involves building a model within the mind of the comprehender” (Hammadou, 1991). It is an interaction between a reader and a text, and characterized as a “psychological guessing game”
(Goodman, 1967). To further explore the process of reading comprehension, models of reading process – bottom-up, top-down, interactive, construction-integration models – are reviewed.
Models of Reading Process
Barnett (1989) categorized the reading models into three basic types—bottom-up, top-down, and interactive. The bottom-up model held reading to be a serial process from the smallest, simplest linguistic features before any higher order process occurred (Hammadou, 1991; Sadoski & Pavio, 2001). Readers built up the meaning of the text by decoding letters, words, and sentences in a linear order (Carrel, 1988).
Therefore, the bottom-up processing is essentially data-driven (Carrel, 1984), moving
from the parts to the whole.
The top-down model, on the other hand, regards reading as a process starting
“from readers’ memory to interpret the meaning of the text” (Sadoski & Pavio, 2001).
Readers begin with their schema, predict the text meaning based on their prior knowledge, and reconstruct the meaning with their own interpretation. They are viewed as an active information processor (Eskey, 1973). Therefore top-down processing is essentially conceptually-driven (Carrel, 1984), moving from the whole to the parts.
Contemporary theories of reading process are interactive model (Rumelhart, 1980). It combines both bottom-up model and top-down model, and emphasizes reading comprehension as the process which readers switch between these two processings. That is, meaning of a text is a dynamic relationship between the text and the reader. Readers build up text meaning through simultaneous integration of their background knowledge (top-down) and accurate processing of printed input (bottom-up) in the text. Interactive model is regarded as a more suitable and accurate model for explaining reading process (Eskey, 2002; Heilman, Blair & Rupley, 1990).
The construction-integration model proposed by Kintsch (1998) views reading comprehension as a circular process of construction and integration. Readers continuously construct new propositions when processing the incoming data and integrate referential inferences to achieve comprehension. In this model, there are two levels of understanding in the reading outcome: textbase model and situation model (Chu, 2002).
In the textbase level, readers process the text at a proposition level, and generate inferences between propositions to bridge textual information and achieve local coherence. They might infer by synthesizing and summarizing the text content to construct and stabilize textbase. Inferences made at the textbase level are text-based
inferences for the present study.
Since reading is an interactive process, readers might also draw inferences from their long-term memory and integrate both textbase and prior knowledge by establishing a situation model. Readers might infer based on their personal experience by doing higher-level thinking and logical reasoning so that they can create global coherence for mental representation of the text. In the present study, inferences drawn from readers’ background knowledge are termed reader-based inferences.
Dual Coding Theory
Research in cognitive science explores issues such as perception, memory, attention, and their functions in various learning tasks. The mental models of information processing have pointed out the important role of imagery in comprehension and memory (Gambrell & Jawitz, 1993; Johnson-Laird, 1983).
Among them, Dual Coding Theory, the theory of cognition in literacy, concerns the nature of language and imagery, and explains how text and illustration interplay in human cognition. In the present study, dual coding theory and its relationship to reading is reviewed as a base to explain the effect of comic strip on reading comprehension.
Dual Coding Theory (DCT) is proposed by Paivio (1971, 1986), and developed by Sadoski & Paivio (2001), especially in reading and writing. The basic concept of DCT is that “cognition in reading and writing consists of the activity of two separate coding systems of mental representation, one system specialized for language (verbal) and one system specialized for dealing with nonverbal objects and events.”(Sadoski &
Paivio, 2001, p43). The linguistic coding system can be referred to as verbal system while the nonverbal coding system can be referred to as imagery system. All the
meaning and knowledge in DCT is explained through direct interconnections between the modality-specific mental representations in the two systems. (Sadoski & Paivio, 2001). The two systems may work independently or interactively. The activity in one system can trigger the activity in the other. That is, language can evoke imagery, and imagery can evoke language. The interconnectedness of the two systems facilitates the interpretation of our environment (Rieber, 1994; Simpson, 1995). In the following, units for text representation and three levels of processing are discussed.
Logogens and Imagens
Logogens are the basic units in the verbal system, and imagens are the basic units in the nonverbal system. Logogens are alternatively called verbal representations, verbal encodings, mental language, and inner speech; imagens are alternatively called nonverbal representations, nonverbal encodings, mental images, or imagery (Sadoski & Paivio, 2001). Both are modality-specific units of various sizes but the hierarchies are qualitatively different. Logogens and the verbal system are organized sequentially such as phonemes, letters, and words. They are constrained by our left-to-right conventional experience. Instead, imagens are organized in a more continuous, integrated way. They are more holistic and simultaneous in nature and cannot be separated into discrete elements easily. These two units are useful in recognition. And the combination of the two provides great flexibility to cognition.
Three Levels of Processing
There are three different levels of processing theorized in DCT: representational processing, associative processing, and referential processing, as shown in figure 1
below.
Figure1.The General Model of Dual Coding Theory (Sadoski & Paivio, 2004)
The initial activation of logogens or imagens is representational processing.
The activation of a representation depends on the combined effect of the stimulus situation and individual differences. Sadoski and Paivio (2004) indicate that in reading a text, the stimulus situation would be the text characteristics, and individual differences would include reading ability, background knowledge, instruction, and so on. Therefore, the legibility of printed linguistic forms and readers’ familiarity may influence the degree of representational processing. Recognition and perception are involved in this level, but meaningful comprehension may not be achieved.
Associative processing involves spreading activation within the same coding system, either verbal or nonverbal. This spreading activation can be between modalities or within modalities in a system. For example, the written word “tree”
can activate visual logogen “tree” and then activate phonological encoding /tri/ in auditory modality. Moreover, the activation of visual logogen “tree” can associate other visual logogens such as “leaves,” “trunk,” “flowers,” “fruit” and so on. Once association is activated, meaning is elaborated and constrained according to the given context and individual differences.
Referential processing refers to the spreading activation between the verbal code and nonverbal code which is associated with meaningful comprehension. Logogens can activate corresponding imagens and imagens can activate corresponding logogens.
There is not a one-to-one referential correspondence between logogens and imagens.
(Rieber, 1994; Sadoski & Paivio, 2004). Sometimes, a word can evoke many images, and an image can evoke many words. The degree of evocation depends probabilistically on verbal and situational contexts and individual life experiences (Sadoski & Paivio, 2001). “Once activated logogens spread their activation referentially to one or more imagens in the nonverbal system, associative processing may occur within that system and, in turn, refer back to the verbal system” (Sadoski
& Paivio, 2004, p.1342). Meaning is defined and elaborated through the spreading activation between and within two codes and interpreted with additional inferred information.
Reading Process in Dual Coding Theory
Dual-Coding Theory is a general theory of cognition that applies directly to the reading process and can account for its various phenomena (Sadoski & Paivio, 2001).
In terms of bottom-up process, DCT assumes that linguistic units derived and organized from natural language are mentally represented in various sensory modalities. For example, readers may use these representations visually and phonologically to perceive grapheme-phoneme correspondences and configurations of letters, words, and word sequences so that the meaning of the text can be interpreted.
Regarding top-down process, in DCT, it involves the use of information from the reader’s memory to interpret the meaning of a text and the processing at lower levels of analysis is partially determined by higher levels of analysis (Sadoski & Paivio,
2001). DCT provides a broader and more specific account of the effects of meaning, coherence, and inference (Liu, 2004). In the reading process, readers are assumed to change their position from moment to moment along with a continuum from feature perception to inferential text modeling in activating verbal and nonverbal mental representations of the text (Sadoski & Paivio, 2001). Verbal-nonverbal connections help readers create alternative, interconnected contexts to integrate the text.
Schema Theory
Schema is what we call background knowledge or prior knowledge. The origin of schema was drawn from Piaget’s (1971) theory of accommodation, positing that children construct meaning by assimilating information into their existing schemata as well as by accommodating existing schemata to new knowledge. The role of background knowledge in language comprehension has been formalized as Schema theory (Bartlett 1932; Rumelhart 1980; Rumelhart & Ortony, 1977). Schema theory research has shown the importance of background knowledge within a psycholinguistic model of reading process (Carrell, 1984; Carrell & Eisterhold, 1983).
It deals with the reading process, where readers are expected to combine their existing prior knowledge with the text they read. According to Carrell (1984), the process of interpretation is guided by the principle that every input is mapped against some existing schema and all aspects of the schema must be compatible with the input information. In the interactive process of reading, meaning understood from the text is not actually in the text, but in the reader. Only after the reader’s schema is activated can one be able to see or hear (Harmer, 2001), so helping readers to build background knowledge is always the focus of reading research.
Schema theory research provides strong evidence for the effectiveness of
pre-reading activities. First, pre-reading activities can not only build new background knowledge of the readers but also activate their existing background knowledge.
Activating readers’ background knowledge before they begin to read may help students’ comprehension (Carrell & Eisterhold, 1983; Coady, 1979; Grabe, 1991; Ur, 1996). Coady (1979) also suggests that interest and background knowledge may be able to compensate for certain syntactic deficiencies. The more sufficient the semantic input provides, the weaker the syntactic control of the text is. Second, schema in the pre-reading activities can arouse readers’ interests and motivate them to read the text with less effort and more willingness to participant in reading. Third, according to Anderson (1985) and Wilson & Anderson (1986), schema provides scaffolding that help to assimilate text material. A schema “provides ideational scaffolding—text fits into slots within the schema, directs allocation of attention, enables inferential elaboration, allows orderly searches of memory, facilitates editing and summarizing, and permits inferential reconstruction” (Wilson & Anderson, 1986, pp. 35-36).
Therefore, with schema provided before reading, readers can be equipped with new background knowledge to activate existing schema, be motivated to have a purpose in reading, and be provided ideational scaffolding so that they can read with confidence.
The Effect of Visual Aids on Reading Comprehension
Visuals have been used in language teaching for a long time and are believed to have the facilitative effects on text comprehension and retention (Glenberg &
Langston, 1992; Liu, 2004; Moore & Scevak, 1997; Mueller, 1980; Omaggio, 1979;
Schallert, 1980). When visual aids illustrate information central to the text, they also increase the retrieval for the text content (Kulhavy, Lee, & Caerino, 1985; Paivio, 1986). Learning is enhanced when mental referential connections are made between
verbal and visual modes (Mayer & Sims, 1994). Moreover, visual aids help motivate learners, focus their attention, and induce more elaborate processing of text information (Peeck, 1987).
The Functions of Visual Aids
Levin, Anglin, & Carney (1987) suggested five major functions of visual aids in reading and these categorizations are consistent with other researchers’ point of view.
1) Decoration: Visual aids can make the text more enjoyable and attractive to arouse readers’ interests. They can focus learners’ attention, and motivate them not only to read but also to infer what was intended but not clearly stated in the text (Peeck, 1987; Wright, 1989).
2) Representation: Visual aids can visualize the content of the text and induce correct interpretation. Visual aids increase understanding (Guri-Rosenbilit, 1989). Readers are likely to have better understanding of the information when they make accurate links between texts and visual aids (Moore & Scevak, 1994)
3) Transformation: Visual aids help readers recode the conceptual information in a more memorable form. Visual aids facilitate text retention (Glenberg & Langston, 1992) and illustrated texts are proved to be nearly twice as memorable as pure texts (Sadoski, Goetz &
Avila, 1995).
4) Organization: Visual aids help readers organize the text to achieve coherence.
They can illustrate the relationship among text elements so that the context enhances readers’ comprehension (Stanovich, 1980) and is used to compensate for the deficiency of word recognition
(Wright, 1989).
5) Interpretation: Visual aids provide the text with more concrete explanation for the readers to interpret. They can help students search for the meaning on their own, contributing to the creation of context in reading comprehension (Wright, 1989).
Levie and Lentz (1982) further categorized the above five functions into four major roles: “1) attentional; 2) affective; 3) cognitive; and 4) compensatory.”(p.218) Kiefer (1995) pointed out that language arts and visual arts share commonalities.
Students think not only through written language but also via visual images. The combination of verbal and nonverbal stimulus helps students to cross over and discover meaning. Pictures relevant to the text content repeat important information.
Such repetition has large effects on memory (Glenberg, 1979), facilitating comprehension and retention.
The Types of Visual Aids in Reading
There are many kinds of visual aids used to facilitate comprehension. Visual representation, according to Braden (1994), generally includes five distinct areas: (1) film (video); (2) signs, symbols, and icons; (3) images and illustration; (4) multi-image (images that are shown concurrently or sequentially); and (5) graphic representation. Among these categories, illustrations and graphic representation are commonly seen in reading classroom and textbooks.
Illustrations
Illustrations are visualization, such as pictures, drawing, photograph or other
work of art, which provide visual representation of textual information. They may represent the text subject by only one image (e.g. the picture of the referent object), or represent the text content by a series of images (e.g. comic strips.) The effect of illustration in reading has been proved by many studies (Levie & Lentz, 1982; Peek, 1987; Pressly & Miller, 1987). Illustrations play a role in both readers’ cognitive domain and affective domain. They not only enhance readers’ comprehension and retention of the texts but also arouse readers’ interests and curiosity which make reading more enjoyable.
Graphic Representation
Graphic representations are visualization, such as charts and graphs, diagrams, and graphic organizers. Charts and graphs are often used to present tabular numeric data in the text. They can be seen as pie chars, bar graphs, line graphs, and so forth.
They not only show the organization of the textual information in groupings but also reveal its quantitative relationship. Both of them make it easier for readers to understand large quantity of data and the relationship among them.
Diagrams, according to Saunders (1994), are visuals that “represent and identify parts of a whole, a process, a general scheme, and/or the flow of results of an action or process” (p.185). They are simplified and structured visual representation of concepts and ideas to clarify the topic of the text. They have been proved effective especially in science instruction when it comes to processes (Winn, 1987).
Graphic organizers or semantic maps are visual display of structured verbal information. They can be used as pre-organizers or post-organizers, and can be teacher-directed or student-constructed. They help students understand the hierarchies of the text and are proposed as tools for teaching thinking (Bellanca, 1990). There are many forms of graphic organizers, such as webs, fishbone charts, Venn diagrams,
KWL tables.
Graphic representation of the text mostly contains some words or phrases in itself while illustration may or may not include verbal information in it. To precisely examine the effect of visuals in the present study, the researcher chooses comic strips as the visual aid which contains no verbal stimulus and can best represent the content of narratives.
Studies on Comic Strips as Visual Aids
The visual aid examined in this study is the comic strip. It is a series of pictures inside boxes that tell a story (Liu, 2004). Generally, comic strips combine both words and images to construct a narrative. Their characteristics are communicative, popular, readable, and they also contain aesthetic perception as well as intellectuality (Harvey, 1994; Inge, 1990; Swain, 1978; William, 1995). Many researchers have suggested using comic strips in language teaching. Swain (1978) in her study found that comic strips interested both good and poor readers and gave suggestions for activities using comic strips in the language classroom. Wright (1979) presented a preliminary readability study of 20 popular comic strips as evidence to show that comics do provide easy reading material for reluctant disabled readers. Wright and Sherman (1996) provided a teaching strategy using Bloom's taxonomy and newspaper comic strips to promote higher-level thinking in elementary and secondary classrooms. They also used a Peanuts comic strip to explain and demonstrate two functions of questioning (centering and expansion). Then, in their 1999 article, they claimed that creating a comic strip can promote writing skills and stimulate children's higher level thinking about explicit and implicit meanings conveyed in the text.
Comic strips have also been used in ESL/EFL classrooms as well. Elkins &
Bruggemann (1971) introduced that American comic strips and cartoons can be useful in ESL classes. Many current popular comic strips have cultural, social, and political significance and discuss the American way of life. They encouraged ESL teachers to consider certain evaluative questions before using a particular comic strip. William (1995) investigated the effect of comic books as instructional materials on low–intermediate level ESL students. He found that comic strips can help ESL students gain a pragmatic awareness of English and guide them to become more active language learners. Liu (2004) conducted an experiment investigating the effect of comic strips on high and low-level ESL learners’ reading comprehension of high and low-level texts. The results of immediate recall protocols revealed that low-level students receiving high-level text with the comic strip performed significantly better than those receiving high-level text only. Liu (2004) explained that comic strip might lead students to notice the text’s linguistic input, and this interaction enhanced students’ comprehension and output.
Studies on the Effect of Sequencing of Visual Aids Presentation on Reading Comprehension
The sequencing of visual aid presentation on reading comprehension needs to be explored in the present study because few studies investigated this issue on EFL learners, and the conclusions determining the effect of visual presented before or after the text are still controversial in the previous studies. Empirical studies on the effective sequencing of visual aids presentation are reviewed as follows.
First, a plethora of empirical evidence supporting the usefulness of providing illustrated instructional material during reading has amassed over the years (Mayer, 1994). Levie and Lentz (1982) reviewed 23 studies which included comparisons of
learning from illustrated text and text-alone and found that the group mean for those reading illustrated text was superior to that of the group reading text alone.
Purnell and Solman (1991) conducted the experiments to examine the use of illustrations on L1 high school students’ reading comprehension. There were 3 groups—(a) text alone, (b) illustrations of the same content alone, (c) both the text and illustrations of the same content. The results indicated that comprehension performance was superior when the text and illustration of the same content presented together than repeatedly exposure to either the text or the illustration.
Gambrell and Jawitz (1993) investigated the effectiveness of visual aids among four groups – (1) text with induced mental imagery instruction, (2) text with story illustrations, (3) text with both of induced mental imagery instruction and story illustrations, and (4) text-alone. The participants were 120 fourth graders who read a 925-word story with 5 text-relevant illustrations. The result showed that Group Three significantly out-performed all the other groups followed by Group One, Group Two, and Group Four. They concluded that mental images and illustrations independently enhanced reading performance, and the combination of the two produced impressive increases in children’s comprehension and retention.
Mayer (1989) explored the effect of illustrations on scientific texts on vehicle braking systems. There were four groups: (1) labeled illustrations, (2) illustrations without labels, (3) labels without illustrations, and (4) no labels and no illustrations.
The results showed that students in the labeled illustrations group performed better on explanative information and problem-solving transfer.
Liu (2004) investigated the role of comic strips on ESL college students’ reading comprehension. He conducted an experiment of two (English proficiency: high, low) by two (text difficulty: difficult, easy) by two (comic strip support: with, without) factorial design, and measured students’ comprehension by written recall protocols.
There was only one significant difference revealed: low-level students performed better when receiving the high-level text with the comic strip than the other low-level students with high-level text only. However, the comic strip did not support comprehension when low-level students read easy texts or when high-level students read either easy or difficult texts. He concluded that visuals should reflect the text’s linguistic complexity, or else reading comprehension or recall can be hindered.
Omaggio (1979) conducted a study to examine the effect of pictures on L1 and L2 reading on university students. There were three groups: (1) students given a text in French (L2), with some of them receiving one of the pictorial contexts, (2) students given the same text in English (L1), with some of them receiving one of the pictorial contexts, and (3) students given a set of pictures related to the text. After reading, the first two groups were given comprehension tests, while the third was asked as a control to construct a story from the pictures. The results showed that pictorial stimuli had no effect on comprehension in the native language (English), but comprehension in French (L2) was improved by exposure to pictorial stimuli. The degree of improvement differed according to the pictures they received---The picture which depicted the scene from the beginning of the story was the most helpful. Her finding confirmed the long-assumed effectiveness of visual aids in second language study, and meanwhile indicated the importance of careful consideration in selecting visual aids.
The results of all the research above were consistent with Dual Coding Theory that retention and comprehension is better facilitated with both verbal and nonverbal codes activated than respectively. But what are reviewed above only attested the effect of during-reading condition. More studies showing the effects of different sequence of presenting visual aids are reviewed in the following.
Mayer and Anderson (1991) conducted a study on the reading of the operation of
a bicycle tire pump by college students. There were two groups. One was given a verbal description of that operation before the presentation of the illustration (words-before-pictures) while the other was given the verbal description during the illustration presentation (words-with-pictures). The results showed that words-with-pictures group outperformed the words-before-pictures group on problem-solving tests, but did not differ significantly on the performance of verbal recall test. In another study by Mayer and Sims (1994), they found that students who received concurrent presentation of illustrations and narratives performed better on the problem-solving test than students who received successive presentation (illustrations before/after narratives) or no instruction.
These two studies compared the during-reading condition with post-reading and pre-reading conditions and found that during-reading condition is more effective than the other two. Still, there are other studies comparing the effect of pre-reading and post-reading conditions.
Peverly (1981) investigated the effect of visuals, texts, and sequencing of the texts and visuals on readers’ retention of information. There were five conditions in Peverly’s study: (a) pictures before text (DT), (b) pictures after text (TD), (c) pictures simultaneously with text (ST), (d) the same picture twice (DD), and (e) the same text twice (TT). Free recall and short answer measures were used to assess reading comprehension of college students. The findings showed that the sequence of effect from the most robust to the least was TT, ST, DT, TD, and DD. The results not only confirmed the effect of simultaneous presentation of picture and text but also showed that presenting pictures before text had greater effect than presenting pictures after the text.
Visual aids are often used as pre-reading activities followed by pre-reading questions in English textbooks. They serve as a bridge to activate existing prior
knowledge and build new knowledge and to raise learners’ attention and motivation.
Visual aids commonly used in the pre-reading activities are pictures, cartoons, advance organizers, and diagrams. They help readers predict and overview the incoming information and are believed to facilitate reading comprehension. When the new material becomes relevant to students’ background knowledge, it is more easily encoded in their knowledge base (Perin, 1988).
Rivers and Temperley (1978) emphasized the importance of using illustrations with reading passages to provide additional meaning to texts. Using visual aids to provide appropriate background knowledge can not only display semantic content but also free low-level readers from lexical and syntactical difficulties. Hudson (1982) compared two pre-reading activities: 1) students viewed a set of cue pictures, and then discussed and made prediction of the text; 2) students learned some key words of the text in advance with a set of vocabulary cards. He found that the former type of pre-reading activity had a significantly greater facilitating effect on reading comprehension than the latter. Therefore, presenting visuals before reading is regarded as an effective way for activating background knowledge and facilitating comprehension.
However, Moore and Readence (1984) reviewed 23 studies of graphic organizers through quantitative and qualitative meta-analysis and concluded that graphic organizers in the post-reading position demonstrated greater effects than that in the pre-reading position.
Moreover, Brody & Legenza (1979, 1980) investigated the location of the picture (pre- or post-reading passage) and the type of picture (overview or specific incident) on reading comprehension. Ninety-two college students who were randomly assigned to pre-reading and post-reading group read a 1200-word passage and took multiple-choice questions to assess their comprehension. The findings of this study
supported the idea that the location of pictorial attributes was a significant instructional variable and students who viewed pictures placed after reading outperformed those who viewed pictures placed before reading.
Summary
To sum up, we can see that the effects of visual aids are positive and supported not only by Dual-coding theory but also by Schema theory. The former claims the facilitation of referential activation between verbal and nonverbal codes on reading comprehension while the latter advocates of building readers’ background knowledge of the following text by using visuals. Besides, many previous studies supported the facilitative effects of visual aids on reading comprehension.
However, from the review, whether visual aids are most effective when presented before the text, during the text, or after the text is still uncertain. During-reading condition is supported by most studies which can also be explained by Dual-coding theory. Peverly’s (1981) study found the group pre-reading visuals performed better than the group post-reading visuals and this result can be explained by Schema theory.
However, from the review of Moore and Readence (1984) and the results of Brody and Legenza’s (1979, 1980) study, visuals in post-reading position are more effective than in the pre-reading position. In order to help us understand this discrepancy, the researcher would like to explore if sequencing of visual aids presentation play a role on readers’ text comprehension.
Moreover, among the literature reviewed, comic strips, commonly loved by high school students, are seldom used to investigate their effect and the effect of sequencing on reading comprehension. The present study is therefore aimed to provide empirical evidence of supporting the effect of comic strip on EFL learners
and help determine the best sequencing of comic strips presentation. Most importantly, the results of the study are hopefully to inform the practitioners the effect of comic strips and the best sequencing of their presentation for instructional and practical use.
