The major goal of the present study is to investigate the cohesive relations between textual and visual elements in the dialogues of elementary school English textbooks in Taiwan. Three phases of analysis, including textual analysis, visual analysis and intersemiotic analysis, were conducted to analyze 12 dialogues in Book 5 of Han-lin, Kang-hsuan, and Hess edition. Texts of the dialogue are analyzed according to Halliday’s (2004) Transitivity System. Pictures in the dialogues are explored based on Kress and Van Leeuwen’s (1996) Visual Grammar. Then, following Liu and O’Halloran’s (2009) Intersemiotic Cohesive Device, the degree of Intersemiotic Parallel Structure achieved between text and image is analyzed, including Parallelism, Partial Parallelism and No Parallelism. The results were summarized and discussed in the following six subsections: higher frequency of Relational, Material and Mental process of the texts, prevalence of Action and Reaction process of the visuals, predominance of Parallelism between the texts and visuals.
Higher Frequency of Relational, Material and Mental Process of the Texts In the present study, the results of textual analysis demonstrated that Relational, Material and Mental process are the three main process types in the dialogues of elementary school English textbook of the three editions. Behavioral process and Existential process are less frequent. Verbal process was not found in this study.
Among all textual processes, Relational process which refers to states of being, conditions or qualities ranks the highest percentage in the texts. The finding is consistent with a number of previous studies on junior high school English textbooks
69
(Ardini, 2012; Attar, 2014; Pertama, Rukmini, & Bharati, 2018), senior high school English textbooks (Anggun, 2016) and picture books (Hermawan & Sukyadi, 2017), which found that Relational process predominates the process type in the written text.
For example, Emilia, Moecharam and Syifa (2017) examined the gender issue in Indonesia junior high school English textbook and they found that Relational process was extensively used to depict male and female characters’ appearances and personality. Pertama, Rukmini, and Bharati (2018) found that Relational process is mainly used to provide information of certain object which facilitates junior high school students’ comprehension of English textbook. Overall, these findings suggested that Relational process show high frequency in English learning and reading materials. Possible explanation might be that English textbooks rely heavily on Relational process in order to describe the qualities or characteristics of the characters or to identify the objects in the written texts. Relational process which functions to characterize and identify can provide scaffolding for language learners to construct the knowledge and comprehend the texts (Lim, 2011).
Material process which deals with dynamism and actions ranked the second highest percentage of all process type in the dialogues. The result corroborated the findings of earlier research, which suggested that Material process is dominant in the texts of English textbook (Ardini, 2012; Attar, 2014; Anggun, 2016; Pertama, Rukmini, & Bharati, 2018) and picture books (Hermawan& Sukyadi, 2017).
Analogous to Ardini’s (2012) study on junior high school English textbook, Material process is extensively used in the descriptive and procedural reading texts to present the steps or the sequence of actions. In Anggun’s (2016) research on senior high school English textbook, Material processes are mainly applied to describe actions and behaviors carried out by the participants. In general, the results implied that
70
Material process was dominant in English textbooks and reading materials. One possible reason might be that the prevalence of Material process in the texts contributes to the development of the plot by describing concrete and physical actions carried out by the characters. Presnyakova (2011) also indicated that texts oriented toward elementary school students usually include more tangible process type in order to facilitate novice readers’ comprehension.
Mental process which concerned with human being’s experience of the world of consciousness is the third most frequently employed process in the dialogues.
Previous studies also yielded similar findings, which suggested that Mental process is prevalent in English textbooks of junior high school (Attar, 2014; Emilia, Moecharam
& Syifa, 2017; Pertama, Rukmini, & Bharati, 2018) and senior high school (Anggun, 2016). In Anggun’s (2016) research on senior high school English textbook, Mental process is used to describe how the participants feel in the descriptive texts. In line with Presnyakova’s (2011) research on English Language Arts textbooks, Mental process are frequently used interpersonally to construe the perceptions and thoughts of the participants involved in a conversation. Yet, slightly different from Guijarro &
Sanz’s (2008) research on a picture book, Mental process shows the highest frequency among all textual processes. Possible explanation may be that, the picture book under analysis draws on Mental process heavily (e.g. love, wish) to build the development of the story in which the characters show how much they love each other. In all, The results revealed that Mental process appeared frequently in some English textbooks in order to illustrate the characters’ perceptions, thoughts and feelings.
In the following subsections, two dominant visual processes in the dialogues:
Action process and Reaction process are discussed in detail.
71
Prevalence of Action and Reaction Process of the Visuals
In the current research, results of visual analysis revealed that there is a predominant use of Action and Reaction process in the elementary school English textbook among three editions. Analytical and Symbolic process are less frequent and show similar occurrence, whereas Classificational and Mental process are very infrequent in the present study.
Action process is the most dominant process type of all in the pictures of the dialogues. The finding is consistent with results of previous studies, which suggested that Action process is prevalent in the pictures of junior high school English textbooks (Attar, 2014; Pertama, Rukmini, and Bharati, 2018) and picture books (Guijarro and Sanz, 2008; Hermawan and Sukyadi, 2017). For example, in Hermawan and Sukyadi’s (2017) research on Indonesian picture books for children, Action process is mainly employed to describe main characters’ actions. The results also correspond to Attar’s (2014) study, which found that Action process in the pictures of the dialogues plays the role of visually presenting the action done by character’s body, gesture or tools in Iranian junior high school textbook. Overall, the predominance of Action process in the English textbook functions to illustrate physical, concrete and tangible action carried out by the characters, which may help readers to interpret the abstract concept of the written text (Pertama, Rukmini, & Bharati, 2018)
Reaction process which ranks the second highest percentage of all process type is realized almost as much as Action process in the present study. The relatively high occurrence of Reaction process is compatible with the findings of previous studies (Attar, 2014; Guijarro & Sanz, 2008; Hermawan& Sukyadi, 2017; Pertama, Rukmini,
& Bharati, 2018). Analogous to the results of Guijarro and Sanz’s (2008) research on children’ narrative story, there is a predominant use of Reaction process since the plot
72
is developed by eyelines between the characters in order to convey their love to each other. In Pertama, Rukmini, and Bharati’s (2016) study on junior high school English textbook, they found that Reaction process was extensively used to create the interaction between the characters. In general, the results implied that Reaction process was dominant in English textbooks. Possible reason might be that the Reaction process in the image depicts the bidirectional interaction between the characters, which may facilitate readers’ understanding of written text and development of the plot (Hermawan& Sukyadi, 2017).
In the next subsection, the results of Intersemiotic Parallel Structure are discussed in detail.
Predominance of Parallelism between the Texts and Visuals
The results of intersemiotic analysis showed that Parallelism was predominantly achieved in the dialogue of elementary school English textbooks among Han-lin, Kang-hsuan, and Hess edition, followed by Partial Parallelism and No Parallelism.
The results are discussed in the following parts.
The finding showed that Parallelism is predominantly used in the dialogues of elementary school English textbooks among the three editions. The relatively high occurrence of Parallelism in the English textbooks contributes to cohesive ties between different semiotic resources in the dialogues and further assist readers’
comprehension of the texts. The result is consistent with several previous studies (Attar, 2014; Galván, 2019). For instance, in Attar’s (2014) research on Iranian Middle school English textbooks, Parallelism between textual and visual modes showed high frequency in the reading texts. The finding also supported previous research (Ajayi, 2009; Cappello & Walker, 2016; Hibbing & Rankin-Erickson, 2003;
73
Royce, 2002), which suggested that connecting the visual and textual mode effectively may help readers to set up the context, provide pertinent background information and promote their understanding of the reading texts. Since different modes have different affordances, the combination of these two modes can better present the full meanings of texts and facilitate readers’ reading comprehension (Kress, 2000; New London Group, 1996; Unsworth, 2001).
According to Liu and O’Halloran (2009), Parallel Structure plays an important role in developing a comprehensible and meaningful text. The relatively high occurrence of Parallelism in EFL textbooks may be related to learners’ cognitive development and proficiency level (Paivio, 1986). It’s difficult for novice language learners to deal with linguistic information in the text. They cannot easily activate their prior knowledge and construct mental image merely based on textual information because of limited cognitive ability and language proficiency. Thus, Intersemiotic Parallel Structure is especially important for these learners. In addition, novice readers will try to find parallel meanings between visual and textual elements when they read (Short, 1996; Mayer & Anderson, 1992; Mayer, 2004). When the participant and process in the image are identically represented in the written text, learners may be able to make stronger connection between the texts and images and visualize the reading materials through repeated information multimodally (Hibbing
& Rankin-Erickson, 2003; Mayer & Anderson, 1992; Rowsell & Hamilton, 2012) The findings also revealed that Partial Parallelism achieved is relatively lower than Parallelism in the dialogues of elementary school English textbooks among these three publishers. No Parallelism takes up the least percentage among all editions. The result is in line with previous studies (Ajayi, 2009; Bezemer & Jewitt, 2010;
Guichon& McLornan, 2008; Jewitt, 2008; Mayer, 1993; Mayer & Moreno, 2003).
74
Possible explanation may be that it is difficult for readers to comprehend the content if the information in the written texts is not identically represented in the pictures (Mayer & Anderson, 1992). The texts which show lower degree of connection between different modes may cause readers’ exceeding cognitive load and further impede reading comprehension (Bezemer& Jewitt, 2010; Guichon& McLornan, 2008;
Jewitt, 2008; Royce, 2007).
Generally, the findings corroborated previous studies, which proved the importance of cohesive relation between textual and visual mode. The present study also extended the line of multimodal research through examining different educational field (i.e. elementary school English textbook) as well as different bimodal context (i.e. dialogues).
Summary of Discussion
The present study sets out to examine the textual process, visual process as well as Intersemiotic Parallel Structure in the dialogues of elementary school English textbooks in Han-lin, Kang-hsuan, and Hess edition in Taiwan.
In terms of textual process, Relational, Material and Mental process shows high frequency in the texts of the dialogues in the three editions. English learning and reading materials heavily draw on these three textual processes, which contributes to the development of the plot by characterizing and identifying, describing concrete and physical actions, and illustrating the characters’ mental state.
Concerning visual process, Action and Reaction process are prevalent in the visuals of the dialogues in the three editions. The finding is related to the fact that English learning materials rely heavily on Action and Reaction process in order to illustrate characters’ concrete actions and bidirectional interaction, which facilitate readers’ interpretation of abstract concept and reading comprehension.
75
With respect to Intersemiotic Parallel Structure, Parallelism between the texts
and visuals is predominant in the dialogues of the three editions.
Possible explanation for relatively high frequency of Parallelism in EFL textbooks may be associated with readers’ cognitive development and proficiency level.
Pedagogical Implications
According to the findings of the present study, some pedagogical implications are proposed in relation to second language teaching and learning.
First of all, following the prevalent trend of multimodal text, language pedagogy for reading comprehension should be broadened to understand not only linguistic knowledge but also pictorial information. Developing new reading skills and strategies which involves the combination of all modes in multimodal texts to derive meaning is needed. EFL instructors should explicitly teach EFL learners to take advantage of the combination of both textual and visual elements in the reading materials during the meaning-making process. EFL learners should be able to build connections among various elements in the texts and utilize reading strategies to establish the meaning of texts through the references of pictures and further facilitate multimodal reading. Additionally, since the present study suggested the importance of connection between texts and visuals, language reading and learning materials with stronger visually-verbally coherence are suggested to promote EFL learners’ overall understandings and further expedite their learning.
Limitation and Suggestions for Future Research
The results of the present study were subject to some limitations. To begin with, restricted sample size is the possible limitation of the study. Only 12 dialogues of
76
elementary school English textbooks from three editions were analyzed. Because of the relatively small sample size, only descriptive statistics was utilized to interpret the results. Future research is suggested to include more dialogues from different editions to examine if there are other factors which influence the different distribution of textual process, visual process and cohesive relations between the text and picture in the English textbooks. Moreover, the present study only deals with one aspect of Intersemiotic Cohesive Devices, which is Parallel Structure. More multimodal studies on textbook could examine other Intersemiotic Cohesive Devices (e.g. Reference, Theme-Rheme Development, Given-New organization) to extend the research scope and provide insights for the future study.
Conclusions
The present study aims to investigate textual process, visual process as well as Intersemiotic Parallel Structure in the dialogues of English textbooks of elementary school among Han-lin, Kang-hsuan, and Hess edition. After conducting three phases of analysis, the present study found that Relational, Material and Mental process were the three main textual processes of the dialogues in the three editions. Action and Reaction process were the two prevalent visual processes of the dialogues.
Additionally, Parallelism was predominantly achieved in the dialogues among the three editions, followed by Partial Parallelism and No Parallelism. The findings proved the significance of coherence between different semiotic modes in a comprehensible and meaningful text, which corroborated the results of previous research. Future multimodal studies on textbooks are suggested to include more sample size from different publishers and explore other dimensions of cohesive relation between different modalities.
77
REFERENCES
Alesandrini, L., & Rigney, W. (1981). Pictorial presentation and review strategies in science learning. Journal of Research in Science Teaching, 18, 465-474.
Anggun, S. K. (2016). An analysis of descriptive text in english textbook using transitivity system (A case study of reading passages). Journal of English and Education, 4(1), 147-158.
Ardini, S. N. (2012). Genre Analysis on Reading Passages Grade VII English Textbooks. Journal Eternal English Teaching Journal, 1(1).Nguyen, T. H. (2012).
Transitivity analysis of heroic mother by Hoa Pham.
Attar, M. M. (2014). Inter-semiotic cohesion analysis of multimodal elements in Iranian English textbooks (Doctoral dissertation, University of Malaya).
Ajayi L (2012) How teachers deploy multimodal textbooks to enhance English language learning.
Barthes, R. (1964). The rhetoric of the image. In Howard, R. (translated by), ed. The responsibility of forms. Oxford: Blackwell, 21-40.
Barthes, R. (1977). Image music text. London: Fontana.
Bell, T. (1998). Extensive reading: Why? And how? The Internet TESL Journal, 4,Retrieved February 10, 2008, from http://iteslj.org/Articles/Bell-Reading.html Bezemer, J., & Kress, G. (2010). Changing text: A social semiotic analysis of
textbooks. Designs for Learning,3(1), 10-29.
Bowcher, W. L., & Liang, J. Y. (2013). Chinese tourist site entry tickets: intersemiotic complementarity in an ecosocial process. Social Semiotics,23(3), 385-408.
Brown, L. K., Angus, D. C., Marin, M. G., Balmes, J. R., Barker, A. F., Ewart, G., ...
& Nathanson, I. T. (1994). Teaching by principles.
Chen, Y., & Qin, X. (2007). Heteroglossic harmony: engagement resources in
78
multimodal EFL textbook discourse. Foreign Languages and Their Teaching, 12, 15-18.
Chen, Y. (2009). Interpersonal meaning in textbooks for teaching English as a foreign language in China: a multimodal approach.
Chen, Y. (2010). Exploring dialogic engagement with readers in multimodal EFL textbooks in China.Visual Communication,9(4), 485-506.
Chu, P. Y. (2008). A study on grammar instruction in current junior high school English textbooks by junior high school English teachers inKaohsiung City.Unpublished master’s thesis, National Kaohsiung Normal University, Taiwan.
Damayanti, I. L. (2014). Gender construction in visual images in textbooks for primary school students. Indonesian Journal of Applied Linguistics, 3(2), 100-116.
Daley, E. (2003). Expanding the concept of literacy. Educause Review, 38, 33-40. The Nebraska Educator – Pictures for Improving Reading Comprehension
Daly, A., & Unsworth, L. (2011). Analysis and comprehension of multimodal texts. Australian Journal of Language and Literacy, The, 34(1), 61.
Eggins, S. (2004).Introduction to systemic functional linguistics. A&C Black.
Eisner, E. (2002). The arts and the creation of mind. New Haven, CT: Yale University Press.
Emilia, E., Moecharam, N. Y., & Syifa, I. L. (2017). Gender in EFL classroom:
Transitivity analysis in English textbook for Indonesian students. Indonesian Journal of Applied Linguistics, 7(1), 206-214.
Evans, S. (2003). Graphic organizers for Japanese readers of expository texts in English. Language Research Bulletin, 18, 1-17.
79
Fahnestock, J. (2003). Verbal and visual parallelism. Written Communication, 20(2), 123-152.
Francesconi, S. (2011). Images and writing in tourist brochures. Journal of Tourism and Cultural Change,9(4), 341-356.
Fei, V. L. (2007). The visual semantics stratum: making meaning in sequential images.New directions in the analysis of multimodal discourse, Lawrence Erlbaumpp, Mahwah, 195-214.
Fries, P. H., Gregory, M., & Halliday, M. A. K. (1995).Discourse in society: systemic functional perspectives(No. 50). Greenwood Publishing Group.
Gambrell, B., & Jawitz, B. (1993). Mental imagery, text illustrations and Childers's story comprehension and recall. Reading Research Quarterly, 28, 265-276.
Giaschi, P. (2000). Gender positioning in education: A critical image analysis of ESL texts.TESL Canada Journal,18(1), 32-46.
Guijarro, J. M., & Sanz, M. J. P. (2008). Compositional, interpersonal and representational meanings in a children's narrative: A multimodal discourse analysis. Journal of Pragmatics,40(9), 1601-1619.
Guijarro,A. J. M. (2010). A Multimodal analysis of"The Tale of Peter Rabbit" within theinterpersonal metafunction/" Un análisis multimodal" de El cuento de Peter Rabbit desde una perspectiva interpersonal.Atlantis, 123-140.
Guo, L. (2004). Multimodality in a biology textbook. Multimodal discourse analysis:
Systemic functional perspectives, 196-219.
Hayashi, K. (1999). Reading strategies and extensive reading in EFL classes. RELC Journal, 30(2), 114-132.
Halliday, M. A., McIntosh, A., & Strevens, P. (1968). The users and uses of language.Readings in the sociology of language. The Hague: Mouton,139, 170.
80
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman.
Halliday, M. A. K. (1978) Language as a social semiotic: The social interpretation of language and meaning. London: Edward Arnold.
Halliday, M.A.K. (1985). An introduction to functional grammar. London: Edward Arnold.
Halliday, M. A. K., & Hasan, R. (1985). Language, context and text: Aspects of language in a social-semiotic perspective. Oxford: Oxford University Press.
Halliday, M. A. K. (1994). An introduction to functional grammar (2nd ed.). London:
Edward Arnold.
Halliday, M.A.K. and Matthiessen, C.M.I.M. (1999) Construing experience through meaning: A language based approach to cognition. London: Cassell.
Halliday, M. A. K. (2002).On grammar(Vol. 1). Bloomsbury Publishing.
Halliday, M. A. K., & Webster, J. J. (2008).Complementarities in language. The Commercial Press.
Halliday, M. A.K., & Matthiessen, C. M. (2013).Halliday's introduction to functional grammar. Routledge.
Henry, R., & Simpson, C. (2001). Picture books and older readers: A match made in heaven. Teacher Librarian, 28(3), 23-27.
Hermawan, B., & Sukyadi, D. (2017). Ideational and interpersonal meanings of children narratives in Indonesian picturebooks. Indonesian Journal of Applied Linguistics, 7(2), 404-412.
Hsieh, Y. T. (2015). Using the Lexile Framework to analyze the readability of the reading passages in junior high school English textbooks in Taiwan. Unpublished master’s thesis, National University of Tainan, Taiwan.
Jalilehvand, M. (2012). The effects of text length and picture on reading
81
comprehension of Iranian EFL students. Asian Social Science, 8(3), 329.
Jewitt, C., & Oyama, R. (2001). Visual meaning: A social semiotic approach.
Handbook of visual analysis, 134-156.
Jewitt, C. (2003). Re-thinking assessment: Multimodality, literacy and computer-mediated learning. Assessment in education: Principles, policy &
Jewitt, C. (2003). Re-thinking assessment: Multimodality, literacy and computer-mediated learning. Assessment in education: Principles, policy &