This chapter begins with a summary of the major findings in the present study.
Second, according to the findings of this research, a number of pedagogical implications are addressed for junior high school English instruction and textbook compilation. Third, some limitations of the present study and suggestions for future studies are proposed.
Major Findings of the Study
The study sets out to investigate the degree of ideational and interpersonal complementarity in junior high school English textbooks in Taiwan. To that end, multimodal ideational and interpersonal analyses were conducted to analyze 18 texts in Book 5 of Joy and Kang-hsuan textbooks.
Overall, in these two editions of textbooks, ideational meaning was found to achieve slightly higher degree of complementarity in Kang-hsuan edition. Among all the six sense relations, repetition is employed most frequently and no tokens of antonymy were found, suggesting that visuals are inclined to faithfully illustrating the content of the texts. Meanwhile, represented participants and those process types involving activity and movement are presented more, showing that concrete items dominate visual illustration in these two textbooks.
Likewise, the same as ideational meaning, the degree of interpersonal
intersemiotic complementarity is higher in Kang-hsuan edition. However, the mean percentages in both editions are not high, indicating that there are gaps between the interpersonal messages delivered by the visuals and texts. Regarding the two ways of reinforcement of address, GM is used slightly more often than AP, but the difference
77
is modest.
Finally, the comparison between the two dimensions of meaning demonstrated that ideational meaning has higher degree of complementarity in both editions of textbooks than interpersonal meaning. One possible explanation of the relative low degree of interpersonal complementarity is textbook designers’ lack of awareness of interpersonal meaning. Visuals mainly present ideational meaning of text content, with the interpersonal meaning being ignored. Moreover, further investigation of genres manifests that genres may influence visual design and the way visuals and texts complement each other. Ample understanding of the degree of intersemiotic complementarity requires examination of genres and broader context of the texts.
Pedagogical Implications
Based on the research findings of this study, some pedagogical implications are proposed to in relation to second language teaching, learning and textbook
compilation.
For language teaching and learning, as the present study reveals, visuals, along with texts, contribute to the conveyance of meaning in textbooks. Ideationally, visuals often function to assist readers in visualizing the reading content and provide readers with examples, contexts and extra information. Interpersonally, visuals help readers enact various interactions with the texts. Nonetheless, oftentimes, what students encounter and what they are expected to read do not align. In classrooms,
mono-modal reading and interpretation of the texts still dominate what students are expected to do (Serafini, 2011). Written texts continue to be the main focus of
classrooms at the expense of visual images, despite their prevalence in students’ every life (Downes, 1995). Teachers, accordingly, need to broaden their conception of
78
literacy and go beyond traditional curriculum to find ways to incorporate multimodality into English classrooms and their teaching repertoires. Because multimodality is still a new language teaching approach, it should be included in language teachers’ in-service training. Teachers are required to be more
knowledgeable about reading skills, theories and strategies needed to facilitate students’ multimodal reading. Teachers could capitalize on all the potential semiotic resources co-present in the texts to enhance students’ language learning, critical think ability, multimodal literacy and multimodal communicative competence.
Multimodal literacy and multimodal communicative competence is suggested to be developed. Owing to the increased reliance on visual resources to deliver meaning, literacy and communicative competence are no longer sufficient if it is mono-modal.
Take reading skills, for example. Alternative multimodal reading strategies have been proposed that visuals could be the point of departure of students’ reading path. Instead of getting right into the verbal texts, students first attend to the visuals, identify what are presented, what perspectives are used and produce words or phrases to describe the elements in the visuals. After the identification of the elements, teachers could guide students to interpret what these elements are attempting to express or imply.
Based on students’ interpretation of the visuals and their prior knowledge, they can predict the content and exchange ideas with their classmates. Finally, students read the verbal texts to further confirm or contradict their prediction. As revealed in this study, visual-textual complementarity in some texts is more explicit and
straightforward; in some texts is more implicit and metaphorical. Teachers could design differentiated instruction based on students’ proficiency level, starting from explicit level to more implicit level. During the process, it is believed that a richer interpretation of the text can be rendered; meanwhile, students’ interpretative and
79
critical thinking abilities can be developed (Royce, 2002; Serafini, 2011). Besides reading, listening, speaking, and writing skills can also be developed with the application of multimodal texts, which potentially provides a holistic lens to inform the language instruction (D’Andrea, 2011).
In terms of ideational and interpersonal intersemiotic complementarity, salient variations were found to exist among lessons. As many semantic and interactive purposes could be fulfilled by visual images, visuals are not simply decorative or illustrative of the verbal texts, but a system of meaning potential and critical design elements in their own right. Textbook writers should devote due attention to visual design and strive to ensure the complementarity between the visuals and the texts.
In addition, while central to the present study is the issue of visual-textual
complementarity, the work serves as a reminder that language teaching should adopt a macro approach from the level of genre down to specific grammatical features.
Although the majority of texts in junior high school textbooks are dialogues and short passages, genre characteristics, structures and language use could be aptly introduced and taught. Texts in junior high school textbooks should set a model for students;
otherwise, students may end up having difficulties when required to write compositions in later educational stages.
Limitation and Suggestions for Future Research
Like most quantitative and qualitative research, this current study does have its limitations. Some theoretical and pedagogical aspects that may require further exploration are given in this section.
First is the restricted number of texts analyzed. Due to the scope and
exploratory nature of the present study, only 18 texts from one volume of two editions of junior high school English textbooks were analyzed. Owing to relatively small
80
sample size, only descriptive statistics was used to interpret the results. Further studies are suggested to include a larger number of texts from different volumes and editions of junior high textbooks to testify the findings of the present study. With more samples, inferential statistics could be employed to present a more
representative picture.
Second, this study only focuses on junior high school English textbooks in Taiwan. Although it is merely one type of textbook, it may point to a similar
investigation of the multimodal nature of other types of textbooks. Many features and characteristics discussed would call for further research to verify and confirm these findings. Interested researchers could extend their scope to investigate English textbooks for different educational stages (e.g. elementary school and senior high school), from different learning contexts or countries and even of different subject areas. Moreover, other multimodal teaching and learning materials commonly utilized in educational setting also open up wide arena for future studies.
Third, this study only deals with two dimensions of meaning, ideational and interpersonal. To put forward a more robust account of intersemiotic complementarity, future studies could include the compositional metafunction. Given that digital
technologies have made possible a vast variety of layout designs, comparison of the degree of intersemiotic complementarity among all the three dimensions of meaning holds great research potential.
Fourth, the present study arrives at the stage of examining and analyzing the complementarity between visual and textual modes in the English textbooks.
Experimental studies could be conducted in the future to empirically explore the effectiveness of implementing multimodal approach to teaching and learning English in EFL classrooms, evidence of which is still lacking in existing literature.
81
REFERENCES
Andrews, R. (2004). Where next in research on ICT and literacies?. English in Australia, (139), 58-67.
Anstey, M., & Bull, G. (2011). Teaching and learning multiliteracies (ebook):
Changing times, changing literacies. Curriculum Press.
Astorga, M. C. (1999). The text-image interaction and second language learning.
Australian Journal of Language and Literacy, 22 (3), 212-233.
Ajayi, L. (2008). Meaning-making, multimodal representation, and transformative pedagogy: An exploration of meaning construction instructional practices in an ESL high school classroom. Journal of Language, Identity, and
Education, 7(3-4), 206-229.
Ajayi, L. (2009). English as a second language learners' exploration of multimodal texts in a junior high school. Journal of Adolescent & Adult Literacy, 52(7), 585-595.
Baldry, A. (Ed.). (2000). Multimodality and multimediality in the distance learning age: Papers in English linguistics. Palladino editore.
Baldry, A., & Thibault, P. J. (2006). Multimodal transcription and text analysis: A multimedia toolkit and coursebook. London/Oakville: Equinox.
Barthes, R. (1964). The rhetoric of the image. In Howard, R. (translated by), ed. The responsibility of forms. Oxford: Blackwell, 21-40.
Barthes, R. (1977). Image music text. London: Fontana.
Bateman, J., Delin, J., & Henschel, R. (2007). Mapping the multimodal genres of traditional and electronic newspapers. New Directions in the Analysis of Multimodal Discourse’, Lawrence Erlbaum, Mahwah, NJ, 147-172.
82
Bateman, J. A. (2007). Towards a grande paradigmatique of film: Christian Metz reloaded. Semiotica, 2007(167), 13-64.
Bateman, J. (2008) Multimodality and genre: A foundation for the systematic analysis of multimodal documents. Basingstoke: Palgrave Macmillan.
Bell, P. (2001). Content analysis of visual images. Handbook of visual analysis, 10-34.
Bezemer, J., & Kress, G. (2008). Writing in multimodal texts a social semiotic account of designs for learning. Written communication, 25(2), 166-195.
Bezemer, J., & Kress, G. (2010). Changing text: A social semiotic analysis of textbooks. Designs for Learning, 3(1), 10-29.
Bowcher, W. L. (2007). A multimodal analysis of good guys and bad guys in
“Rugby League Week”. New directions in the analysis of multimodal discourse, 239-74.
Bowcher, W. L. (2012). Multimodality in Japanese anti-war placards. InMultimodal Texts from Around the World (pp. 217-245). Palgrave Macmillan UK.
Bowcher, W. L., & Liang, J. Y. (2013). Chinese tourist site entry tickets:
intersemiotic complementarity in an ecosocial process. Social Semiotics,23(3), 385-408.
Caple, H. (2009). 12 Multi-semiotic communication in an Australian Broadsheet: A new news story genre. PERSPECTIVES ON WRITING, 243-254.
Caple, H. (2013). Photojournalism: A social semiotic approach. Palgrave Macmillan.
Carter, R. (2001). Working with texts: a core introduction to language analysis.
Psychology Press.
83
Celce-Murcia, Marianne, Zoltán Dörnyei and Sarah Thurrell. (1997). Direct approaches in L2 instruction: a turning point in communicative language teaching?. TESOL Quarterly, 31(1), 141-152.
Chang, L. Y. (2006). Evaluation of grammar activities in junior high school English textbooks for Nine-year Integrated Curriculum. Unpublished master’s thesis, National Taiwan Normal University, Taiwan.
Chen, Y. (2009). Interpersonal meaning in textbooks for teaching English as a foreign language in China: a multimodal approach.
Chen, Y. (2010). Exploring dialogic engagement with readers in multimodal EFL textbooks in China. Visual Communication, 9(4), 485-506.
Chiang, H. W. (2013). A study of cultural content and cultural awareness in junior high school English textbooks. Unpublished master’s thesis, National Taichung University of Education, Taiwan.
Chu, P. Y. (2008). A study on grammar instruction in current junior high school English textbooks by junior high school English teachers in Kaohsiung City.
Unpublished master’s thesis, National Kaohsiung Normal University, Taiwan.
Cope, B., & Kalantzis, M. (2009). “Multiliteracies”: New literacies, new learning.
Pedagogies: An international journal, 4(3), 164-195.
D'Andrea, L. P. (2011). Using writing to develop communicative competence in the foreign language classroom. BELT-Brazilian English Language Teaching Journal, 1(2), 139-146.
Derewianka, B. (2003). Trends and issues in genre-based approaches. RELC journal, 34(2), 133-154.
84
Downes, T. (1995). Children and electronic media: the home-school connection.
In World Conference on Computers in Education VI (pp. 543-552). Springer US.
Early, M., & Marshall, S. (2008). Adolescent ESL students' interpretation and appreciation of literary texts: A case study of multimodality. Canadian Modern Language Review, 64(3), 377-397.
Eggins, S. (2004). Introduction to systemic functional linguistics. A&C Black.
Ellis, N. C., & Beaton, A. (1993). Psycholinguistic determinants of foreign language vocabulary learning. Language learning, 43(4), 559-617.
Eisenlauer, V. J. (2011). Multimodality and social actions in ‘personal
publishing’text: From the German ‘poetry album’to Web 2.0 ‘social network sites’. Multimodal studies: Multiple approaches and domains’, Routledge, London, 131-152.
Eisenlauer, V. (2013). A critical hypertext analysis of social media: The true colours of Facebook. A&C Black.
Fairclough, N. (2000). Multiliteracies and language. Multiliteracies: Literacy learning and the design of social futures, 162.
Fairclough, N. (2013). Critical discourse analysis: The critical study of language.
Routledge.
Fei, V. L. (2007). The visual semantics stratum: making meaning in sequential images. New directions in the analysis of multimodal discourse, Lawrence Erlbaumpp, Mahwah, 195-214.
Francesconi, S. (2011). Images and writing in tourist brochures. Journal of Tourism and Cultural Change, 9(4), 341-356.
85
Fries, P. H., Gregory, M., & Halliday, M. A. K. (1995). Discourse in society:
systemic functional perspectives (No. 50). Greenwood Publishing Group.
Giaschi, P. (2000). Gender positioning in education: A critical image analysis of ESL texts. TESL Canada Journal, 18(1), 32-46.
Gramley, S., & Pátzold, M. (2004). A survey of modern English. Routledge.
Guijarro, A. J. M. (2010). A Multimodal analysis of "The Tale of Peter Rabbit"
within the interpersonal metafunction/" Un análisis multimodal" de El cuento de Peter Rabbit desde una perspectiva interpersonal. Atlantis, 123-140.
Guijarro, A. J. M. (2011). Engaging readers through language and pictures. A case study. Journal of Pragmatics, 43(12), 2982-2991.
Guijarro, J. M., & Sanz, M. J. P. (2008). Compositional, interpersonal and representational meanings in a children's narrative: A multimodal discourse analysis. Journal of Pragmatics, 40(9), 1601-1619.
Gumperz, J. J. (1971). Language in social groups (Vol. 3). Stanford University Press.
Guo, L. (2004). Multimodality in a biology textbook. Multimodal discourse analysis:
Systemic functional perspectives, 196-219.
Guo, N. S., & Feng, D. (2015). Infusing multiliteracies into English language curriculum: The visual construction of knowledge in English textbooks from an ontogenetic perspective. Linguistics and Education, 31, 115-129.
Hagan, S. M. (2007). Visual/verbal collaboration in print: Complementary differences, necessary ties, and an untapped rhetorical opportunity. Written Communication, 24(1), 49-73.
86
Halliday, M. A., McIntosh, A., & Strevens, P. (1968). The users and uses of language. Readings in the sociology of language. The Hague: Mouton, 139, 170.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman.
Halliday, M. A. K. (1978) Language as a social semiotic: The social interpretation of language and meaning. London: Edward Arnold.
Halliday, M.A.K. (1985). An introduction to functional grammar. London: Edward Arnold.
Halliday, M. A. K., & Hasan, R. (1985). Language, context and text: Aspects of language in a social-semiotic perspective. Oxford: Oxford University Press.
Halliday, M. A. K. (1994). An introduction to functional grammar (2nd ed.).
London: Edward Arnold.
Halliday, M.A.K. and Matthiessen, C.M.I.M. (1999) Construing experience through meaning: A language based approach to cognition. London: Cassell.
Halliday, M. A. K. (2002). On grammar (Vol. 1). Bloomsbury Publishing.
Halliday, M. A. K., & Webster, J. J. (2008). Complementarities in language. The Commercial Press.
Halliday, M. A. K., & Matthiessen, C. M. (2013). Halliday's introduction to functional grammar. Routledge.
Han, W. Y. (2008). A study of the vocabulary size and recycling frequency in elementary school and junior high school English textbooks. Unpublished master’s thesis, National Taiwan Normal University, Taiwan.
He, C. Y. (2004). A comparative analysis of the new words in junior high school English textbooks of the nine-year consecutive curriculum. Unpublished master’s thesis, National Kaohsiung Normal University, Taiwan.
87
Herbele, V. (2010). Multimodal literacy for teenage EFL students. Cadernos de Letras, 27, 101-116.
Hibbing, A. N., & Rankin-Erickson, J. L. (2003). A picture is worth a thousand words: Using visual images to improve comprehension for middle school struggling readers. The reading teacher, 56(8), 758-770.
Hita, J. A., & López, J. L. (2001). The grammar of relational processes in English and Spanish: implications for machine-aided translation and multilingual generation. Estudios Ingleses de la Universidad Complutense, (9), 61-80.
Hsieh, Y. T. (2015). Using the Lexile Framework to analyze the readability of the reading passages in junior high school English textbooks in Taiwan.
Unpublished master’s thesis, National University of Tainan, Taiwan.
Hsu, M. Y. (2008). The effects of multimodal glosses on incidental vocabulary learning. Unpublished master’s thesis. National Taiwan Normal University, Taipei, Taiwan.
Hsu, P. C. (2003). A comparison of language learning activities in two sets of junior high school English textbooks. Unpublished Master’s thesis. National Taiwan Normal University, Taiwan.
Hsu, Y. P. (2010). Evaluating a set of junior high school English textbooks with criteria developed on the basis of CLT principles. Unpublished master’s thesis, Ming Chuan University, Taiwan.
Hsueh, K.H. (2012). Readability analysis and curriculum coherence of readings between junior high school and senior high school English textbooks in Taiwan.
Unpublished master’s thesis, National Taichung University of Education, Taiwan.
88
Huang, C. J. (2010). Content analysis of social class ideology in the junior high school English textbooks. Unpublished master’s thesis, National Taipei University of Education, Taiwan.
Huang, Y. L. (2012). A Study on the vocabulary in junior and senior high school English textbooks: Distribution, frequency, and overlapping. Unpublished master’s thesis, National Changhua University of Education, Taiwan.
Hull, G. A., & Nelson, M. E. (2005). Locating the semiotic power of multimodality.
Written communication, 22(2), 224-261.
Hung, L. C. (2008). Grammatical structure recycling in junior high school English textbooks for nine-year integrated curriculum. Unpublished master’s thesis, National Taiwan Normal University, Taiwan.
Hyland, K. (2001). Bringing in the reader addressee features in academic article.
Written communication, 18(4), 549-574.
Hyland, K. (Ed.). (2011). Continuum companion to discourse analysis. Bloomsbury Publishing.
Iedema, R. A. (2001a). Resemiotization. Semiotica: journal of the international association for semiotic studies. 23-39.
Iedema, R. (2001b). Analyzing film and television: A social semiotic account of hospital: An unhealthy business. Handbook of visual analysis, 183-204.
Iedema, R. (2003). Multimodality, resemiotization: Extending the analysis of discourse as multi-semiotic practice. Visual communication, 2(1), 29-57.
Jewitt, C., & Oyama, R. (2001). Visual meaning: A social semiotic approach.
Handbook of visual analysis, 134-156.
89
Jewitt, C. (2003). Re-thinking assessment: Multimodality, literacy and
computer-mediated learning. Assessment in education: Principles, policy &
practice, 10(1), 83-102.
Jewitt, C. (2005). Multimodality, “reading”, and “writing” for the 21st century.
Discourse: studies in the cultural politics of education, 26(3), 315-331.
Johnson, P. (1981). Effects on reading comprehension of language complexity and cultural background of a text. TESOL Quarterly, 15(2), 169-181.
Kalantzis, M., & Cope, B. (2012). Literacies. Cambridge University Press.
Kamil, M. L., Intrator, S. M., & Kim, H. S. (2000). The effects of other technologies on literacy and literacy learning. Handbook of reading research, 3, 771-788.
Kenner, C. (2004). Becoming biliterate: Young children learning different writing systems. Trentham Books.
Knox, J. (2007). Visual-verbal communication on online newspaper home pages.
Visual Communication, 6(1), 19-53.
Knox, J. S. (2009). Punctuating the home page: Image as language in an online newspaper. Discourse & Communication, 3(2), 145-172.
Kress, G. R., & Van Leeuwen, T. (1990). Reading images. Geelong. Victoria:
Deakin University Press.
Kress, G. R., & Van Leeuwen, T. (1996). Reading images: The grammar of visual design. Psychology Press.
Kress, G., & Van Leeuwen, T. (1998). Front pages: (The critical) analysis of newspaper layout. Approaches to media discourse, 186-203.
Kress, G. (2000). Multimodality: Challenges to thinking about language. TESOL Quarterly, 337-340.
90
Kress, G. (Ed.). (2001). Multimodal teaching and learning: The rhetoric of the science classroom. A&C Black.
Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. Edward Arnold.
Kress, G. (2010) Multimodality: A social semiotic approach to contemporary communication. New York: Routledge.
Labov, W., & Waletzky, J. (2003). Narrative analysis: Oral versions of personal experience. Sociolinguistics: The essential readings, 74-104.
Landert, D. (2014). Personalisation in Mass Media Communication: British online news between public and private (Vol. 240). John Benjamins Publishing Company.
Lemke, J. (1998a). Multimedia literacy demands of the scientific curriculum.
Linguistics and education, 10(3), 247-271.
Lemke, J.L. (1998b) ‘Multiplying meaning: visual and verbal semiotics in scientific text’, in J.R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspectives on Discourses of Science, pp. 87–113. London: Routledge.
Lemke, J. (2000). Multimedia demands of the scientific curriculum. Linguistics and Education,10 (3), 247-271.
Lemke, J. L. (2002). Travels in hypermodality. Visual communication, 1(3), 299-325.
Leonzini, L. (2012). Reading through the language of The Economist: an approach to analyze intersemiotic complementarity in multimodal discourse. Advances in Business Related Scientific Research Journal (ABSRJ).
91
Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2013). New literacies:
A dual level theory of the changing nature of literacy, instruction, and assessment. Theoretical models and processes of reading, 6, 1150-1181.
Lia, Y. L. (2013). A case study of young EFL learners‟ construction of digital stories: social semiotic theory of multimodality. Unpublished master’s thesis, National Sun Yat-sen University, Kaohsiung, Taiwan.
Liao, S. H. (2004). A study on the scope and sequence of pronunciation in junior high school English textbooks. Unpublished master’s thesis, National Chung Cheng University, Taiwan.
Lin, C. Y. (2008). An analysis of collocation instruction which integrates the corpora and the junior high school English textbooks-The case of verb-noun collocation. Unpublished master’s thesis, Ming Chuan University, Taiwan.
Lin, J. F. (2007). Comparisons of the readability among the six versions of English
Lin, J. F. (2007). Comparisons of the readability among the six versions of English