The present study investigates how EFL learners conceptualize the textual and visual connection in the eleventh grade English textbooks. The use of multiple modes in EFL learning has been promoted in recent decades (Ajayi, 2009; Jewitt, 2008;
Unsworth, 2005). Different modes enable EFL learners to attain meanings through different channels (Kress, 2000; Kress & Van Leeuwen, 1996). Yet, little research has investigated EFL learners’ views of bi-modality presented in English textbooks, the most accessible learning materials for Taiwanese students. The current study focuses on the Taiwanese high school students’ conceptions of textual and visual connection in English textbooks and the tie to their language proficiencies.
A Hierarchy of Conceptions
The first research question investigated high school students’ conceptions of textual and visual connection in English textbooks as well as the distribution of conceptions. Through a phenomenographic analysis of the students’ responses, five different categories of conceptions emerged and were ranked in hierarchical order, including Read for textual information only, Read with engaging visual aids, Read bi-modally for facts, Read bi-bi-modally to comprehend and Read bi-bi-modally and critically.
The students conveyed mixed views across different categories. The same student’s ideas were counted as one idea when they belonged to the same subcategories. After the calculation of ideas, the current study yielded 57 ideas in the outcome space. The first and second categories involve little modal awareness of bimodal texts while the third, fourth and fifth categories involve different levels of modal awareness and the use of bimodal resources. Table 2 presents the five categories, followed by the details of each category.
Table 2. Frequencies of Conceptions of Textual and Visual Connection in the Eleventh Grade English Textbooks.
Conceptions Definitions n Frequency (%)
Read for textual information
Read with engaging visual aids Affective responses to the lessons
13 22.8
(1) Being attracted by the visuals 7 12.3
(2) Getting interested in the visuals
5 8.8 (3) Feeling familiar with the
visuals
1 1.7
Read bi-modally for facts Using both textual and visual modes to retrieve
Read bi-modally to comprehend Using textual and visual modes to enhance comprehension
21 36.9
(1) Understanding the meanings 11 19.3
(2) Guessing the meanings 7 12.3
(3) Anticipating contents 3 5.3
Read bi-modally and critically Analyzing and 11 19.3
Table 2 shows that many students (36.9 %) perceived textual and visual
connection as “Read bi-modally to comprehend”. A considerable amount of students (22.8 %) expressed the view “Read with engaging visual aids” and still other students (19.3 %) conceptualized textual and visual connection as “Read bi-modally and critically”. A few students (12.3%) believed that the bi-modal connection in textbooks was used for capturing facts. The remaining students (8.7%) mentioned the textual mode only. To delineate each conception, the details will be presented in the following sections with students’ ideas. A sequential identifying number will be given to
represent each student. For example, “#1” refers to “number one” or “the first participant”.
(1) Read for textual information only
Read for textual information only concerns how the participants read with a text-oriented mindset. This category accounts for the least percentage among all (8.7%).
The subcategories are ‘Textual understanding’ and ‘Words memorization’. ‘Textual understanding’ refers to knowing words or the meaning of content through textual mode. The participants reported that they relied only on textual mode when reading textbooks. For instance, 3 out of 16 students stated that:
#1: I can’t see why the pictures are here. They do not help me read and understand.
evaluating of the bi-modal connection (1) Making connections between
textual and visual modes
7 12.3 (2) Evaluating the use of textual
and visual modes
3 5.3
(3) Organizing contents 1 1.7
Total 57 100
#6: The pictures are not helpful because I use Chinese translation to help me understand the paragraphs.
#14: I usually read the article first before noticing the pictures. ...Sometimes I ignore the pictures because I enjoy looking at the pictures only after I fully digest the article.
‘Words memorization’ refers to remembering the target vocabulary. For example, one student reported that:
#1: When I start reading the lesson, I look at the bolded words first…to memorize words such as "come from".
The student noticed the bolded words and memorized the phrase by decoding the textual information.
(2) Read with engaging visual aids
Read with engaging visual aids concerns that the participants were aware of and drawn to the visuals in the lessons. Thirteen out of 16 participants held this view, which constitutes 22 percent of the total conceptions. This category involves ‘Being attracted by the visuals’, ‘Getting interested in the visuals’ and ‘Feeling familiar with the visuals’.
‘Being attracted by the visuals’ refers to the view that the visuals help the participants fixating their attention to the lessons and that the visuals were eye-catching. For example, three out of 16 participants stated that:
#1: Whenever I read the textbook, I tend to notice the pictures ….because I don't understand the words and the pictures are more appealing to me.
#11: These pictures assist me and give me motivation to read the lesson.
#12: While I look at the pictures, I am attracted by them. And I want to continue learning the rest of the lesson.
‘Getting interested in the visuals’ refers to the fact that the participants became
interested in the lessons because of the visuals. For instance, 3 out of 16 participants considered that:
#2: I like Lesson 4 Twilight because of the movie and the pictures are interesting.
#5: Without pictures in the lesson, I will not even try to read because the lesson will not be interesting
#12: When I look at the picture, I want to continue reading to see what these pictures are about.
‘Feeling familiar with the visuals’ refers to the familiarity the participants had upon looking at the visuals. One participant out of 16 of stressed that the visuals were familiar to him. For example, one out of 16 participants held this view:
#7: The pictures of cartoon characters are familiar ….and are close to our lives.
(3) Read bi-modally for facts
Read bi-modally for facts emphasized the use of both textual and visual modes to retrieve facts in the lessons. The facts are specific information and factual knowledge such as settings and vocabulary. This category concerns the basic recall of the data previously presented. Seven out of 16 participants retrieve information by processing both the textual mode and the accompanying visuals. The subcategories include
‘Remembering contents’, ‘Recalling contents’ and ‘Retaining contents’.
‘Remembering contents’ refers to the fact that the participants read with the aids of textual and visual modes to capture and remember facts. Three out of 16
participants stated that when they read the lessons, they tended to look at the pictures to remember the content of the lessons. For instance, 3 out of 16 students said that:
#3: Pictures help me memorize things.
#7: Pictures help me memorize the settings, vocabulary or the content.
#12: It may be easier to memorize the content if every sentence is accompanied with a picture.
Recalling contents’ concerns the fact that the participants brought back what they have read back to mind through both modes. For example, 3 out of 16 students
reported that:
#4: Upon seeing the pictures, I know which story the lesson is about.
#5: Pictures help me recall my memories, especially stories.
#12: You can look at the pictures to recall what the lesson is about before taking tests.
‘Retaining contents’ refers to the situation where the participant was able to remember the reading contents for a longer period of time through both modes. For example, one student reported that:
#7: I need pictures to leave deeper impression about the lesson.
(4) Read bi-modally to comprehend
In Read bi-modally to comprehend, the participants employed both textual and visual modes to enhance their reading comprehension. This category, accounting for 36.9 percent of the total conceptions, concerns a higher cognitive level where both visual and textual modes are processed and connected with personal experiences to comprehend the readings and make meanings. In addition, this category emphasizes that textual and visual modes provide partial meanings and reading with the aid of the two modes enables the students to achieve full meanings. Three subcategories include
‘Understanding the meanings’, ‘Guessing the meanings’ and ‘Anticipating contents’, as presented in the following sections.
‘Understanding the meanings’ emphasizes using both visual and textual modes to achieve overall comprehension. For instance, 3 out of 16 students stated that:
#1: The pictures are helpful in understanding the content of the lesson. ....In
the beginning, pictures make you understand the meanings of words.
#3: Pictures guide you to read this article but they become supportive after reading.
#12: The pictures show me the details that are not indicated in the article.
The article may not have described a lot about their other features such as detailed descriptions of appearance or personality.
‘Guessing the meanings’ refers to taking guesses, making hypothesis and predicting based on visual clues. For instance, 3 out of 16 students reported that:
#9: The picture doesn't have the name of the person, but I guess his name is James based on the lesson article.
#15: The pictures are used to take a guess of the content before reading, and are used to assist me to read.
#16: The pictures are somehow giving me hints to the puzzle... I can start solving the puzzle after taking the hints.
‘Anticipating contents’ refers to the status where the participants prepared for what they might read later. For example, 3 out of 16 students stated that:
#1: Pictures may lead you into the article. It's like the beginning of reading.
#5: When it comes to the lesson, we don't want to read at all. But when there are pictures in a lesson, we may at least take a look at the content.
#10: I usually look at the pictures before I read…The pictures may have something to do with the title.
(5) Read bi-modally and critically
Read modally and critically concerns analyzing and evaluating of the bi-modal connection. This category, accounting for 20.3 percent of the total conceptions, features critical thinking, which is a higher cognitive level than the previous
categories. Twelve out of 16 participants read bi-modally to analyze the textual and
visual connection and find textual and visual connections. The subcategories involve
‘Making connections between textual and visual modes’, ‘Evaluating the use of textual and visual modes’ and ‘Organizing contents’. The following sections delineate each subcategory.
‘Making connections between textual and visual modes’ emphasized finding textual and visual ties to interpret the meanings combined. For instance, 3 out of 16 students stated that:
#3: I tend to keep reading to see if there is any picture that can match the descriptions of the article. Some pictures were matched with the content of the article. Others were little connection. And still others were quite different from what I previously guessed.
#4: I will look at the pictures before and during reading. For example, when I read the first paragraph and I don't quite understand some sentences, I will probably browse through the pictures in the lesson to see if there is any connection.
#13: I will see if there is any similarity or difference between pictures and the article. If there is a huge difference, I’d rather not keep reading....because I may feel bored when seeing the pictures and words are not matched.
‘Evaluating the use of textual and visual modes’ refers to involving
metacognitive strategies such as making analysis and judgment of the textual-visual connection. This subcategory concerns using metacognitive strategies. The
participants held the view that textual and visual modes were able to project partial meanings and that the meaning could be fully realized through interpreting both modes. Also, inconsistency between the textual and visual modes was identified and a brief evaluation of the bi-modal connection was reported. For example, 3 out of 16 students stated that:
#2: If I see the pictures, I will think that the lesson is about travel. Actually, these pictures have something to do with the lesson. But the lesson is mainly about jobs.
#5: Pictures like the ones about means of transportation assist me to compare and contrast them with the lesson content.
#14: The lesson is about the meanings behind the commercials. I think….I won't think of the commercials when seeing this picture. If I haven’t read the lesson title, I might think that the lesson is about where these two people go and have fun. The picture here is not matched with the reading content.
‘Organizing contents’ refers to leveraging textual and visual modes to sort out the reading contents. For instance, one student reported that:
#7: When there are lots of main ideas, I can organize the content by using pictures.
To have a full representation of the students’ conceptions, each student’s conceptions were counted for further analysis, as presented in Table 3.
Table 3. Each Student’s Conceptions
Categories Subcategories 1
(1) Being attracted by the
visuals Ⅴ Ⅴ Ⅴ Ⅴ Ⅴ Ⅴ Ⅴ 7
(2) Getting interested in the
visuals Ⅴ Ⅴ Ⅴ Ⅴ Ⅴ 5 (3) Feeling familiar with the
visuals Ⅴ 1
Read bi-modally (2) Evaluating the use of textual
and visual modes Ⅴ Ⅴ Ⅴ 3 (3) Organizing contents Ⅴ 1
Total 57
* A sequential identifying number for each student
The current study aims to explore differences in conceptions of textual and visual connection in English textbooks, rather than to emphasize on labeling individual student. Thus, based on the students’ responses to the interviewed questions, each student was allowed to be included in one or more subcategories to fully represent the students’ conceptions. For example, Student 14 had “Read bi-modally and critically”, the more advanced conception and “Read for textual information only”, the less developed conception.
The Influence of Proficiency on the Student’s Conceptions of Textual and Visual Connection in the Eleventh Grade English Textbooks
The second research question investigates how English proficiency regulates students’ conceptions of textual and visual connection in the eleventhgrade English textbooks. Of the total 16 participants, half of them were randomly recruited from the language gifted (LG) class while the other half were randomly recruited from the regular social science (RSS) class. From the monthly exam in October, 2016 to the monthly exam in March, 2017, the LG students have scored at least ten points higher than the RSS students. The monthly English exams included vocabulary, grammar, cloze and reading comprehension sections and the coverage included both lessons in the textbooks and outside readings. On average, the LG students demonstrated considerably higher English proficiency than RSS students.
Regarding the second question, some patterns of proficiency difference in regulating conceptions were revealed. Overall, the RSS group generated more conceptions than LG group, as presented in Table 4.
Table 4. Frequency of Conceptions in Two Groups
Conceptions RSS LG Total
Read for textual information only 7% (4) 1.8 (1) 8.8% (5)
(1) Textual understanding 5.3% (3) 1.8% (1) 7% (4)
(2) Words memorization 1.7% (1) 0 1.7% (1)
Read with engaging visual aids 12.3 (7) 10.5% (6) 22.8 %(13)
(1) Being attracted by the visuals 5.3% (3) 7% (4) 12.3% (7) (2) Getting interested in the visuals 7% (4) 1.8% (1) 8.8% (5) (3) Feeling familiar with the visuals 0 1.8% (1) 1.8% (1) Read bi-modally for facts 8.8% (5) 3.5%(2) 12.3% (7)
(1) Remembering contents 3.5% (2) 1.8% (1) 5.3% (3)
(2) Recalling contents 5.3% (3) 0 5.3% (3)
(3) Retaining contents 0 1.8% (1) 1.8% (1)
Read bi-modally to comprehend 21.1%(12) 15.8%(9) 36.9%(21)
(1) Understanding the meanings 10.5% (6) 8.8% (5) 19.3%(11) (2) Guessing the meanings 7% (4) 5.3% (3) 12.3% (7) (3) Anticipating contents 3.5% (2) 1.8% (1) 5.3% (3) Read bi-modally and critically 8.8%(5) 10.5%(6) 19.3%(11)
(1) Making connections between textual and visual modes
5.3% (3) 7% (4) 12.3% (7)
(2) Evaluating the use of textual and visual modes
3.5% (2) 1.8% (1) 5.3% (3)
(3) Organizing contents 0 1.8% (1) 1.8% (1)
Total 58% (33) 42% (24) 100% (57)
To find out the differences between two proficiency groups, the data are drawn from Table 4 and presented in Figure 1.
Figure 1. Two Groups’ Percentages of Conceptions
According to Figure 1, three major patterns were found. First, it is noticeable that RSS group generated more conceptions in all of the categories except for “Read bi-modally and critically”. Read bi-bi-modally to comprehend in RSS group accounts for 21.1 percent of the total conceptions, which is higher than that in LG group (15.8%).
Read bi-modally for facts in RSS group accounts for 8.8 percent of the total conceptions whereas the same category takes up 3.5 of the total conceptions. Read with engaging visual aids in RSS group (12.3%) is higher than that in LG group (10.5%) by 1.8 percent. Read for textual information only in RSS group takes up 7
7.0%
percent of the total conceptions whereas the same category in LG group accounts for only 1.8 percent. However, Read bi-modally and critically in LG group takes up 10.5 of the total conceptions, which is higher than RSS group (8.8 %).
Second, it can be observed that Read bi-modally to comprehend remains the dominant conception in both groups, followed by Read with engaging visual aids, whereas Read for textual information only remains the least percentage in both groups.
Third, the two groups have similar percentages in Read with engaging visual aids, which constitutes 12.3 percent of RSS group’s conceptions and 10.5 percent of LG group’s conceptions.
The second research question aims to investigate how different proficiency groups differ in their conceptions. In order to have a full understanding of how each group viewed the textual and visual connection in English textbooks, the following sections demonstrate the percentage of each conception in two groups, as presented in Table 5.
Table 5. Percentage of Each Conception in Each Group
Conceptions RSS LG
Read for textual information only 4(12%) 1(4.2%) Read with engaging visual aids 7(21.2%) 6(25%) Read bi-modally for facts 5(15.2%) 2(8.3%) Read bi-modally to comprehend 12(36.4%) 9(37.5%) Read bi-modally and critically 5(15.2%) 6(25%)
Total 33(100%) 24(100%)
To illustrate the distribution of conceptions in RSS and LG group, Figure 2 and 3, drawn from the data in Table 5, presented the percentage of each conception in two groups separately.
Figure 2. The Percentage of Conceptions in LG Group
According to Figure 3, it is significant that LG group tended to have more
advanced conceptions than lower level conceptions. LG group has more percentage in Read bi-modally to comprehend (37.5%) and Read bi-modally and critically (25%) than in other three categories. Read bi-modally for fact constitutes 8.3 percent of the LG’s conceptions. Read for textual information only, the lowest conception, takes up the least percentage (4.2%) among all. It is noted that Read with engaging visual aids takes up 25 percent of the LG’s conceptions.
37.5%
25%
25%
8.3%
4.2%
Figure 3. The Percentage of Conceptions in LG group