CHAPTER THREE
3.2 Data Analysis
3.2.1 Analysis of rhetorical move structure of academic lectures
Although there have been many studies on move structures, there has not been a standard procedure that all researchers follow to conduct a rhetorical move analysis.
Connor et al. (2007) put forth a framework which provides clear principled guidelines in terms of how to describe and identify moves and steps of a genre. Although their
82
framework was originally created for computer-aided automatic move tagging, it nonetheless provides general guidelines informing researchers how a rhetorical move analysis can be conducted. Ten steps are included in Connor et al.’s framework regarding the steps of conducting a move analysis (p. 34):
1. Determine rhetorical purposes of the genre.
2. Determine rhetorical function of each text segment in its local context;
identify the possible move types of the genre.
3. Group functional and/or semantic themes that are either in relative proximity to each other or often occur in similar locations in
representative texts. These reflect the specific steps that can be used to realize a broader move.
4. Conduct pilot-coding to test and fine-tune definitions of move purposes.
5. Develop coding protocol with clear definitions and examples of move types and steps.
6. Code full set of texts, with inter-rater reliability check to confirm that there is clear understanding of move definitions and how moves/steps are realized in texts.
7. Add any additional steps and/or moves that are revealed in the full analysis.
83
8. Revise coding protocol to resolve any discrepancies revealed by the inter-rater reliability check or by newly ‘discovered’ moves/steps, and re-code problematic areas.
9. Conduct linguistic analysis of move features and/or other corpus-facilitated analyses.
10. Describe corpus of texts in terms of typical and alternate move structures and linguistic characteristics.
Connor et al. (2007) indicated that a move analysis typically starts with the researcher reading the target text multiple times to see the overall structure of the text and evaluate the function of each text segment and its local purpose to identify the boundaries between moves. It was also noted by Connor et al. that multiple readings and reflections of the texts are required before clear move categories emerge.
In addition to following Connor et al.’s (2007) suggestions regarding identifying general structure of text and determining boundaries between moves, the researcher also followed Lee’s (2016) suggestion of watching the target lecture videos for analysis before starting reading and annotating the corresponding transcripts. Lee maintained that such step would allow the researcher to actually see and hear how lecturers were making transitions in lectures, and to observe topic shifts achieved by physical movement (e.g., writing on the blackboard) or non-verbal features (e.g.,
84
gestures). By doing this, it would also be easier for the researcher to divide individual lecture transcripts into the phases of beginning, main body, and ending. Therefore, the researcher followed Lee’s suggestion and watched the entire lecture videos before moving on to read and divide the lecture transcripts into phases.
More specifically, how the researcher analyzed a lecture transcript was as follows: firstly, before starting marking any annotations to the corresponding transcript, the researcher had finished watching the video of the lecture for analysis.
When watching the lecture video, the researcher also took notes on the transcript to help the segmentation later. For example, lengthy pauses might signal topic changes.
By watching the lecture videos and noting these paralinguistic features that could not be seen from plain texts, it better helped the researcher to segment the lecture into different phases as well as to understood the functions the lecturer was trying to achieve with both language and non-verbal features.
Before moves and steps were identified, the lecture transcript had first been divided into three main phases: beginning, main body, and ending. When the researcher divided the lectures into three phases, the criteria used by Lee (2016) in identifying boundaries between lesson phases were also adopted to facilitate the segmentation process (p. 102):
1. Explicit linguistic reference to lesson shift
85
2. Changes in prosody plus physical movements
3. Lengthy pause plus a discourse marker (e.g. okay) produced with a falling tone
4. Lengthy pause plus non-verbal behavior (e.g., gesture, shuffling paper) By using the criteria and watching the lecture video, the researcher was able to identify the boundaries and divided the lecture transcript into three parts representing the three phases (i.e., beginning, main body, ending). When the researcher had trouble determining the boundaries, she reread the transcript and checked the lecture video again for segmenting the transcript into phases more properly. The same process was repeated until the twenty lecture transcripts were segmented into three phases.
In some cases, the boundaries between phases were not that clear. As such, the lecture videos in fact played an essential part in that they provided other non-verbal criteria for the researcher to identify the boundaries. For example, sometimes lecturers would move from the center of the stage to the side of the room to display information or slides from a computer, which often suggests that they are going to move on to the next topic. Other times, there would be a lengthy pause accompanying with the lecturer looking around the class, as a way to make sure that students have no questions, before the lecturer could start a new topic. As such, Lee’s (2016) criteria and the lecture videos allowed the researcher to separate the thirty lectures into the
86
three phases of beginning, main body, and ending before more detailed move/step category identification were carried out.
After the three phases of all the lectures were identified, the researcher followed the frameworks proposed in the previous studies on move structure of academic lectures (Cheng, 2012; Lee, 2009; Samraj & Petrovic, 2015; Thompson, 1994), as presented in Table 1, and Connor et al.’s (2007) guidelines of conducting a rhetorical move analysis to analyze and identify the recurrent move and step categories in the literature corpus and the CS corpus.
The coding process involved reiterative reading of the lecture transcripts, which was a recursive process where the researcher read each lecture transcript and coded its moves and steps. One transcript was reread and recoded multiple times until all the move and step categories appearing in the transcript were identified. The annotation of the current transcripts was conducted manually in their electronic form, instead of using a tagging system. As the real-time nature of spoken discourse is likely to produce embedded move structures (Thompson, 1994), if a tagging system is used to assign each segment to one move/step category, as how one usually works, the potential rhetorical flexibility may not be able to be captured (Bhatia, 1993). In some cases, one sentence could contain more than one move/step, so in MS Word, the researcher thus coded them using the “comment” function for annotation and for
87
recording the overlapped moves/steps, as showed in Figure 1.
Figure 1. An example of annotating moves/steps in MS Word
Before the whole data set was coded, the researcher had completed an initial pilot coding. As Connor et al. (2007) suggested that pilot coding is needed in order to fine-tune the coding protocol, two lectures from literature and computer science were randomly chosen from the corpora for the pilot coding. This step is especially
necessary to the present study in that the move and step categories of lecture introductions and closings were identified by previous studies based on lectures of various disciplines (Cheng, 2012; Lee, 2009; Thompson, 1994), but those of the main body of lectures were identified based only on lectures of law (Samraj & Petrovic, 2015). It is likely that the moves and steps identified in law lectures would differ from those in lectures of other disciplines to some degree. To ensure the move and step categories could be applied to analyze the move structures of the current literature and computer science lectures, a pilot coding was thus necessary. The pilot coding is expected to help the researcher to make some revisions on the moves and step categories if differences are found, and to further fine-tune the definitions of move
88
and step categories to code the rest of the transcripts more properly.
In the initial pilot-coding process, the researcher mainly based on the move and step categories identified in the previous studies to analyze the two lecture transcripts randomly selected from the literature and the CS corpora respectively. More
specifically, the researcher adopted Thompson’s (1994) and Lee’s (2009) move frameworks of lecture introductions as references to analyze and identify the move and step categories in lecture transcripts of the beginning phase. Samraj and
Petrovic’s (2015) framework was mainly utilized for analyzing the phase of the main body. For the ending phase, Cheng’s (2012) framework was used to identify move and step categories.
After the initial pilot coding, based on the occurrences of discrepancies and the emergence of new move/step categories, the researcher made some revisions to the existing move frameworks (i.e., Cheng, 2012; Lee, 2009, Samraj & Petrovic, 2015;
Thompson, 1994). Based on the revised working protocol, the researcher then coded the rest of the lecture transcripts in terms of their moves and steps. It should be noted that the coding process was recursive in nature because new move/step categories could emerge as more lectures were coded. Therefore, the framework would not be complete until the last lecture had been coded.
During the annotation process, many more additional move/step categories had
89
been identified. Therefore, the framework used as the coding protocol for the present study actually contain move categories of the modified ones from the existing move frameworks (i.e., Cheng, 2012; Lee, 2009, Samraj & Petrovic, 2015; Thompson, 1994), and the move categories identified additionally in the current study, as displayed in Table 3.
90
Table 3. The move framework as the coding protocol of the current study
Moves Steps
BEGINNING
M1: Warming up S1: Making a digression
S2: Housekeeping S3: Looking ahead
S4: Getting started / Greeting M2: Setting up lecture framework S1: Announcing the lecture topic
S2: Indicating the scope
S3: Outlining the lecture structure S4: Presenting the aims
M3: Putting the topic in context S1: Showing the importance of the topic S2: Relating “new” to “given”
S3: Referring to earlier lectures M4: Asking probing question or
raising issues for discussion M5: Showing topic interest MAIN BODY
M6: Announcing the topic
M7: Showing future course plan or lecture outline
M8: Defining the topic M9: Describing the purpose M10: Showing topic relevance M11: Telling asides or anecdotes
M12: Contextualizing the topic S1: Describing the topic S2: Explaining the topic S3: Interpreting the topic S4: Applying the knowledge S5: Demonstrating the process
S6: Highlighting important information S7: Referring to/reading the text
M13: Evaluating the topic and arguing for a viewpoint
M14: Referring to content discussed in previous lectures
M15: Interacting with students
M16: Summarizing the main points
91
M17: Pep talk
M18: Reminding students of course-related information
ENDING
M19: Concluding the lecture S1: Summarizing the content of the lecture S2: Indicating the end of a lecture
M20: Cooling down S1: Looking ahead
S2: Housekeeping
S3: Addressing non-course related matters M21: Farewell
Overall, the coding protocol contains a total of one moves and twenty-three steps. Compared to the move frameworks proposed by the previous studies (i.e., Cheng, 2012; Lee, 2009; Samraj & Petrovic, 2015; Thompson, 1994), this study has added two additional moves and one step to the beginning phase, nine additional moves and four steps to the main body phase, and further proposed a new framework of lecture endings through extensively revising Cheng’s (2012) framework of lecture closings, including revising her move-only framework into a dual-scheme framework of moves and steps, and adding new moves and steps.
More particularly, in the beginning phase, a new step, getting started, was added to move 1 to specify lecturers’ greeting and explicitly indicating the start of the class, which was actually ‘borrowed’ from Lee’s (2016) move framework of EAP classroom lessons. Two additional moves were also identified in this phase, which are move 4 (asking probing question or raising issues for discussion) and move 5 (showing topic interest).
92
In the main body phase, the researcher kept the four moves and three steps identified by Samraj and Petrovic (2015), but she revised move 3 step 1 describing a legal process into describing a topic, to better fit the current data. Also, to keep the
current move framework a move-step dual scheme, the researcher collapsed the two sub-steps illustrating the topic and analyzing the topic of the step applying the topic in Samraj and Petrovic (2015) into one step applying the knowledge. Moreover,
additional nine moves and four steps were further identified and included in the current coding protocol, which are: move 7 (showing future course plan or lecture outline), move 9 (describing the purpose), move 10 (showing topic relevance),
move11 (telling asides or anecdotes), move 13 (evaluating the topic and arguing for a viewpoint), move14 (referring to content discussed in previous lectures), move 15
(interacting with students), move 17 (pep talk), move 18 (reminding students of upcoming assignment), move 12 step 3 (interpreting the topic), move 12 step 5
(demonstrating the process), move 12 step 6 (highlighting important information), and move 12 step 7 (referring to/reading the text).
As for the coding protocol of the ending phase, it should be noted that the coding protocol for the current ending phase is quite different from Cheng’s (2012)
framework of lecture closings which contains three stages (pre-ending, ending, and post-ending) and twelve teacher’s moves. Originally, this study aimed to base on
93
Cheng’s (2012) move framework to help identify the move/step categories in the ending phase of the current lectures. However, the present study did not identify as many moves (or strategies as termed by Cheng) in the current lectures as Cheng did in her data, which is probably due to the rather limited corpus of the ending phase in the current study. Also, considered that the present study aimed to investigate the move structure of entire lectures, the pre-ending stage in Cheng’s framework could in fact be considered part of the main body phase. The researcher thus decided not to adopt Cheng’s three-stage classification of lecture closings in the present study.
Moreover, another issue with Cheng’s (2012) framework is that it actually does not include any “steps”. To provide a more detailed framework in terms of how moves are realized by different steps with different communicative purposes, the researcher further referred to the move structure of the closing phase proposed by Lee (2016) based on EAP classroom lessons to facilitate the current analysis. Finally, based on what was identified in the current data as well as the move frameworks in the closing phases identified in Cheng (2012) and Lee (2016), a revised framework of the ending phase was proposed by the present study.
Particularly, some moves identified by Cheng (2012) were revised into steps to form a general move and some moves and steps proposed by Lee (2016) were adopted to illustrate the move structure of the ending phase in the current lectures. More
94
specifically, move 19 step 1, summarizing the content of the lecture, is a combination of Cheng’s (2012) move categories of coming to a conclusion of lecture content and summarizing or reviewing key points. The category dismissing the class or leave-taking goodbyes and wishes in Cheng (2012) has been further divided into two steps
in the current protocol: move 19 step 2 (indicating the end of a lecture) and move 21 (farewell). Some strategies proposed in Cheng (2012) were given new move/step names following Lee (2016) to better capture the parallel features of moves across phases: indicating the plan for the future in Cheng (2012) has been revised into move 20 step 1 (looking ahead); Explaining course-related issues and raising questions about course related issues in Cheng (2012) have been collapsed as move 20 step 2
(housekeeping). Move 20 step 3, addressing non-course related matters, was also added by the current study to include the similar strategies in Cheng (2012): asking if students have questions, answering students’ questions, and explaining non-course related matters.
After the coding of the moves and steps in the lecture transcripts was completed, a second coder was invited to code and analyze a randomly-selected set of lecture data to establish inter-rater reliability (Nunan & Bailey, 2009). Kanoksilapatham (2005) highlighted the importance of inviting experts with relevant disciplinary background for cross-checking the coding of rhetorical moves and steps. The reason behind it is to
95
avoid potential misinterpretations due to the researcher’s unfamiliarity of the lecture tradition of the target discipline. Following Kanoksilapatham’s (2005) suggestion, a second coder with a background of computer science was invited for the coding process. The second coder is a current PhD student of computer science and a lecturer who has been teaching related courses in a university in Taiwan for more than six years.
The researcher first explained the coding scheme and the definitions of each move/step to the second coder before he was asked to code the transcripts. The
researcher also provided the second coder with examples of different moves and steps to help the second coder better understand how moves and steps realize their purposes in texts. The purpose of inviting a coder with computer science background was mainly to crosscheck the researcher’s interpretations of the lecture transcripts of computer science. Therefore, after the coding steps and coding protocol were
introduced, the second coder was asked to code a randomly-selected 20% of the total lecture transcripts of computer science. The number of disagreements between the researcher and the second coder was recorded for the calculation of agreement rate.
To be considered reliable, Nunan and Baliey suggest that more than 85% agreement between coders is acceptable (2009). The agreement rate between the researcher and the second coder was about 90%. For the codes that the researcher and the second
96
coder had disagreements with, they discussed the differences and reread the transcripts or watched the lecture videos until consensus was reached.
After the coding process was completed, a move framework containing all the moves and steps identified from all the lectures was generated. The total occurrences of each move and step were also calculated. To determine the conventionality of moves/steps, or their stability (Kanoksilapatham, 2005), the frequencies of each move and step in literature lectures and computer science lectures were recorded
respectively. Considering the small sizes of the current corpora and the small number of lecturers, a relatively strict approach was taken, following Lee’s (2016) suggestion:
a move/step was considered conventional if it appeared in 80% or higher of the lectures, but optional when it occurred in less than 80% of the lectures.
Based on the frequencies of the moves and steps, conventional and optional moves and steps of the two disciplinary corpora were then identified. The
conventional and optional moves/steps included in the two disciplinary lectures were then compared to examine whether there are disciplinary differences between the move structures of the two disciplines. How the two disciplines differed in terms of move structures were also further discussed.