To explore how reporting the results of research is realized in Abstract, Results, and Discussion sections of RAs, the present study integrated genre analysis with corpus-based text analysis. The former is used to examine the organizational structure of the three sections of RAs in order to identify the common moves and move structures used to present results, while the latter is aimed at investigating the linguistic features of these moves by compiling an RA corpus and using software for a more quantitative analysis of text. This chapter starts with a discussion of the corpus compilation and criteria for selecting RA samples, followed by an explanation of the coding scheme developed in the present study. After that, the process of move coding and tagging is presented. The next section then reports how linguistic realizations of the moves of reporting results are analyzed with the help of computer software.
Finally, explication of the qualitative comparison of the discourse contexts where different moves can occur is presented.
Data Collection
The data for the present study include text of three main sections of research articles, namely Abstract, Results, and Discussion, which includes the Discussion section and/or other sections following Discussion or ending an RA. Forty-eight research articles reporting empirical research are selected from four prestigious journals, two in applied linguistics (AL) and two in computer science (CS).
The journals in applied linguistics are Applied Linguistics (AL) and TESOL Quarterly (TQ), two SSCI journals that are well-established and exert great influence on both research and pedagogy in this field. Both are published on a quarterly basis.
41
To ensure unity of the RAs selected, the first empirical study in each issue of the two journals from 2005 to 2007 was selected. As some issues focus more on reviewing books or presenting special topics with no empirical studies included, the issues before 2005, in a reversed manner, were selected for replacement in order to have 12 articles from each journal. An example of this is the third issue of TESOL Quarterly in 2005, which mainly deals with research and teaching perspectives toward pronunciation and which does not include any empirical studies. As a result, the first empirical study in the fourth issue in 2004, the latest issue before 2005, was selected to make up for the 2005 issue that lacks empirical studies.
The field of computer science is developing rapidly as computer technology keeps improving. Many journals in this discipline, therefore, are published on a monthly or more frequent basis. To retain unity in terms of time of publication, two major computer science journals that are also published on a quarterly basis were used.
The two journals in computer science are ACM Transactions on Computer-Human Interaction (CH) and ACM Transactions on Information and System Security (IS), the former emphasizing software, hardware and human interactions with computers, and the latter devoted exclusively to the study, analysis, and application of information and system security.
Sections for analysis were identified on the basis of the authors‘ uses of headings in the articles. There was no problem to identify Abstract; however, the Results and Discussion sections were sometimes troublesome. Some of them, according to Yang and Allison (2003), use conventional functional headings (e.g. Results), varied functional headings (e.g. Findings or Results and Discussion, corresponding to Results), and content headings that report the main findings but lack evident markers of results in the headings. The sections, in the present study, were first identified according to their functional headings; in other words, Abstract, Results, and
42
Discussion. As Yang and Allison (2003, 2004) pointed out, not necessarily all the RAs possess and/or end with a Discussion section, they may use varied functional headings such as Conclusion, Discussion and Conclusion, or Pedagogical Implications. As a result, the Discussion section and/or other sections that follow or end an RA were all included and treated as one Discussion section in the present study. Besides the combination of headings as proposed by Yang and Allison (2003), authors might use topic-related headings. Therefore, if the authors used topic-related headings within their RAs, the overall organization of the RA were carefully read and broadly examined before they are categorized into Results and Discussion sections.
As the main focus of the present study is to investigate how research findings are reported across the three sections of Abstract, Results, and Discussion, taking a corpus-based approach, these three sections were selected from the original electronic file (*.pdf) and converted into three text files (*.txt). Three corpora were then compiled: an Abstract corpus, a Results corpus, and a Discussion corpus, respectively, for further analysis. During the converting process, non-verbal data, such as tables, figures, pictures, charts, algorithms, or diagrams, were deleted as they could not be shown in pure text format; however, the titles and notes above or below the non-verbal data were retained so that in the phase of text analysis, where non-verbal data are located in the text could be identified. The treatment of data generally followed Sinclair‘s clean-text policy (1991), a process used to remove non-verbal graphics and other codes so as to keep the text unprocessed and clean of any other codes.
Data Analysis: Move Analysis
The next phase is a genre analysis of the various sections in concern. A coding scheme (see Table 3.1) was developed based on previous studies on these sections
43
that are reviewed in the last chapter. However, some modifications were made. First of all, only moves, but not steps, were used in the coding scheme. Second, as the present study aims to investigate the presentation of results across RA sections, all the results-related moves across sections, as proposed by previous studies, are included;
in other words, some moves may occur in one section but not in another. Such a coding scheme is thus more suitable for comparing the moves in the three sections so that common moves across sections and moves that are distinctive in only one of the three sections can be identified. The coding scheme was modified throughout the process of coding in hope of developing a more feasible system to reflect the information structure of presenting research findings in the three RA sections.
Table 3.1. The coding scheme applied for the analysis of RAs.
Abstract Results Discussion
Reporting raw data (numbers, graphs, tables, or figures) AR RR DR Interpreting Results and Findings
Interpretation and/or suggestions of obtained data. AI RI DI
Providing Reasons/Explanations for Results
Relationship with studies carried out in the past. AC RC DC
Indicating Limitation/weaknesses
Stating the limitations and shortcomings of the study. AB RB DB Indicating Implications/Applications
Pedagogical implications or possible applications derived from obtained results.
AA RA DA
Need/Suggestions for Future Studies
Stating possible suggestions for future studies. AF RF DF
44
As shown in Table 3.1, in total, ten possible moves can be used to represent different rhetorical functions of reporting results in the three sections. The code for a specific move was then tagged at the beginning of a part of text where a move starts.
A move may consist of one or more than one sentence. However, as English is itself a language with sophisticated sentence structures, it is possible to include more than one rhetorical function in a sentence by using complex sentence structures. Some studies coded such a sentence with two moves (Samraj, 2002), while some other studies coded the sentence with the most salient purpose (Yang & Allison, 2003). In the present study, for sentences with more than one rhetorical function, it was coded for its most conspicuous purpose; in other words, only the main move is coded. An inter-rater, a graduate student who has received training in academic writing at both undergraduate and graduate levels participated in the analysis with an inter-rater reliability of 89.4%.
With the tagging of the move codes onto the text of the three corpora, we then retrieved and calculated the frequencies of the moves as well as the move patterns in the three sections. Therefore, it was possible to examine whether the three sections follow the same move patterns or possess any distinctive move patterns. In addition, by using ―Word List‖ in AntConc, as shown in Figure 3.1, a natural language processing software, the obligatory and optional moves could be identified according to their frequencies and distributions in the three sections. AntConc is a free software that entails several data management functions that are helpful to text analysis. These functions include Concordance, Concordance Plot, File View, Clusters, N-Grams (part of Clusters), Collocates, Word List, and Keyword List.
45
Figure 3.1. AntConc to obtain list of move frequency in the corpus.
To examine move patterns, the moves of each section were listed in a text file, which were examined using the function of ―N-grams‖ under ―Cluster‖ in AntConc to identify possible move patterns or move cycles in each separate section, as shown in Figure3.2.
Data Analysis: Content Analysis
After examining texts from a more quantitative perspective, it is beneficial to investigate them qualitatively; in other words, to examine the contexts where moves of reporting research findings occur in the various sections of a single RA. For example, Swales and Feak (2004) indicated that reporting research findings is presented in different levels of generality across sections. For example, one excerpt taken from TESOL Quarterly could show this variation in levels of generality.
46
Figure 3.2. AntConc to obtain list of move patterns in the corpus.
//AR// Results indicated that when listening to speech with correct primary stress, participants recalled significantly more content. (TQ03, Abstract)
Subjects listening to Version A remembered a significantly greater number of ideas than subjects listening to Version B (p = .001) or to Version C (p = .02).
//RR// Subjects listening to Version A remembered significantly more main ideas than subjects listening to Version B (p = .001) or to Version C (p = .05) (TQ03, Results)
//DR// The mean scores for each experimental group on the recall task …:
Group A scored higher than Group C, which in turn scored higher than Group B. (TQ03, Discussion)
From the excerpt above, it can be seen that excerpts taken from one single study can show different levels of generality in the three sections. Research findings are stated in the most general way in Abstract as this section focuses only on the most vital information that attracts readers‘ attention. As for the Results section, the research data are presented in the most specific manner by providing statistical or
47
empirical evidence to describe and prove a certain phenomenon. The Discussion section draws conclusion from the data in a less specific yet not too general manner, namely between the former two sections mentioned before. Therefore, language use in reporting research findings across sections should reveal language use from specific (Results), to less specific (Discussion), and eventually to the general way (Abstract).
From this qualitative perspective, sectional variations and disciplinary variations were both identified. Therefore, when examining the data in the present study, a similar coding process would be adapted to the excerpt quoted above.
Data Analysis: Linguistic Realizations of Reporting Research Findings After the coding process, linguistic realizations, including high-frequency verbs, high-frequency modal verbs, lexical bundles, and use of voice, to report results were analyzed. Computer software AntConc was used to retrieve words or sentences with similar structures from the RA texts.
Data management, including frequency analysis, concordance, and move patterns, was carried out for the corpora of the three sections. For example, to get the high-frequency verbs used in these three sections; more specifically, what verbs and modal verbs are more frequently used to report research findings, ―Word List‖ was used to identify the frequency list of words. However, as some word forms may exist as both verbs and nouns, the concordance lines under the function of ―Concordance‖
were examined to ensure only the frequencies of verbs. In addition, ―N-grams‖ was used to examine the high-frequency word chunks, or lexical bundles, used in reporting results, as shown in Figure 3.3. For a more detailed description of how these words or phrases are used in the discourse contexts of the corpora, ―Concordance‖ provides the KWIC (key words in context) and shows the words and/or phrases occurring before
48
Figure 3.3. AntConc used to identify the lexical bundles in the corpus.
and after a specific word or word chunks. In this way, not only a list of the frequently used verbs and phrases to report results but also sentence structures and specific text contexts that accompany these words and phrases were obtained.
After the high-frequency words and lexical bundles were examined, the RA corpus was divided into three subcorpora of Abstract subcorpus, Results subcorpus, and Discussion subcorpus, which were respectively examined in terms of use of active versus passive voice in the main clauses by applying the NLP tool of Sentence Extractor provided on the website of Compleat Lexical Tutor (http://www.lextutor.ca/
tools/ex_sentences/). The tool separated all the sentences so that individual sentences could be identified in terms of use of voice (See Figure 3.4). Examination of active and passive sentences was done by hand, and each sentence was coded with either active or passive according to the use of verb used in the main clause.
49
Figure 3.4. Sentences of Abstract extracted by Sentence Extractor.
Data Analysis: Disciplinary Variations in Reporting Results
As the present study focuses on RAs from two disciplines, the final step was to compare the findings of both disciplines in hope of retrieving similarities as well as differences in terms of reporting research findings in the two disciplines of applied linguistics and computer science.
First of all, the RA corpus was divided into an AL subcorpus and a CS subcorpus, each consisting of 24 RAs. To examine disciplinary variation in reporting research findings, the frequency of moves, move patterns, and high-frequency verbs were analyzed following similar steps as mentioned earlier in this section. The results of analyses from the two subcorpora were then compared.
50
CHAPTER FOUR