CHAPTER 2 LITERATURE REVIEW
2.4 Major Findings and Research Gap
33
using different modalities did not show difference on factual lecture or reading comprehension in 30-minute delayed posttests (Horwitz, 2017; Kirkland, 2016;
Mueller & Oppenheimer, 2014).
On the contrary, the benefits of encoding have been proved to be more helpful when completing global processing tasks in the empirical study of Mueller and Oppenheimer (2014). In the posttest containing multiple-choice and short-answer questions, longhand participants outperformed their laptop counterpart on conceptual and application tasks. However, this superiority wasn’t significant in Kirkland’s (2016) study on lecture comprehension and Horwitz’s (2017) study on reading
comprehension. There was no difference in the performance between two groups with different note-taking modalities. One reason may be that conceptual comprehension was tested in multiple-choice questions in these studies. Another may be that listening and reading are two different information processing systems and that their results could not be directly compared. The generalization of the results from previous studies are still debatable and further research is therefore needed.
2.4 Major Findings and Research Gap
Note-taking, with its encoding and external storage functions, is generally considered an aid to learning. Notably, taking generative, non-verbatim notes that require learners’ inferencing creates meaningful learning and suggests stronger encoding benefits. In reading comprehension, Van Dijk and Kintsch’s (1983) model depicts multiple levels of meaning construction during reading. Actively engaging in reading, e.g., note-taking, is said to encourage deeper understanding such as situation
34
model to take place. As technology has been gradually incorporated into educational settings, a new issue considering note-taking modalities has emerged. Input
comprehension may be impacted because of the shift from handwriting to typing and the subsequent influence on cognitive processing. On the one hand, handwriting, with more kinesthetic engagement than typing, exclusively creates a sensory-memory trace that enhances learning and recall. On the other hand, the easiness of using a keyboard, the flexibility in terms of editing and the incomparable production speed still give typing the overall advantage over handwriting.
Regarding the comparison between longhand and laptop note-taking, note content analysis has revealed more words and verbatim overlap in laptop notes. In order to evaluate learning outcome from note-taking, various comprehension task types ranging from local to global processing have been used. Longhand note-taking also leads to better results in global-conceptual questions on lecture comprehension.
However, the findings in text comprehension did not show difference between two groups.
Previous research directly addressing the issue of longhand versus laptop note-taking either focus on lecture comprehension or fall short of speaking to real-world settings (Bui et al., 2013; Kirkland, 2016; Mueller & Oppenheimer, 2014). Respecting a L2 graduate school context, no study to date has investigated the potential different influences of longhand versus laptop note-taking on research paper comprehension.
The present research was designed in order to fill in this gap and perhaps provide insights for higher education teachers and learners.
35 CHAPTER 3 METHODOLOGY
The present research sets out to uncover the potential differences between longhand note-taking and laptop-based note-taking. How these modalities of spontaneous text note-taking impact reading comprehension, including local and global understanding, is also investigated.
Of the few previous studies on this topic that can be found, those that have been undertaken were mostly set in lecture conditions. Some were not natural in design, and participants were explicitly asked to take a certain type of notes (verbatim or organized) (Bui et al., 2013). Others assessments were limited to word-level recall (Lin & Bigenho, 2011). More recently, Mueller and Oppenheimer (2014) discovered the benefits of longhand over laptop note-taking in aiding conceptual understanding;
however, again this experiment was carried out under lecture settings.
Only one study to date has directly addressed this issue in reading
comprehension (Horwitz, 2017). However, no significant correlations were found in note-taking modalities or text comprehension. One reason may be that participants were not taking natural notes. They had been told that other participants would read their notes, and may therefore have taken more general notes instead of personally meaningful notes. In addition, reading and listening are fundamentally different, leading to inconsistency in test performance.
This research follows Mueller and Oppenheimer’s (2014) study by applying a similar procedure and comprehension test. The current experiment also takes Horwitz’s (2017) study into consideration by applying a similar reading
36
comprehension test. The goal of this research design is to form a better understanding of the impact of note-taking modalities on research paper comprehension. The
research methodology will be described in the following five sections: Section 3.1 begins by providing information about the participants; Section 3.2 describes the materials while Section 3.3 illustrates the instruments used in this study; Section 3.4 then outlines the procedure of data collection; Section 3.5 will provide insight into methods of data analysis; and finally, Section 3.6 summarizes the chapter and contains the author’s hypothesis.
3.1 Participants
The participants of the present study consisted of 30 graduate students from National Taiwan Normal University. Four participants were excluded; two because of not having taken any notes, and two because of not following the instructions. The majority majored in Teaching English to Speakers of Other Languages (TESOL) while others majored in linguistics; both MA programs were offered by the
Department of English. They were all foreign language learners of English. In order to apply for the TESOL graduate program, students had to reach at least B2 (Vantage) level of the Common European Framework of Reference for Languages (CEF). Score concordance comprised passing the high-intermediate level of General English
Proficiency Test (GEPT), getting more than 92 on the TOEFL iBT test or reaching 6.5 on the IELTS test. Participants from the Linguistics program in the present research have also reached the B2 level by passing these tests or receiving certain
certifications. During the training of their graduate study, English passages from
37
research papers or textbooks were selected as classroom materials. All lectures were also delivered in English. Moreover, in most courses, students were asked to deliver a presentation based on assigned or self-selected research papers. Before graduation, they were also required to either present their papers at an academic conference or pass a subject examination. To prepare for exams, students needed memorize passages and have a deep understanding of related research papers. In short, the participants were all similar in terms of English proficiency and were all familiar to reading English research papers.
During the present reading experiment, participants were randomly assigned to the longhand note condition or the laptop note condition, in which they used different modalities to take notes from the reading passage. Participants were between 22 to 30 years old in both groups.
3.2 Material and Design 3.2.1 Reading Source
Research papers were chosen as the target material for two reasons. First, participants in the present study were not only familiar with but were also motivated to read the research papers because, as previously mentioned, the research papers were closely related to graduate students’ study routine. Second, reading research papers may be more challenging and may highlight the functions of note taking.
Learners have been found to undergo deeper mental processing when dealing with more difficult tasks (Oded & Walters, 2001). Since research papers are more
complicated in nature and contain higher density of knowledge than common reading
38
materials, being actively involved in reading (i.e. taking generative notes in this case) may bring exceptionally positive outcomes.
Table 3
Information of the Reading Material Title of the
Research Paper
Parents and children in supermarkets: Incidence and influence
Authors Bill Page, Anne Sharp, Larry Lockshin, Herb Sorensen (2018) Total Words 6393 words
Abstract This research looks at the primary householder purchase context of grocery shopping and establishes the incidence of children accompanying adult shoppers. It identifies the effect of their presence on the spend, time taken to complete the trip and the route taken in-store. Observations are analyzed using exit interviews and structured observation of the in-store location of shoppers across two Australian states and four grocery retail outlets. Refuting the commonly held assertion that taking children shopping makes people spend more, accompanied shoppers do not spend more than unaccompanied shoppers, but rather shop 15% faster, tending to avoid busy areas in-store.
This has implications for store layout and services offered.
Products for children and parents need to be placed in areas where parents are more comfortable (that is, less busy areas), but also merchandised in ways that make it easy for parents to shop at their faster pace. The balance of these two needs is a direction for future research.
Following previous studies (Mueller & Oppenheimer, 2014; Slotte and Lonka, 1999), the criterion for choosing the reading materials was that the content be interesting but unfamiliar to as many participants as possible in order to prevent
39
different levels of understanding (Lindblom-Ylänne, Lonka, & Leskinen, 1996). The journal article Parents and children in supermarkets: Incidence and influence (Page, Sharp, Lockshin, & Sorensen, 2018), was thus selected from the Journal of Retailing and Consumer Services as the target reading text (Table 3). This article was chosen because participants majoring in TESOL and Linguistics have not taken courses on customer marketing. However, shopping in supermarkets is a part of daily life that everyone must have experienced so it would not be too difficult for the students comparing to subjects such as quantum mechanics or linear algebra. On top of that, participants would be familiar with its structure as a research paper. The reading text consisted of following sections: Introduction, Literature and Research Questions, Method, Results, Discussion and Implications, and finally Conclusions and Future Research. During the experiment, the title, names of the authors and the abstract were excluded from the text, leaving a remaining 6393 words in total. Moreover, the article found interesting and unpredictable results. Participants had to fully understand the text rather than rely solely on common knowledge to score high on the
comprehension test.
3.2.2 Design.
The present study used a pre-experimental, between-subject design, striving to examine the impact of note-taking modalities on reading comprehension and the differences between the contents of longhand notes and laptop notes. During the reading experiment, participants of different genders and from different programs were randomly assigned to the longhand note condition or the laptop note condition
40
(see Table 4). Thus, the between-subjects independent variables is that participants either took notes by laptop or by longhand (n=13/group). The dependent variables are first, quantitatively speaking, the number of factual and conceptual questions that participants answer correctly and the word count; and second, the qualitative note contents under two modalities.
Table 4
Information of the Participants
3.3 Instruments
3.3.1 Note-taking Instruments
During reading, participants in the longhand group took notes on provided B5-size loose-leaf paper with their own stationery. Personal pens with different colors were allowed in order to elicit natural note-taking habits. While previous research provided blank printer paper (Horwitz, 2017), the present study adopted loose leaf paper with embossed lines (see Figure 1). This decision was made as it is not easy to write accurately without lines, and some learners’ note-taking outcome may have been affected if blank paper had been used. On the other hand, this decision comes with a trade-off, as common ruled paper may limit learners’ note-taking strategies. It is
Grouping Numbers Gender Numbers Program_study Numbers Longhand
41
difficult to draw pictures or graphics when printed lines are used. Therefore, embossed paper, with invisible lines slightly raised or indented, created a condition where learners could not only take linear notes following the texture of the lines, but also draw charts or mind maps. More creativity in notes was hoped to be observed using embossed paper.
Participants in the laptop group took notes on a personal laptop. They were asked to type on a blank document of Microsoft Word (see Figure 2). All tools in Microsoft Word (e.g. color, font, typeface, etc.) were enabled in order to elicit natural note-taking habits. However, to prevent distractions, there was no access to the Internet and the participants were not allowed to use other applications on the laptop.
3.3.2 Reading Comprehension Test
Rather than including a pretest, there is only a post-reading test in this study.
Regarding the results from Horwitz’s (2017) study, more improvement was seen in factual questions than in conceptual questions. The reason being that more specific Figure 1. Loose leaf paper with
embossed lines used in the present study.
Figure 2. A blank Microsoft Word document used in the present study.
42
knowledge was needed to answer factual questions. Since participants generally received lower scores on a factual pretest, more room was left for improvement in the posttest. Therefore, comparing improvement on factual or conceptual questions is relatively unnecessary.
During the reading comprehension test, participants responded to twenty self-created multiple-choice questions in total (see Appendix A). The questions had been administered to a few populations with similar background to test the comprehension of the questions. The test included ten factual questions (question number 3, 4, 5, 6, 7, 8, 9, 11, 12 and 14) and ten conceptual questions (question number 1, 2, 10, 13, 15, 16, 17 ,18, 19 and 20). The test created by the researcher followed the definition of factual questions and conceptual questions from previous studies (Horwitz, 2017;
Muller and Oppenheimer, 2014). Factual knowledge of the text was evaluated with recall and definition tasks. For example, “What technique did the researchers use to investigate shopper movements through the store?” and “What does ‘basket size’ in the research mean?” Participants had to recall or explain specific terms to show their understanding of detailed information. On the contrary, the conceptual questions included examining the participants’ general understanding of the whole paper.
Conceptual-application tasks and comparison tasks were also included, for example, participants had to answer questions such as: “Why are the research questions important?” “How can the research findings help manufacturers and retailers?” and
“What may have caused the different results of the present research from the findings in Thomas and Garland’s (1993) study?” Being able to grab the main ideas of the passage, cause and effect of certain events, and compare and contrast between various
43
studies were all necessary in order to provide answers to the global questions.
Conceptual questions were specifically designed so that learners could not answer correctly relying solely on their prior knowledge. Each correct respond was given 1 point with a potential max score of twenty points.
In scoring the comprehension test, each multiple-choice question accounted for one point. The maximum score in total was twenty. The author scored all the
responses.
3.3.3 Leximancer System
The qualitative note contents were analyzed using the Leximancer system, a concept-mapping algorithm (see Figure 3 for example). In the sequential two-staged extraction of the texts (i.e., semantic extraction and relational extraction), the
Leximancer system took a step further than simply presenting word count. It could discover co-occurrence information, classify core concepts, provide a meaningful title for each concept, and present the relationship between each concept by analyzing the relative concept co-occurrence frequency. Below, the definition of certain terms in Leximancer will be defined.
Figure 3. An example of Leximancer processing.
44
The Leximancer User Guide (Leximancer Pty Ltd., 2018) defines the term Concept as follows:
Concepts in Leximancer are collections of words that generally travel together throughout the text. For example, a concept building may contain the keywords mill, warrant, tower, collapsed, etc. These terms are weighted according to how frequently they occur in sentences containing the concept, compared to how frequently they occur elsewhere. (p.9)
The Leximancer User Guide (Leximancer Pty Ltd., 2018) defines Concept Map as follows:
Aside from detecting the overall presence of a concept in the text, the concept definitions are also used to determine the frequency of co-occurrence between concepts. This co-occurrence measure is what is used to generate the concept map.(p.9)
The Leximancer User Guide (Leximancer Pty Ltd., 2018) defines Theme as follows:
The concepts are clustered into higher-level ‘themes’ when the map is generated.
Concepts that appear together often in the same pieces of text attract one another strongly, and so tend to settle near one another in the map space. The themes aid interpretation by grouping the clusters of concepts, and are shown as coloured circles on the map. (p.12)
In addition, with Leximancer’s patented algorithm, the Concepts in a text were first ranked by connectedness, i.e., how they co-occurred with other concepts (Leximancer Pty Ltd., 2013). Afterwards, starting from the top of the ranking, the
45
algorithm generated a Theme group based on the top concept. It then moved on to the Concept ranked next and either 1) put it into the nearest Theme group if the concept is near enough or 2) started a new Theme groups based on that concept. Therefore, Concept can be considered the micro-level while Theme is more of the macro-level.
3.4 Procedures of the Study
This section describes the procedures of the present study (see Table 5).
Participants completed the study in groups. Before the experiment, classrooms were preset either with loose-leaf paper or laptops according to the conditions. Materials presented in a pamphlet were placed aside each note-taking medium. Instructions were printed on the first page, followed by the research paper in the following pages.
Participants were instructed to read the article and take notes for an upcoming test.
They were asked to study as if they were preparing for a class. They were further reminded to use their natural note-taking strategy during reading, however, writing notes on the pamphlet was forbidden. The researcher read aloud the instructions and the participants could ask for clarification of the process. This introduction time took about 10 minutes.
The participants then turned to the second page of the pamphlet and started reading at the same time. They had 50 minutes to read the research paper and take spontaneous text notes.
46 Table 5
The procedures of the study.
After reading and taking notes on the research paper, all the participants had 30 minutes to finish the reading comprehension test. Each of them received a hard copy of the test and write their answers directly beside each question. Reading materials and notes were unavailable to the participants at this stage. The test sheets were collected were submitted for later analysis.
3.5 Data Analysis
3.5.1 Analysis of comprehension test.
To answer the first research question, after the scores were calculated in the comprehension test, they were measured through SPSS Statistics. A one-way multivariate analysis of variance (one-way MANOVA) was used to understand whether there were differences in performance in the comprehension test between note takers from the two groups. MANOVA was chosen over ANOVA as the tool since it could assess more than one dependent variables. The independent variable was note-taking modality (longhand versus laptop), whilst two dependent variables were test performances on local questions and global questions.
Introduction ORLaptop note-taking on a document of Microsoft Word
47 3.5.2 Analysis of note content.
To answer the second research question and understand the differences between longhand notes and laptop notes, word counts and note contents were measured.
Before content analysis, all longhand notes were transcribed into digital text format.
The relationship between word counts of the two modalities and reading test performance was evaluated using Pearson Product-Moment Correlation tests.
The relationship between word counts of the two modalities and reading test performance was evaluated using Pearson Product-Moment Correlation tests.