CHAPTER 3 METHODOLOGY
3.6 Summary and Hypothesis
47 3.5.2 Analysis of note content.
To answer the second research question and understand the differences between longhand notes and laptop notes, word counts and note contents were measured.
Before content analysis, all longhand notes were transcribed into digital text format.
The relationship between word counts of the two modalities and reading test performance was evaluated using Pearson Product-Moment Correlation tests.
As for the note contents, they were analyzed using the concept-mapping system, Leximancer, as mentioned in Section 3.3.3. In order to elicit more accurate results of the note concepts, obvious spelling mistakes and typos were corrected; for example, shooper into shopper, generlization into generalization, and etc. In addition, common abbreviations used among subjects were changed back into the original words; for instance, ppl into people, Ch or Cdn into children, bwn into between, yrs into years, and etc. With Leximancer, the core concepts and themes of longhand notes and laptop notes could be respectively discovered. The results from different concept maps would then be compared.
3.6 Summary and Hypothesis
The participants of the present study were graduate students from the linguistic program and the TESOL program. The goal of the experiment was to test the influences of note-taking modality (longhand versus laptop) on comprehending a journal article. Half of the participants took notes on loose leaf paper with embossed lines; the other half took digital notes on a blank document of Microsoft Word. Both their test performance and note content was subsequently analyzed.
48
Previous studies have shown that generative notes enhance the encoding process (Kiewra, 1985). While technology has been introduced to education settings, laptop use for note-taking has been found to result in more verbatim notes (Mueller &
Oppenheimer, 2014). Specifically, the more prominent impact from taking verbatim notes has been found to be upon conceptual knowledge as opposed to factual recall (Bretzing & Kulhavy, 1979). While these results were collected in listening note-taking conditions, findings from Horwitz’s (2017) research on reading comprehension showed no significant differences between longhand versus laptop note-taking
groups. However, considering the insufficiency in Horwitz’s (2017) posttest, it was still hypothesized that longhand note-takers would outperform laptop note-takers in reading comprehension in the present study, in relation to the first research question.
Moreover, considering the second research question targeting the quantitative and qualitative differences between longhand and laptop notes, word count was
hypothesized to be higher for the laptop group; in addition, it was hypothesized that laptop note-takers will take more verbatim notes during the learning process.
49 CHAPTER 4
RESULTS
Previous research has discovered the benefits of taking notes such as enhancing comprehension and information recalled (Armbruster, 2000; Bui, Myerson, & Hale, 2013; Peverly, Garner, & Vekaria, 2014). A few recent studies then tried to evaluate the effects of note-taking using pen-and-paper or laptops (Horwitz, 2017; Kirkland, 2016; Mueller & Oppenheimer, 2014). With inconsistence findings from Mueller and Oppenheimer’s (2014) study under lecture condition and Horwitz’s (2016) study under reading condition ahead, the overall purpose of the present study was to follow-up these studies to investigate whether longhand note-taking is more beneficial to reading comprehension. The current study focuses on the learning outcome after taking laptop or laptop notes during reading a piece of research paper. Not only their test performances but also their note contents and how they perceived the process of note-taking were reported. The results were often compared to findings from Mueller and Oppenheimer (2014) and Horwitz (2016) studies because the testing conditions and design are similar.
The aim of the present study is to investigate the character of note-taking during reading research papers, how different note-taking modalities (laptop and longhand) influence learning outcome and how are the two kinds of notes different quantitatively and qualitatively. Participants were graduate students from the Department of English in NTNU. They were all foreign language learners of English with similar language proficiency. During the experiment, participants first took notes while reading a piece of chosen English research paper. They then completed a reading comprehension test.
50
The reading passage and the notes were not available to the participants during the tests. The process, i.e. the encoding function, of note-taking is thus the focus of the present study.
This chapter is comprised of three sections. Results in response to the research questions will be thoroughly elaborated. Sections 4.1 reports the performance of two note-taking groups (laptop and longhand) on the reading comprehension test. Section 4.2 discusses the quantitative and qualitative differences between the contents of longhand and laptop notes. Finally, section 4.3 gave a summary of the overall results.
4.1 Which kind of note-taking modality (i.e., longhand or laptop) leads to better reading comprehension?
Participants were divided into two groups using different note-taking modalities, longhand or laptop. The longhand group took notes with pens on embossed line paper while the laptop group typed their notes in a Microsoft Word file. Post-reading
comprehension test were completed by all participants. The test consisted of twenty questions, including ten factual questions and ten conceptual questions. The
maximum score was twenty.
The effects of note-taking modality over the performance would first be examined using a One-way MANOVA and a One-way ANOVA. Afterwards, a Two-way
ANOVA were applied to evaluate the relationship between note-taking modality, conceptual performance and their influence on factual performance.
In order to evaluate whether participants’ individual background variables (i.e.
note-taking modality, gender and program studied) affect their performance on
51
reading comprehension and also to control overall significance level, a One-way Multivariate Analysis of Variance (MANOVA) was first applied to analyze the data.
When overall Wilk’s Lambda reached significant difference, a One-way Analysis of Variance (ANOVA) was then applied in both variables, i.e. factual and conceptual test performances, to investigate if there were any statistically significant difference between longhand and laptop note-taking groups. If the results were significant, Post Hoc test would be applied. When the variance between groups was homogenous, Scheffe’s Test would be applied. On the other hand, Dunnett’s T3 Test would be applied when the variance between groups was heterogeneous.
Participants’ individual background, i.e. gender and program studied, did not influence their test performance. No significant difference was found in participants with different genders (Wilks Λ(1,24) = 0.806, p > .05); no difference was found respectively in conceptual and factual questions either (Conceptual: F(1,24) = 2.351, p > .05; Factual: F(1,24) = 0.025, p > .05). Considering the programs the participants studied, test performance was not affected as well (Wilks Λ(1,24) = 0.939, p > .05;
Conceptual: F(1,24) = 0.001, p > .05; Factual: F(1,24)=1.215, p > .05).
Table 6 presents descriptive statistics results of the participants’ performance on reading comprehension. Table 7 shows inferential statistics of different groups.
According to Table 6 and 7, the overall MONOVA does not show significant
interaction between note-taking modality and overall comprehension test performance (Wilks Λ(1,24) = 0.992, p > .05). On factual questions, participants performed equally well in both conditions, (longhand: M = 7.692 , SD = 1.974; laptop: M = 7.539, SD = 1.713), F(1, 24) = 0.483, p > .0, which is consistent with the results of previous
52
studies comparing longhand and laptop note-taking effects (Horwitz, 2017; Kirkland, 2016; Mueller & Oppenheimer, 2014).
On conceptual questions, there was no significant difference between groups with different modalities as well, F(1, 24) = 0.94, p > .05). The longhand group (M = 7.923, SD = 0.954) had similar test performance comparing to their laptop
counterparts (M = 8.000, SD = 1.291), which is consistent with Kirkland’s (2016) results and Horwitz’s findings. However, the results from the present study is inconsistent with Mueller and Oppenheimer’s (2014) findings in which longhand note-takers outperformed their laptop counterparts in conceptual questions.
Table 6
Descriptive statistics of the participants’ performance based on note-taking modality and question type
Program study Numbers of
Participants Minimum Maximum Mean SD
Longhand Factual Qs 13 4 10 7.692 1.974
Conceptual Qs 13 6 9 7.923 0.954
Laptop Factual Qs 13 3 10 7.539 1.713
Conceptual Qs 13 5 10 8.000 1.291
53 Table 7
MANOVA Inferential statistics of participants’ performance based on note-taking modality and question type
Variable Wilks Λ
F
Conceptual Qs Factual Qs
Note-taking Modality 0.992 0.94 0.483
①Longhand ②Laptop
4.2 Are there any quantitative (i.e., word count) and qualitative (i.e., idea units) differences between longhand and laptop notes? If so, what are they?
4.2.1 Quantitative differences between longhand and laptop notes.
For the convenience of analysis, longhand notes were first transformed into digital form. Considering the total number of words produced, there is no significant
difference among participants with different background (gender: F(1,28) = 0.034, P
> .05; program studied: F(1,28) = 0.873, P > .05). In addition, according to Table 8, at first glance, the mean of laptop (M =206.20) is slightly higher than the longhand notes (M = 163.13). However, this word count difference between longhand group and laptop group was statistically insignificant.
54 Table 8
Note-taking modality and notes word count Variable Sample
Size Mean SD Levene
Statistics F P
Note-taking Modality 5.718 0.785 0.384
①Longhand 13 173.62 58.27
②Laptop 13 206.08 118.51
n=26
*p<.05 **p<.01 ***p<.001
Pearson Product-Moment Correlation tests were then used to investigate the relationship between note content (i.e., word count) and test performance (on factual questions and conceptual questions). The test combined the data from longhand group and laptop group so that the relationship between word count and test performance will be analyzed regardless of the note-taking modality. According to Table 9, word count and test performance (on factual questions and on conceptual questions) did not show statistically significant correlation using Pearson correlation tests, resulting in a correlation value of r = .012, p > .05 (word count and factual questions) and r = -0.109, p > .05 (word count and conceptual questions).
Table 9
Pearson Product-Moment Correlation of word count and test performance Word_Count Factual_Qs Conceptual_Qs
Word_Count 1 0.12 -0.109
Factual_Qs 0.12 1 .567**
Conceptual_Qs -0.109 .567** 1
55
While the results were inconsistent with findings from Mueller and
Oppenheimer’s (2014) listening note-taking research that participants who took more notes were reported to perform better, they replicate Horwitz’s (2017) results under reading condition that the correlations between word count and test performance were not significant. Earlier research investigating longhand note-taking during lecture condition had similar findings (Chaudron, Loschky, & Cook, 1994; Hsieh, 2006).
Both studies concluded that the total of words participants produced during note-taking could not predict their test performance.
4.2.2 Qualitative differences between longhand and laptop notes:
Leximancer content analysis.
While there were no significant differences in the word numbers and in their effect on post-reading comprehension performance, Leximancer, a concept mapping algorithm that can discover co-occurrence information, has presented quite different concept maps for the two kinds of notes comparing to the concept map of original study. Below, the data analysis results of the original study text, longhand notes and laptop notes will be displayed. In this section, the Themes and their co-occurring Concepts of the study will first be presented, and the results from the laptop notes and longhand notes will then be listed and compared.
4.2.2.1 Results of Themes from different materials.
Figure 4 shows the results of the original study Parents and Children in
Supermarkets: Incidence and Influence (Page, Sharp, Lockshin & Sorensen, 2018).
56
Noted that Theme circles are merely boundaries. The size of the circles does not indicate the importance or prevalence of a Theme. Moreover, according to the Leximancer User Guide (Leximancer Pty Ltd., 2013):
The size of a concept’s dot reflects its connectivity in the concept map. In other words, the larger the concept dot, the more often the concept is coded in the text along with the other concepts in the map. Connectivity in this sense is the sum of all the text co-occurrence counts of the concept with every other concept on the map.
In the map in Figure 4, the major eight Themes include children, shoppers, store, shopping, accompanied, in-store, number and wider. These are the prominent
concepts discussed in the study.
Figure 4. Leximancer map: Theme circles of the study text.
Laptop notes from different note-takers were assembled into a single Microsoft Word file and then underwent the operation of Leximancer. Themes of the laptop
57
notes are presented in Figure 5. The eight major are as follows: children, time, store, shoppers, in-store, requests, kids and survey.
Figure 5. Leximancer map: Theme circles of the laptop notes.
Finally, longhand notes that were transformed into digital form were assembled into one Microsoft Word file and underwent Leximancer analysis. Figure 6 shows the five major Themes of longhand notes: time, children, behavior, areas and space.
Figure 6. Leximancer map: Theme circles of the longhand notes.
58
At first glance, the map of the original text and the map of the laptop notes share more similarity as they both contain eight Themes when the theme sizes are set at 45%. In contrast, there are only five themes in the map of longhand notes. Also, original text and laptop notes share up to four identical concepts, children, store, shoppers and in-store, while the longhand notes Themes only include one identical concept, children, comparing to the original text.
Since the notes were taken during reading rather than after reading, the higher degrees of similarities between laptop notes and original texts may result from more acts of copying and typing exacts words by laptop note-takers. Therefore, it can be concluded that more verbatim notes were taken during laptop note-taking compared to longhand note-taking.
Figure 7. Leximancer map: Concepts of the study text.
59
4.2.2.2 Results of concepts from different materials.
Figure 7 shows the top-ranked Concepts of the Leximancer results from the original text. In the order of ranking, the top ten Concepts are children, shoppers, store, shopping, accompanied, research, trip, behavior, present, spend and etc., i.e., they are the keywords that travel together more in the study text.
Top-ranked Concepts of the laptop notes are presented in Figure 8. The top-ten Concepts are sequentially children, store, shopping, time, shoppers, spend, size, behavior, influence, and products. Comparing the top ten concepts of laptop notes and the original text, six of them are the same concepts: children, shoppers, store,
shopping, behavior and spend. Furthermore, when narrowed down to the top-five concepts, four out of five concepts of the laptop notes resemble those of the original text: children, shoppers, store and shopping.
Figure 8. Leximancer map: Concepts of the laptop notes.
On the other hand, according to Figure 9, the top-ten concepts of the longhand notes contain time, children, shopping, size, grocery, faster, navigation, maps, density
60
and in-store. Comparing the results of the original text and the longhand notes, only two out of the top-ten concepts are the same: children and shopping.
Figure 9. Leximancer map: Concepts of the longhand notes.
The observation of the Concepts has shown that the similarity between the results of the original text and the laptop notes are higher (sex identical concepts out of ten), compared to the results between the original text and the longhand notes (two out of ten).
In sum, it could be concluded that from both macro level (Theme) or micro level (Concept) observation, compared to longhand notes, the results of laptop notes shared more similarities with those of the original text. The findings in relation to the higher similarity could inferred that more verbatim notes were taken by laptop note-takers, i.e. they tended to copy and type in the exact words from the original text rather than putting the important points into their own words.
61
4.3 Summary of the Quantitative and Qualitative Results
To sum up, quantitatively speaking, participants from laptop note-taking and longhand note-taking conditions performed equally well in the comprehension test.
There was also no significant difference between these two groups of note-takers regarding both factual questions and conceptual questions. The longhand group had similar test performance with their laptop counterparts. In addition, considering the word counts of the notes of the laptop group and the longhand group, there were surprisingly no significant difference, Moreover, the number of notes taken did not influence test performance. However, qualitatively speaking, the content of the notes of the two groups are different. Not only are the Theme numbers (eight) identical between laptop notes and longhand notes, they share more similar Concepts as well.
Therefore, while it seems that the comprehensive results may not be different between two modalities, the notes taken were widely varied.
62 CHAPTER 5 DISCUSSION
Past research has established the effect of taking notes during a lecture or while studying (Armbruster, 2000; Bui, Myerson, & Hale, 2013; Di Vesta & Gray, 1972;
Peverly, Garner, & Vekaria, 2014; Mueller, & Oppenheimer, 2014). Research focus has then moved on to the effect of taking laptop notes and longhand notes during a lecture. The present research studies note-taking while reading, in this case research paper. It aims to discuss the differences of note-taking with two modalities: laptop or longhand, which one benefits the reading comprehension more and how the note-taking contents are different. There are limited studies that directly address the comparison of note-taking with laptop or longhand. Mueller and Oppenheimer’s pioneering study (2014) was done in the lecture situation where learners listened and took notes, and Horwitz’s study (2017) was done in the reading situation where learners read a textbook passage. The results of the present study will often be compared to Mueller and Oppenheimer’s (2014) and Horwitz’s (2017) since the experiment conditions were similar. Chapter five will be divided into two sections:
Section 5.1 addresses the relationship between note-taking modality and the learning outcomes; Section 5.2 goes further and discusses the content of laptop notes and longhand notes.
5.1 Note-taking and Reading Comprehension Test Performance
The first research question aimed to examine the performances of note-takers using different modalities in a reading comprehension test after they had finished
63
reading research paper and taking notes. It was hypothesized that longhand note-takers would outperform laptop note-note-takers in the comprehension test. However, the quantitative results from the comprehension test shows that there was no interaction between note-taking modalities and the overall comprehension performance. Nor was there statistical interaction between note-taking modalities and (1) factual question comprehension and (2) conceptual question comprehension. These results replicate Kirkland’s (2016) research in a lecture setting and Horwitz’s (2017) research in reading condition. However, in Kirkland’s research, there was a note studying session before the test. Thus, the results would not be compared with the present research. On the other hand, the present results were inconsistent with Muller and Oppenheimer’s (2014) finding that longhand note-takers had better listening comprehension
performance on conceptual questions comparing to laptop note-takers.
The main reason of such conflicting results may lie in the fundamental difference of audio and visual input (Lund, 1991). In a lecture condition, students are under more time pressure as they cannot go back to what they have missed while listening. They have to take notes as soon as possible. Since people write a lot slower than they write, they have to organize their thoughts into refined keywords as they write. On the contrary, when people take reading notes, they have less time pressure. They can go over the parts they don’t understand or consider important again and again. Some of them take notes whenever they encounter a salient idea while other may summarize the paragraph with a few words after each section. In other words, there are more choices of taking reading notes comparing to lecture notes. This may be one possible reason of the various results from the two conditions.
64
Moreover, in a lecture condition, note-takers have to transfer audio input into written notes, which requires a lot of mental efforts to accomplish the task. However, when note-takers take reading notes, they have visual aid from the reading passage so
Moreover, in a lecture condition, note-takers have to transfer audio input into written notes, which requires a lot of mental efforts to accomplish the task. However, when note-takers take reading notes, they have visual aid from the reading passage so