Instruments

40

(see Table 4). Thus, the between-subjects independent variables is that participants either took notes by laptop or by longhand (n=13/group). The dependent variables are first, quantitatively speaking, the number of factual and conceptual questions that participants answer correctly and the word count; and second, the qualitative note contents under two modalities.

Table 4

Information of the Participants

3.3 Instruments

3.3.1 Note-taking Instruments

During reading, participants in the longhand group took notes on provided B5-size loose-leaf paper with their own stationery. Personal pens with different colors were allowed in order to elicit natural note-taking habits. While previous research provided blank printer paper (Horwitz, 2017), the present study adopted loose leaf paper with embossed lines (see Figure 1). This decision was made as it is not easy to write accurately without lines, and some learners’ note-taking outcome may have been affected if blank paper had been used. On the other hand, this decision comes with a trade-off, as common ruled paper may limit learners’ note-taking strategies. It is

Grouping Numbers Gender Numbers Program_study Numbers Longhand

41

difficult to draw pictures or graphics when printed lines are used. Therefore, embossed paper, with invisible lines slightly raised or indented, created a condition where learners could not only take linear notes following the texture of the lines, but also draw charts or mind maps. More creativity in notes was hoped to be observed using embossed paper.

Participants in the laptop group took notes on a personal laptop. They were asked to type on a blank document of Microsoft Word (see Figure 2). All tools in Microsoft Word (e.g. color, font, typeface, etc.) were enabled in order to elicit natural note-taking habits. However, to prevent distractions, there was no access to the Internet and the participants were not allowed to use other applications on the laptop.

3.3.2 Reading Comprehension Test

Rather than including a pretest, there is only a post-reading test in this study.

Regarding the results from Horwitz’s (2017) study, more improvement was seen in factual questions than in conceptual questions. The reason being that more specific Figure 1. Loose leaf paper with

embossed lines used in the present study.

Figure 2. A blank Microsoft Word document used in the present study.

42

knowledge was needed to answer factual questions. Since participants generally received lower scores on a factual pretest, more room was left for improvement in the posttest. Therefore, comparing improvement on factual or conceptual questions is relatively unnecessary.

During the reading comprehension test, participants responded to twenty self-created multiple-choice questions in total (see Appendix A). The questions had been administered to a few populations with similar background to test the comprehension of the questions. The test included ten factual questions (question number 3, 4, 5, 6, 7, 8, 9, 11, 12 and 14) and ten conceptual questions (question number 1, 2, 10, 13, 15, 16, 17 ,18, 19 and 20). The test created by the researcher followed the definition of factual questions and conceptual questions from previous studies (Horwitz, 2017;

Muller and Oppenheimer, 2014). Factual knowledge of the text was evaluated with recall and definition tasks. For example, “What technique did the researchers use to investigate shopper movements through the store?” and “What does ‘basket size’ in the research mean?” Participants had to recall or explain specific terms to show their understanding of detailed information. On the contrary, the conceptual questions included examining the participants’ general understanding of the whole paper.

Conceptual-application tasks and comparison tasks were also included, for example, participants had to answer questions such as: “Why are the research questions important?” “How can the research findings help manufacturers and retailers?” and

“What may have caused the different results of the present research from the findings in Thomas and Garland’s (1993) study?” Being able to grab the main ideas of the passage, cause and effect of certain events, and compare and contrast between various

43

studies were all necessary in order to provide answers to the global questions.

Conceptual questions were specifically designed so that learners could not answer correctly relying solely on their prior knowledge. Each correct respond was given 1 point with a potential max score of twenty points.

In scoring the comprehension test, each multiple-choice question accounted for one point. The maximum score in total was twenty. The author scored all the

responses.

3.3.3 Leximancer System

The qualitative note contents were analyzed using the Leximancer system, a concept-mapping algorithm (see Figure 3 for example). In the sequential two-staged extraction of the texts (i.e., semantic extraction and relational extraction), the

Leximancer system took a step further than simply presenting word count. It could discover co-occurrence information, classify core concepts, provide a meaningful title for each concept, and present the relationship between each concept by analyzing the relative concept co-occurrence frequency. Below, the definition of certain terms in Leximancer will be defined.

Figure 3. An example of Leximancer processing.

44

The Leximancer User Guide (Leximancer Pty Ltd., 2018) defines the term Concept as follows:

Concepts in Leximancer are collections of words that generally travel together throughout the text. For example, a concept building may contain the keywords mill, warrant, tower, collapsed, etc. These terms are weighted according to how frequently they occur in sentences containing the concept, compared to how frequently they occur elsewhere. (p.9)

The Leximancer User Guide (Leximancer Pty Ltd., 2018) defines Concept Map as follows:

Aside from detecting the overall presence of a concept in the text, the concept definitions are also used to determine the frequency of co-occurrence between concepts. This co-occurrence measure is what is used to generate the concept map.(p.9)

The Leximancer User Guide (Leximancer Pty Ltd., 2018) defines Theme as follows:

The concepts are clustered into higher-level ‘themes’ when the map is generated.

Concepts that appear together often in the same pieces of text attract one another strongly, and so tend to settle near one another in the map space. The themes aid interpretation by grouping the clusters of concepts, and are shown as coloured circles on the map. (p.12)

In addition, with Leximancer’s patented algorithm, the Concepts in a text were first ranked by connectedness, i.e., how they co-occurred with other concepts (Leximancer Pty Ltd., 2013). Afterwards, starting from the top of the ranking, the

45

algorithm generated a Theme group based on the top concept. It then moved on to the Concept ranked next and either 1) put it into the nearest Theme group if the concept is near enough or 2) started a new Theme groups based on that concept. Therefore, Concept can be considered the micro-level while Theme is more of the macro-level.