This chapter introduces the methodology of the present study in detail. The content of this chapter involves different phases of the pilot study and the formal study. The first section presents the pilot study, and the second section introduces the formal study.
Pilot Study
The researcher of this study conducted a pilot study to clarify whether the listening materials were feasible and teachable and if questionnaires were appropriate for
collecting necessary information for the formal study. As McKay (2006) explained, the necessity of a pilot study might be important for further research experiments. McKay claimed that the purpose of a pilot study was to find out difficulties and problems in the experimental design and which items could be confusing or misleading. Hence, the researcher conducted this pilot study which contained a pre-test, a note-taking activity and two questionnaires which gathered students’ background information and attitudes toward the activity.
Participants
There were 28 students in note-taking with a formal style (NTF) and 20 students in note-taking with personal styles (NTP). All of these 48 students were freshmen and majored in English in a southern university in Taiwan. They had been taking a listening course for one semester and did not participate in the formal experiment. The researcher selected the first group to receive the note-taking activity with a formal style and the second group to receive the activity with personal styles.
Instruments and Materials
In the pilot study, the researcher utilized a listening comprehension test (see
Appendix A) as the pre-test to divide them into different proficiency levels. The listening comprehension test was selected from Interactions 1 which is designed for intermediate learners to enhance their English listening and speaking. The length of the listening comprehension test was one minute; it contained 106 words for the pre-test and the content was related to a weather forecast. In the listening comprehension test, each item contained one point, and the total score was twenty. In the pilot study, the note-taking activity (see Appendix B), which contained 206 words, was selected from Interactions 1 for practice, and the length of the listening activity was one minute and 45 seconds.
In accordance with McKay (2006), a questionnaire provided an efficient method for a researcher to gather a deal of information in a short time with little cost. Furthermore, Freeman and Long (1997) indicated that “questionnaires are often used to get language learners to self-report their attitudes or personal characteristics” (p. 35). Hence, two questionnaires designed by the researcher were utilized to collect listeners’ feedback in the pilot study. The first questionnaire (see Appendix C) was designed to gather all participants’ background information and experiences of English learning, and the second questionnaire (see Appendix D) was designed to collect participants’ feelings and
attitudes regarding the first practice of note-taking activities.
Data Collection
In the pilot study, the researcher utilized a listening comprehension test, a
note-taking activity and two questionnaires to collect data from 48 students. Students in both groups had to take the listening comprehension test and fill out the first
questionnaire of background information before the note-taking activity, so that the researcher could realize their listening proficiency before conducting note-taking activities. After taking the listening comprehension test, 28 students in the first group received the note-taking activity with a formal style (NTF), which meant students had to read directions before listening. They used these tips to take notes to complete the activity and then filled out the second questionnaire. Twenty students in the second group
received note-taking activities with personal styles (NTP), which meant students did not read directions provided in the book before listening, but were encouraged to take notes with their personal style during the listening process. After listening, students in NTP used their notes to complete the activity and then filled out the second questionnaire.
Data Analysis
In the pilot study, the researcher found that the questionnaires could not collect some participants’ feedback toward the experiment; therefore, he decided to edit and add some questions in the formal study. Moreover, the researcher planned to add the third
questionnaire in the formal study because data which were collected from previous questionnaires were not sufficient to analyze. The results of the pre-test indicated that the difficulty of the listening contents was suitable for the freshmen in the pilot study since most students could answer the listening comprehension test and finish the following activity; thus, the researcher could utilize note-taking activities of Interactions 1 for his formal study.
Formal Study Participants
The participants in the formal study were different from the students in the pilot study. In the formal experiment, the researcher selected two new classes. The first class had sixty students, and the other class had fifty-six students, a total of one hundred and sixteen students in the formal experiment; all of them majored in English in a southern university in Taiwan. These students have been learning English since they were fifth graders, and therefore they have been learning English for over eight years. All students were freshmen in college and had received listening instruction for one semester. Sixty students in the first class received note-taking activities with a formal style (NTF), and fifty-six students in the other class received note-taking activities with personal styles (NTP). In each class, students were divided into three proficiency levels: high,
intermediate and low proficiency levels by the scores of the pre-test. Thus, in NTF, each proficiency group contained twenty students; in NTP, high and intermediate proficiency groups each contained nineteen students, and the low proficiency group contained eighteen students.
Instruments and Materials
In this study, listening materials and instruments were used to facilitate the
researcher to conduct the experiment. The process included one listening comprehension test, eight note-taking activities, and three questionnaires. The Interactions series
published by McGraw Hill were designed for English listening and speaking skills; thus, the researcher selected Interactions 1 Listening/Speaking for participants. There are four strands in Interactions series, and each of them was designed for different level learners.
The authors of Interactions series stated that language proficiencies are articulated from the beginning through advanced level within each strand. Hence, the researcher of this study selected listening activities and a listening comprehension test from Interactions 1 for the participants.
The Listening Comprehension Test
In the formal study, the listening comprehension test (see Appendix E) was utilized as the pre-test and post-test of this study because the researcher planned to measure the effects of note-taking activities using the same listening comprehension test. Each item contained one point, and the total score was fifteen. In addition, participants were allowed to take notes when listening.
Questionnaires
In order to collect all of the participants’ opinions, the researcher designed three questionnaires for the study. The first questionnaire (see Appendix C) focused on participants’ English learning experiences and background information. This questionnaire could help him to understand participants’ English learning in their previous schools. The second questionnaire (see Appendix D) collected participants’
attitudes toward the first note-taking activity. The third questionnaire (see Appendix F) aimed to survey all of the participants’ thoughts toward note-taking activities after taking the experiment of this study, and this questionnaire could facilitate the researcher to know more about participants’ thoughts toward the whole procedure.
Eight Note-taking Activities
In Interactions 1 Listening/Speaking, every chapter involves one note-taking activity for learners to practice the note-taking strategy. Therefore, the researcher of this study
employed those activities for the participants. The following table shows the length and words of each activity.
Table 4
Information on the Eight Activities
Topics Length Words 1. Living to eat, or eating to live? 2:34 343 2. Home
3. Cultures of the world 4. Health
5. Entertainment and the media 6. Social life
The experiment aimed to discover the effects of note-taking activities on students’
listening comprehension. The researcher of this study offered two types of note-taking activities for participants: (1) NTF meant participants read note-taking directions which advised them how to take notes and completed the listening activities; (2) NTP meant participants were encouraged to take notes in their own styles. Therefore, participants in this group had to use their own ways to take notes to complete the listening activities.
Because students’ scores on the listening comprehension test might be affected by their listening proficiency and types of note-taking activities the researcher of this study
provided, the independent variables were students’ listening proficiency and the two types of note-taking activities in this study. And the dependent variable was the score of the listening comprehension test after receiving note-taking activities.
Three open-ended questionnaires were utilized through the experiment in this study because participants’ background information and opinions might play an important role for the findings of the study. McKay stated: “Open-ended questions allow respondents to write in their own answers. They typically take one of two forms: fill-in or short answer”
(p. 37). Hence, the researcher designed three questionnaires in the experiment. The first questionnaire collected students’ background information in English learning. The second questionnaire surveyed students’ attitudes toward the note-taking activity because the researcher thought that their expectations and opinions were worth discussing. The third questionnaire aimed to investigate students’ thoughts after receiving note-taking
activities.
In the beginning of the formal study, a pre-test (see Appendix E) was used before the note-taking activities because the purpose of the pre-test was to define participants’
listening proficiency. Since this study also focused on the effects of note-taking activities on participants with different proficiency levels, the pre-test could assist the researcher in defining students’ proficiency levels as low, intermediate and high levels. A post-test which is the same as the pre-test was utilized after note-taking activities because the purpose of the post-test was to evaluate students’ performance after receiving activities.
The pre-test was taken in the first week of this formal experiment because the scores of the pre-test could help the researcher to classify students’ proficiency levels and record their initial scores before conducting the note-taking activities. After taking the pre-test,
all of the students had to answer the first questionnaire that involved questions of
background information. In the formal study, the pre-test and the first questionnaire took about twenty minutes to conduct. The researcher used statistics to analyze the frequency and percentages that were collected from the first questionnaire to understand students’
backgrounds and English learning experiences. Besides, the answers of the pre-test were not provided to students because this listening comprehension test was also utilized as the post-test.
In the second week, the researcher of this study randomly selected class A to receive NTF and class B to receive NTP. In the first practice, students in NTF could read
directions of the note-taking strategy before listening because each activity provided several directions for students. They had to listen to the content of the first activity (see Appendix G) twice and then complete the activity. Then, students in NTF answered the second questionnaire after completing the note-taking activity. Conversely, students in NTP were not given any directions before listening, and were encouraged to take notes in their own styles. The researcher of this study did not provide any information for students in NTP, so they took notes by themselves. Students in NTP had to utilize their notes to complete a note-taking activity; then they also answered the second questionnaire after the activity.
From the third to the ninth week, the researcher totally provided 7 note-taking activities for the two classes, and each activity was employed per week. After receiving note-taking activities, all of the students had to take the post-test in the tenth week, which had the same content as the pre-test; then they had to answer the third questionnaire.
At the end of the experiment, the researcher interviewed some students who did not
share their feedback in the open-ended questions of the third questionnaires. Table 5 shows the experimental procedure of this research.
Table 5
The Experimental Procedure of the Formal Study
NTF NTP
Other note-taking practices (3rd week - 9th week) questionnaire after listening.
2. Complete the third questionnaire after listening.
Interviews (11th week)
Select students who did not share feedback toward note-taking activities in the 3rd questionnaire.
Select students who did not share feedback toward note-taking activities in the 3rd questionnaire.
Data Analysis
There were three parts of data analysis in this research. The first part of data analysis was the difference between the pre-test and the post-test, which demonstrated the effects of note-taking activities at different proficiency levels. The researcher of this study utilized the paired sample t-test to analyze the data, so that he could know the effects of note-taking activities on participants’ listening performance and the effects among three proficiency groups after completing the experiment. The second part of data analysis was the effects of two types of note-taking activities on students’ listening abilities. The third part was participants’ opinions and attitudes toward note-taking activities. Thus, the paired sample t-test were employed to analyze the data the researcher collected through the pre-test and the post-test, and the descriptive statistics was utilized to illustrate students’ feedback on each question.