The instruments that were applied in the present study included a pretest, four posttests, a delayed posttest of the target grammar structure, a CRT, and a feedback questionnaire of the song instruction. Each test will be explained in greater detail in the subsequent subsections.
3.2.1 Pretest of the Target Grammar Structure
The aim of the pretest was to determine if any participants had already learned the target grammar structure (i.e., the past subjunctive mood) before the experiment.
The pretest of the target grammar structure (see Appendix A) was a cloze test that included thirty multiple-choice items with three options. Of the thirty multiple-choice items, only ten focused on the target grammar structure, and the other twenty served as distracters, which included other grammatical points that the participants had learned in the previous five semesters. An example of a test item containing the past subjunctive mood is listed below:
If I ____ you, I would ask my parents for help.
(A) were (B) am (C) is
(D) I am not sure.
During the test, the participants were required to read the test item and then to choose the best option to fill in the blank. An option (D) was added to every test item to prevent participants from guessing the correct answer. Disregarding the twenty distracters, the experiment used the ten target test items to evaluate the student’s previous knowledge for the research. Participants who got the correct answers for more than six target grammar structure items were assumed to have knowledge of the target grammar structure, and their data on the following target grammar structure tasks were excluded from the final analysis. Prospective participants were those who failed to answer correctly on more than four target test items. After the pretest, the data of one participant in the sung group, one participant in the spoken group, and six participants in the control group were deleted because their scores were higher than six. Therefore, there were 30 participants in the sung group, 31 participants in the spoken group, and 25 participants in the control group.
3.2.2 Immediate Posttests of the Target Grammar Structure
The aim of the immediate posttests of the target grammar structure was to determine if the participants had learned the target grammar structure without explicit teaching. During the instruction, the participants were exposed to four different songs, and immediately after each song had ended, the participants were required to take a posttest.
The administration and the grading policy of the immediate posttests were identical to the pretest of the target grammar structure in all other respects (see Appendix B, C, D, and E).
3.2.3 Delayed Posttest of the Target Grammar Structure
The aim of the delayed posttests of the target grammar structure was to investigate participants’ learning retention of the target grammar structure two weeks after the instruction. The delayed posttest was identical to the pretest of the target grammar structure (see Appendix A).
3.2.4 The CRT
As noted earlier, the CRT was used to determine the participants’ preferred modality in real-time language processing (visual and auditory). The CRT in the present study included forty test items (see Appendix H). Each test item contained a listening comprehension exercise and a multiple-choice comprehension question. Among the forty test items, ten were designated as incongruent items, which meant the audio text did not match the visual textual caption. The other thirty test items, termed congruent items with matching audio text and visual captions, served as distracters so that the participants were unaware of the incongruent items. The proportion between the two types of items was based on the guidelines provided by Leveridge and Yang (2013).
The CRT items were presented to the participants on Microsoft Office PowerPoint 2007.
When the test was administered, it was described as a comprehension test, and the participants were instructed to answer the test items through their understanding of the dialogue. To answer each test item, the participants were required to listen to a dialogue with a simultaneous caption on the projector. The participants were not
informed whether the test was a listening test or a reading comprehension test, nor were the participants instructed to rely on a particular input. The two modalities were simply presented and the participants had to comprehend the dialogue with both modalities.
The CRT focused on the performances of the participants on the incongruent items. If the participant’s answer was dominantly influenced by the caption, the participant got one point for the visual modality in the test item. If the participant’s answer was dominantly influenced by the audio, the participant would get one point for the auditory modality. When the participant’s points were aggregated and placed along the visual and auditory modality spectrum, participants who got more points for the visual modality than the auditory modality were considered as “visual-modality learners” while participants who got more points for the auditory modality than the visual modality were considered as “auditory-modality learners.” As for those participants who got the same points for both visual and auditory modality, they were considered “balanced-modality learners.” Participants’ scores on the pretest, posttests and delayed posttest of the target grammar structure would be analyzed to see if there was any effect of their preferred real-time processing modality (visual/auditory/balanced) on these tests.
3.2.5 Feedback Questionnaire of Song Instruction
The questionnaire’s instructions (see Appendix G) were given in Chinese to ensure full comprehension by the participants. The questionnaire was divided into three sections. The first section contained questions about the participants’ background, including gender, age, and number of years spent learning English. The second section graded the participants’ opinions and attitudes toward song instruction using a four-point Likert scale. In this section, participants had to respond to 15 questions by
checking “Strongly agree,” “Agree,” “Disagree,” and “Strongly disagree.” For coding,
“strongly agree” represents 4 points, whereas “strongly disagree” indicates 1 point. The last section consisted of 5 open-ended questions where the participants would express their unstructured feedback about the song instruction. Their responses would offer interesting insight and further research directions.