Tao, Y.-H., Yeh, R. C. Personal response System: A model-based case study in Taiwan,
14th Annual Meeting of Asia Pacific Region of Decision Sciences Institute, Shanghai,
July 4-8, 2009
Personal Response System: A Model-Based Case Study in Taiwan
Yu-Hui Tao C. Rosa Yeh
National University of Kaohsiung National Taiwan Normal University
TAIWAN, R.O.C. TAIWAN, R.O.C.
ytao@nuk.edu.tw rosayeh@ntnu.edu.tw
ABSTRACT
Personal response system (PRS) is increasingly adopted in Taiwan’s higher education. A small-scale case study was conducted to compare the perceptions of PRS usage of Taiwan’s college students on those issues reported by UK and USA literature, and at the same time to empirically test an integrated model that has not been tested in prior PRS studies.
INTRODUCTION
Instant response system (IRS) includes the interactive white board (IWT) and personal response system (PRS). There has been an increasing trend of PRS application in Taiwan’s higher education. However, to our knowledge, there is no related studies outside the continents of Europe, America, and Australia. This is a situation of concern since culture or region has been generally recognized as having a moderating effect in the technology acceptance model (TAM) and related theories. Meanwhile, only a few PRS studies touched upon theories in relation to education, and none on the technology-based theories for establishing students’usage of such educational technology in classroom. In order to shorten the learning curve of PRS applications in Taiwan’shigher education, it is important to compare how Taiwan’s college students perceive PRS on those issues that had been reported in UK and USA so that more effective implementation strategies or practices can be adopted. A small-scale study was conducted with over fifty students registered in a course in a university in southern Taiwan.
RESEARCH DESIGN
The research model is an integration of technology acceptance model (TAM) and expectation confirmation theory (ECT) since they are supplementary to each other as implied by Premkumar and Bhattacherjee (2008). Therefore, this research used learning performance in place of the usage intentions and actual usage of TAM, which not only was more meaningful in practice, but also provided a fine link from TAM to ECT. The research model can be seen in Figure 1 in the later section.
The PRS study was conducted in a required course--System Analysis and Design--offered by the Department of Information Management at the National University of Kaohsiung in Taiwan. In the eighteen-week course design, the instructor taught the first five chapters of the textbook “System Analysis and Design in a
Changing World, 4th Edition” by Stazinger, Jackson and Burd, and 10 groups of students prepared and presented the remaining 10 chapters in the following 10 weeks. In each of the three-hour session, PRS was used to do the roll call at least twice and to perform at least three Q&A’s regarding the materials. The students first experienced how PRS was used in the first five meetings taught by the instructor before designing their own PRS application in their presentation. Fifty-three students registered in this course, but one dropped out after the first mid-term exam. Forty-seven students filled out the online questionnaire at the end of this course.
The questionnaire has two sections. The first section collected data on questionnaire items from existing PRS literature and the second section contained items for measuring the TAM and ECT constructs adopted from the literature. Students were asked to respond to each questionnaire item (except item 22) using a 5-point Likert scale with 1 representing “extremely disagree”and 5 representing “extremely agree”.
ANALYSIS AND DISCUSSION
Descriptive statistics is used to analyze the questionnaire items in the first section, and simple regression is used to analyze the measurement items for the research model. Regression is used because of the small sample size of 47 in this study. Table 1 enumerates the questions with supporting references and average scores calculated from the 47 observations. To give meaning to this survey result, we summarized our interpretations into nine points which are listed below.
Table 1 Questionnaire items, references and average scores
# Item & references Score
1 I prefer the Harvard style that students preview class materials and answer some questions at the beginning of each class meeting, and then the instructor adjusts teaching outline and materials accordingly. In the class, PRS is still used to guide students to answer quick questions and discussions for clarifying some misconceptions and perceptions.(Anonymous, 2008)
3.66
2 I prefer not to preview class materials before each class meeting, but to use PRS during the class for identifying students’understandings for reinforcing certain topics. (http://www.habook.com.tw/default.as)
3.88
3 I prefer to preview class materials before each class meeting, and use PRS during the class for dynamically adjusting class materials and schedule. (http://www.habook.com.tw/default.as)
4.02
4 I prefer that PRS is not used to monitor student attendance and cumulate points for participation. (D’Arcy etal., 2007)
3.49
5 Using PRS increases students’class attendance. (Briggs, 2006) 3.72 6 Using PRS increases students’class concentration.(Murphy, 2008) 3.68 7 Using PRS helps the instructor quickly get the quick answers from students.
(Skiba, 2006)
3.7
8 Using PRS helps the instructor discover students’misconceptions. (Linsenmerier, et al., 2006)
3.13
9 I prefer increasing the discussion time and the depth after the PRS Q&A. (Linsenmerier, et al., 2006)
3.7 10 I prefer some discussions before a quick Q&A with PRS in class.(d’Inverno
et al., 2003)
3.53
PRS can effectively support peer instruction and discussions.(Linsenmerier, et al., 2006)
12 I can accept the delay in class schedule for increasing the use of PRS. (d’Inverno et al., 2003)
3.3 13 I can accept taking a quiz using PRS in class.(Vernaza, 2007) 3.17 14 I prefer using PRS for a simulation test before a conventional test.(Vernaza,
2007)
3.21
15 PRS increases the interactions between an instructor and the students.(Skiba, 2006)
3.68
16 PRS increases the interactions among students.(Skiba, 2006) 3.51 17 The use of PRS perfectly fits the student profile of the Millennium generation
who were born after 1981. (Murphy and Smark, 2006)
3.55
18 I am willing to rent or buy the remote device if the school implements a large-scale use of PRS in many courses. (Briggs, 2006)
3.77
19 The use of PRS may be better for the students if the remote device were changed from infrared- to RFID-based.(Murphy, 2008)
3.66
20 The learning effect will be increased if the question types can be extended from simple multiple choices to open-ended questions by replacing the remote device to a WiFi-based mobile phone, notebook computer or PDA.
(Beuckman et al., 2006; Anonymous, 2008)
3.74
21 Experts suggest that it is more appropriate to use PRS 4 times at most in a class session.(Jensen et al., 2008)
3.74
22 How often do you wish to use PRS in class?(Brewer, 2004)
a. Every 50-minute session 15 (31.91%)
b. Every week (three 50-minute sessions) 22 (48.81%)
c. Every two weeks or longer 7 (14.89%)
d. Never 3 (6.38%)
1) Students in general held positive attitude toward the use of PRS in class as shown from their responses toward items 1-3 which list the PRS application strategies. However, the average scores of items 3 and 2 are both higher than item 1, which seemed to indicate that Taiwan’scollegestudentspreferlesstime-consuming ways to prepare for class when PRS is used. The average score of 3.49 for item 4 also echoed such stereotype because it revealed that more students rejected the idea of using PRS to monitor their attendance and participation.
2) Items 5 and 6 were intended to confirm whether the use of PRS can increase students’attendance and concentration in class. The average scores were relatively high among all items and seemed to lean toward such positive assumptions.
3) The result from item 7 mildly confirmed that instructors could quickly get some answers from the students. However, the average score of 3.13 for item 8 indicated a less support from the students that instructors are able to discover students’ misconceptions with the use of PRS. It has something to do with how the instructor responded after obtaining students’responses.
4) In terms of whether PRS really can help students learn class materials, the result from item 11 indicated that students recognized the effects of peer instruction and discussion better when there are discussions before or after the PRS Q&A (3.81). In
particular, the result from item 9 reflected a general feeling of insufficient quality time after the quick PRS Q&A, and item 10 further expressed students’need tohave more discussion before making the quick answers. Many students would accept some delay in class schedule for the increasing use of PRS, as indicated by item 12.
5) In addition to using PRS for quick Q&A in class, can students accept taking tests using PRS? The low 3.17 score of item 13 indicated that students did not like the idea of using PRS to conduct the quiz evaluation toward their final grades. Even the idea of serving as a practice test before a conventional test (item 14) was not well received, which scored only an average of 3.21, slightly higher than that of item 13.
6) For increasing class interactions, students believed more on the effect between an instructor and the students as shown by the 3.68 score on item 15 than that among the students (3.51 on item 16). However, the average scores may be higher if the instructor could design and facilitate the use of PRS better, which sometimes may cause delays in the class schedule as pointed out in item 12.
7) One interesting question, item 17, regarding whether PRS was more suitable for current college students born after 1981? The average score of 3.55 was not as high as expected, which implied either PRS was suitable for all generations or PRS might not be as attractive or natural to fit in the learning framework(Murphy and Smark, 2006). 8) About the PRS handheld device, more students were willing to rent or buy the remote device if a large-scale implementation is in place by item 18. Students also agreed for a better alternative such as RFID-based remote device (3.66 on item 19), or a WiFi-based mobile phone, notebook computers or PDA (3.74 on item 20)
9) Students in general accepted the suggestion of using 4 times of PRS in a class session as indicated by item 21. However, when asked differently on the PRS usage frequency as shown in item 22, 21.27% of the students answered over two weeks or never. Perceived ease of use 3.70 Perceived usefulness 3.45 Learning performance 3.29 Confirmation 3.40 Satisfaction 3.11 Intention to continue using 3.34 H1 0.18 H2 0.372** H3 -0.006 H0.455***4
Technology Acceptance Model
Expectation Confirmation Theory
H5
0.525***
H6 0.502***
Figure 1 Integrated research model and results
The research model, hypotheses and the results are all summarized in Figure 1, with the average scores under the variable names within the square, and the path coefficient with corresponding significance level on the path with the hypothesis number. From the macro view, it is clear that the ECT hypotheses, H4-H6, were all
performance, H2, among TAM hypotheses, H1-H3. This research substantiated the
ECT model in PRS usage. Specifically, learning performance influences the confirmation, which in turn influences the satisfaction of students, and thus leading to the intention to continue using PRS in the future. However, what were considered the two most robust relationships in TAM, H1 and H3, were not supported in PRS adoption while the supposedly weaker relationship, H2, was supported in this study. There are three possible explanations. First, in this study of PRS for Taiwan’s college students, the sample size was too small to adequately test prior hypotheses statistically. Secondly, PRS has its unique features as a technology to be applied in higher education which may require some modifications. Thirdly, a study on PRS may require a large-scale investigation with more sophisticated research model.
CONCLUSIONS AND IMPLICATIONS
Most of the observations above about the college students’perceptions of PRS usage in Taiwan were consistent with the originating literature, although Taiwan’s college students exhibited more stereotypical attitude of less effort in class participation. However, while the integrated research model on the ECT side is encouraging, the TAM portion presented a deviated result from the majority TAM studies. Our study showed that perceived usefulness was neither impacted by perceived ease of use nor significantly impacting the learning performance when PRS was used.
REFERENCES
Anonymous (2008). An instance feedback learning environment, Converge Online,
http://www.convergemag.com/story.php?catid=43&storyid=29722.
Beuckman, J., Rebello, N. S. and Zollman, D. (2006). Impact of a classroom interaction system on student learning. Physics Education Research Conference, edited by L. McCullough, L. Hsu, and P. Heron, American Institute of Physics. Brewer, C. A. (2004). Near real-time assessment of student learning and understanding in Biology courses, BioScience, 54(11):1034-1039.
Briggs, L. L. (2006). Response devices keep FSU students focused. Campus
Technology, http://www.campustechnology.com/article.aspx?aid=41307.
D’Arcy,C.J.,Eastburn,D.M.and Mullally,K.(2007).Effectiveuseofapersonal response system in a general education plant pathology class. The Plant Health
Instructor, DOI: 10.1094PHI-T-2007-0315-07.
d’Inverno,R., Davis, H. and White, S. (2003). Using a personal response system for promoting student interaction, Teaching Mathematics and its applications, 22(4): 163-169.
Jensen, R., Meyer, L., and Sternberger, C. (2008). Three technological enhancements in nursing education: Informatics, instruction, personal response systems, and human patient simulation. Nurse Education in Practice, doi:10.1016/j.nepr.2008.10.005. Murphy, B. and Smark, C. (2006). Convergence of learning experiences for first year tertiary commerce students –Are personal response systems the meeting point?
Journal of American Academy of Business, 10(1): 186-191.
Murphy, T. (2008). Success and failure of audience response system in the classroom,
SIGUCCS, October 19-22, Portland, Oregon, U.S.A.
Skiba, D. J. (2006). Got large lecture hall classes? Use Clickers, Nursing Education
Vernaza, K. M. (2007). Using personal response system technology and concept check modulesto improvestudents’learning experience:A casestudy,37th
ASEE/IEEE Frontiers in Education Conference, Session F1D-16~17, October 10-13,
