T
EACHING-L
EARNING
T
IPS
Educational Technology Centre
1
THE HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY
issue: 22/98
Case studies of IT supporting teaching and
learning in HKUST
The application of information technology (IT) to support teaching and learning has got a lot of attention in the past two decades world-wide, especially in universities. And the focus has gradually evolved from how to integrate IT in teaching and learning to how to ensure effective integration of IT in supporting teaching and learning with sound pedagogical back-up.
This issue of Teaching and Learning Tips reports on two attempts by faculty and staff in this University to make use of IT for teaching and learning purposes. There are in fact many examples of innovative uses of IT in HKUST. The ones that are mentioned here are those that have the involvement of the Instructional Development Unit (IDU) to some extent and represent different approaches to adopt IT in teaching. Both projects have gone through stages of trial-run, implementation and evaluation with thought provoking findings that are being shared here. Faculty and staff who have used IT in their teaching activities are always welcome to make use of Teaching and Learning Tips as a forum for discussing their experience.
The PRS System
Prof. Nelson Cue of the Physics Department has for some years been experimenting and developing technology that he called “learning aid” for students. His idea was to deploy technology that would enhance students’ participation in lectures. He felt that genuine learning would take place in a lecture only if students were actively engaged in thinking and problem solving. But with the exception of a few, most students traditionally were quite passive in lectures. When an instructor posed a question, only a few students would have the chance to interact with the instructor. Most would just sit and listen. In order to get the majority of the students involved in some meaningful interaction process, he envisaged technology that gave every student in the lecture the chance to interact with the instructor.
The costly “Classtalk” system
Initially, he experimented with a system called “Classtalk” (by Better Education Inc.). The system allowed students to make responses to questions posed by the instructors using calculator-type handsets that were connected to a central computer. A computer program then processed the students’ responses and gave an immediate feedback to the instructor and students. Two laboratories in the Physics departments were installed with such a system. The “Classtalk” system though quite useful requires the whole classroom to be wired, which is a very costly installation.
The new PRS
Prof. Cue then collaborated with his colleagues and developed a wireless system which is now called Personal Response System (PRS, formerly known as FON). PRS has functions very similar to those of Classtalk but is much cheaper, mainly because the handsets used are more affordable and wiring the classroom is no longer necessary. The system uses a receiver to collect signals sent to it by handsets used by students. The handsets are adapted from remote control devices used for home AV equipment. The receiver then relays the signals to a computer which processes the data to give an immediate feedback to the instructor and students. Within seconds, a chart showing the number and percentage of students choosing each response option will be shown on the lecture theatre screen.
THE HONG KONG UNIVERSITY OF
SCIENCE AND TECHNOLOGY 2
Educational Technology Centre
“Stimulating interactions and active learning is the aim”
Both systems allowed students to respond to questions posed by instructors privately and at ease. “This removes the threat associated with speaking publicly in lectures. Students do not have to risk a ‘loss of face’ when they give the wrong answers. They also do not have to compete with their classmates for the attention of the instructor.” Prof. Cue remarked. “And the most important part of learning takes place while mistakes are discussed and different approaches to questions are identified.”
“Good MC questions are essential.”
Compared to Classtalk, PRS is much more flexible and affordable. The only limitation of PRS is that the handsets do not support input of long answers. Only multiple-choice type answers (‘A’ to ‘E’) can be inputted using the handsets. That means instructor can only ask multiple-choice type questions. “Developing good multiple-choice type questions that ask for application of knowledge, contrasting different data, synthesising information and with sound personal judgement is not an easy job ,” Prof. Cue added.
Students’ evaluation of the PRS
A number of faculty members have already used PRS in their lectures. An evaluation of the effectiveness of the system has also been made. Initial results are quite interesting. What follows is a summary of the findings. Questionnaire surveys have been conducted to collect students’ view about PRS in Fall 97/98 (with a 50% response). Generally speaking, students felt that PRS did make them think more in lectures. About 60 or 50% of the student-respondents chose “Agree” or “Strongly agree” to statements like the ones below:
• I do more thinking during PRS classes than in lecture classes.
• I learn the subject matter of this course easier and with greater depth with PRS than without it.
Another positive effect is that students seemed to be attending lectures more regularly and paid more attention in lectures than before (Roughly 30% of the respondents said so). 45% of the students preferred lectures with PRS whereas only 22% of the students preferred the traditional lectures.
Very similar results were found from surveying two sections using PRS in Spring 97/98.
Faculty’s evaluation of the PRS
Views of faculty members who have used PRS generally echoed the findings from the surveys. They said PRS made students think more and understand better. The students also seemed to attend lectures more regularly. However, some said they often had to spend more time in preparing for the lectures, especially in writing questions used with PRS. One faculty member pointed out that in conducting a PRS session, a series of questions would be needed, which have to be specially designed to help students to understand the key concepts of the subject.
“Less is beautiful”
There was also the view that PRS slowed down their teaching which could be a problem if the course had a lot of content to cover. That brings to a key question in curriculum planning, which has to do with matching content coverage with the average ability level of the students. Prof. Cue in one of the seminars on PRS pointed out : “Covering more was not necessary better. In fact, “more” could be “less” if the students did not understand what they were taught!”
T
EACHING-L
EARNING
T
IPS
Educational Technology Centre
3
THE HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY
In using PRS, Prof. Cue also stressed the importance of having well thought out questions that probe the understanding of the key concepts or points; and allowing and encouraging students to discuss with their peers before answering those questions.
PRS is now installed in three lecture theatres (LT C, D and E). There are plans to implement a campus-wide PRS in stages. All the lecture theatres and some of the large classrooms will be equipped with PRS in the coming semester. Students will in future be supplied with their own personal handsets encoded with their IDs. There will also be portable PRS available for loan.
COSSET - A Web-based System for Teaching Evaluation
Instructional Development Unit (IDU) has been exploring the possibility of using the World Wide Web to collect students responses to course evaluation questionnaire for some years now. It began in 1995 when Dr.
Mavis Kelly, the former head of IDU and others got a research grant from UGC1 to develop a prototype system
for students to do their course evaluation on the Web. In 1997, another 1.4 million was awarded by UGC2 to
implement the system on a large scale, together with the development of a web-based teaching evaluation
system to generate on-line evaluation instruments for individual instructors3.
A web-based system called COSSET (Centralised On-line System for Students Evaluation of Teaching) was built in early 1998. In March, 11 instructors and their 1140 students from the School of Engineering (a total of 11 course/sections) participated in the first trial of the on-line system. The trial was quite successful and students indicated a strong preference for doing course evaluation on-line. Subsequently in May 1998, instructors from 17 course/sections of SENG used COSSET for their end-of-course evaluation.
The operation of the COSSET
COSSET collects students’ responses to questionnaire surveys, like the end-of-semester course evaluation through the WWW. To use COSSET, students have to access the COSSET website through a Web browser like Netscape or Internet Explorer. They will then be asked to supply their CCST usernames and passwords to logon to the system. Once logged on, students will be prompted to select their evaluation subject (i.e. course to be evaluated) and response to the pre-set on-line questionnaire accordingly. The system will collect all submitted responses electronically for data processing and report generation.
The issue of using CCST usernames and passwords as logon key
Decision to use such a logon method was made only after much deliberation by the team. On the one hand, there is a need to make sure that the person doing the evaluation is in fact one of the students enrolled in the course to be evaluated and to prevent any student from doing the evaluation more than once. On the other hand, asking students for their usernames for identification, though administratively simpler, as compared to other methods such as the use of random codes issued to students beforehand, would pose a threat to students’anonymity. Finally it was decided to give the method a try to gauge students’ reactions. Students were given the assurance that their identities would be stripped from the system after initial identification had been made and in no way could they be matched to the answers they made to the questionnaire.
1 The research is a sub-project of the “Evaluation Student Experience Project”.
2 The research is a sub-project of ‘ESEP - Implementation”.
THE HONG KONG UNIVERSITY OF
SCIENCE AND TECHNOLOGY 4
Educational Technology Centre
Students’ evaluation of the System
In the end, through a paper survey with the respondents, we found that majority of the students (65 %) found the log-on method, using their usernames and passwords acceptable. Only 7% objected to the arrangement. 64% said the log-on method would not affect the way they responded to the evaluation. Only 4% said they would not do the evaluation because of the log-on method. And the more encouraging results came from the exceptional high response rate ( 88%) with 72% preferred the on-line system to the paper-based system. 72% found the system easy to use and only about 1% said they had great difficulty.
Comparison of results of paper and on-line modes
In the March trial, we also asked the instructors to split their classes into two random halves, with one group doing the questionnaire on-line and the other on paper, thus allowing a comparison of the results collected through the two different media. 9 of the 11 sections agreed to the arrangement. A comparison of the results showed that the response patterns were very similar. Chi Square statistics showed that significant difference was detected in less than 7% of the questions answered by students.
Faculty’s evaluation of the System
Very positive comments were received. However a few instructors raised their concern that the Web-based system would allow any student in a section to do the evaluation, even if he/she has never turned up in class and has little information on which to base his/her opinion about the course. Another concern is when students do their evaluation together, for example, in the computer barn, they can discuss among themselves since they are not allowed to so do in class. This provides the chance for peer pressure to influence the results.
Results from the May trail-equally encouraging
Results from the 17 sections using COSSET for their end-of-semester Spring 97/98 course evaluation confirmed the findings in the March trial. The slightly lower response rate of 73% was probably because the students were given only 5 days to respond. Students written comments (on-line) once again showed their preference for a web-based system of course evaluation.
Planning ahead
On the whole, the COSSET has been quite successful in fulfilling its function. Consideration is now given to offer the system to all instructors in the Fall 98/99 semester, thus giving them an additional option when conducting their end-of-semester course evaluation.
Do you want to try? Please tell your Office Manager your preference about mid-term or email Mr. Tak S. Ha of the IDU. His email address is “ettsha”. All questions, suggestions and comments are welcome.
The Editor Teaching-Learning Tips is an ETC publication which aims to provide quick and practical ideas for lecturers and teaching assistants to enhance teaching effectiveness. It will be published four times annually. Contributions of ideas and suggestions of topics are heartily welcome. Please contact Winnie Wong, editor at ext. 6809 or email “etwinnie”.
