2 Backgrounds
5.1 Experiment
We have an experiment with Jhongli commercial high school. The experimental subjects consist of 4 students and 4 tutors. The educational level of tutors is college student and master. The students are grade one students in senior high school. We provided one-to-one tutoring. The tutoring time is one hour per week. We asked the tutors come to the computer lab and do the online tutoring activity. Because this is the first experiment, we want to know the real situation about this experiment. And if something happened, we could help or adjust immediately. Before the online tutoring, we introduced our system and illustrated how to operate to tutors. In the students’ side, their class teacher had explained the operations of online tutoring system. Students
can stay in their home and do the online tutoring activity.
We observed that they do the online activities very fluently. Sometimes the student faced some problems and he did not know how to solve, the tutor will give the student some opinions and solve the problems. The student also asks the homework questions to the tutor actively. Their subject is math, and they found that they write something in sketchpad, the writings sometimes do not showed in the hangout. They had no other problem in the Hangout.
38
5.2 System Usability Test
When this system is completed, we invited some tutors and students to use it. Before they use the system, we illustrated this system how to use it. If they had any questions, we could help them instantly. After they had enough know about this system, we let them to fill out the questionnaire of system’s usability.
We adopt the questionnaire System Usability Scale (SUS) developed by John Brooke [33] to evaluate the users’ view of system’s usability. The SUS has ten items which is composed of five positive statements and five negative statements. Five-point scale ranges from 1 as strongly disagree to 5 as strongly agree was used for the measurement. The SUS scores have a range of 0 to 100.
To calculate the SUS score, first sum the score contributions from each item. Each item’s score contribution will range from 0 to 4. For odd items (1,3,5,7,and 9) the score contribution is the scale position minus 1. For even items (2,4,6,8 and 10) the score contribution is 5 minus the scale position. To obtain the overall value of SUS, need to multiply the sum of the scores by 2.5. Total score is 100.
Totally, we had 58 participants, 37 were students and 21 were tutors. Among the participants, the age of participants ranged from 19 to 30 years old; 25 were female, and 33 were male. Most of them had e-learning experience. We thought that they were very familiar with Internet, because they had above four years Internet experience. Their partial personal information was shown in Table 5-2 and Table 5-3.
39
Table 5-1 Personal Information of Participants
Role Participants Gender Age(years old) E-Learning
Experience(y/n)
Table 5-2 Personal Information of Participants Role Education level Internet experience
(Years)
Spend hours on the Internet per day(Hours)
All tutors’ questionnaires were valid questionnaires. In these 37 students’ questionnaires, 35 questionnaires were valid, 2 questionnaires was invalid. We divide the questionnaire results into two parts to explain.
40
5.2.1 Tutors’ SUS Questionnaire Result Analysis
The following is that we show the SUS score of tutors’ questionnaire results and explain some findings in the results.
Table 5-3: Descriptive Statistics of Tutors’ SUS Questionnaire
N Mean Median Min Max SD
Statistics 21 78.6 80 52.5 100 14.3
Listed in table 5-3 are the “SUS scores” of Tutors’ SUS Questionnaire. As summarized in table 5-4, the mean SUS score is 78.6, the maximum is 100 and the minimum is 52.5. Since the Mean and the Median are 78.6 and 80, respectively, these scores indicate that the Social Network based Online Tutoring system is usable.
Table 5-4: Tutors’ SUS Questionnaire and Statistics of Each Item
System Usability Scale Mean SD
1. I think that I would like to use this system frequently 4.3 0.6
2. I found the system unnecessarily complex 2 0.8
3. I thought the system was easy to use 4.2 0.7
4. I think that I would need the support of a technical person to be able to use this system
2.3 1.2
5. I found the various functions in this system were well integrated 4 0.7 6. I thought there was too much inconsistency in this system 1.9 0.7 7. I would imagine that most people would learn to use the system very quickly 4.1 0.7
41
8. I found the system very cumbersome to use 1.6 0.6
9. I felt very confident using the system 4.4 0.6
10. I needed to learn a lot of things before I could get going with this system 1.9 0.7
This survey in Table 5-4 revealed the following findings:
The odd items gained the highest mean. It indicated that most tutors felt very confident while using the Social Network based Online Tutoring system. They showed that this system achieved good characteristics such as attractive to interact with, easy to use, and quick to learn.
The standard deviation of 4th item was relatively high. It indicated that some tutors were familiar with Online Tutoring system, but some were not. Some tutors still needed a technical person who illustrates how to use this online tutoring system. Therefore, some needed technical supports, but some could use the system totally by themselves
5.2.2 Students’ SUS Questionnaire Result Analysis
The following is that we show the SUS score of students’ questionnaire results and explain some findings in the results.
Table 5-5: Descriptive Statistics of Students’ SUS Questionnaire
N Mean Median Min Max SD
Statistics 35 71.8 70 50 92.5 13.1
Listed in table 5-5 are the “SUS scores” of Students’ SUS Questionnaire. As summarized in table
42
5-6, the mean SUS score is 71.8, the maximum is 92.5 and the minimum is 50. Since the Mean and the Median are 71.8 and 70, respectively, these scores indicate that the Social Network based Online Tutoring system is usable.
Table 5-6: Students’ SUS Questionnaire and Statistics of Each Item
System Usability Scale Mean SD
1. I think that I would like to use this system frequently 4.1 0.5
2. I found the system unnecessarily complex 2.3 0.9
3. I thought the system was easy to use 4.2 0.6
4. I think that I would need the support of a technical person to be able to use this system
2.5 1.1
5. I found the various functions in this system were well integrated 3.9 0.7 6. I thought there was too much inconsistency in this system 2.2 0.9 7. I would imagine that most people would learn to use the system very quickly 4 0.8
8. I found the system very cumbersome to use 1.8 0.7
9. I felt very confident using the system 4.1 0.7
10. I needed to learn a lot of things before I could get going with this system 2.6 1.1
This survey in table 5-6 revealed the following findings:
The 3rd items gained the highest mean. It indicated that most of students thought that this is an easy to use system.
The next three with higher ranks were the 1st, 7th, and 9th items. It indicated that most of students would like to frequent to use this system, and could imagine that other people who can quick to learn and they felt confident when they were using the system.
43
The standard deviation of 4th item and 10th item were relatively high. It indicated that some students thought that they need support from technical person, and they need to learn a lot of things before they could get going with this system. But some students were not. They could use this system directly by themselves.
5.3 Qualitative Research
After finishing interaction with the online tutoring system and filling out the SUS questionnaire, we had interviews with some students and some experts who have plentiful experiences in online tutoring. We want to know their thoughts or ideas on our online tutoring system.
There are twenty-three people were interviewed. Seven of them were female, and sixteen were male. The age ranges from 20 years old to 30 years old. The average time for an interview was approximately 5 to10 minutes. Their responses were record in document forms.
Our first question was asked them how did them feel about this online tutoring system. These interviewees described their feelings and thoughts. We got some useful information. Their responses were leading to four main categories.
Satisfaction
Most of interviewees said that they felt the online system interface is beautiful and user-friendly.
They found the various functions in this system were well integrated and will continue to use this system.
Operation
They said that it is very simple, easy and convenient to use this system. They felt confident when
44
they were using the system.
System conception
They thought that social network based online tutoring system is a creative and innovative idea. It is a new way to online tutoring.
Learning
Some interviewees said that this online tutoring system could enhance the learning effect for students in remote districts and increase more interaction with students.
The experts had given some advices.
1. Account Integrated. Because now they need to sign in the online tutoring system and sign in the google. If we could integrate the account, it must be convenient for tutors and students.
2. Recording. Their online tutoring videos need to be recorded. If the teaching videos had recorded, they could see or edit the videos. For students, they could review it. For the tutors, they could know about the teaching situation and improve their teaching skills.
3. When tutors and students do the online tutoring at their home, we could not know the real situation of tutoring. If something happened suddenly, we can not solve it instantly. Let two parties do the tutoring activities in school computer lab, most situations we could control, the problems they faced will reduced.
45
6 Conclusion and Future works
6.1 Conclusion
In our system, we create a social network based online tutoring system. We use the Google+
hangout to be tutoring software. Google+ hangout is funcional enough for presently tutoring, and it is free to use Google+ hangout. Start a hangout like enter a privacy classroom. Tutors could use the tools of the classroom to share resources, explain complicated diagrams and review papers with students. Students could discuss problems with their tutors, and receive their tutor’s responses instantly. Our online tutoring system also provides discuss forums for them. Tutors could share their teaching experiences in tutor’s discuss forum, and could do their best to answer the questions that students posted in student’s discuss forum. The teaching journals which tutor writed were recorded in our database. We could understand the learning situation from this teaching journals.
Through the results of the SUS questionnaire, we found that most of users thought that this online system is usable and could enhance student’s learning effectiveness. It could increases the interaction with tutors and students. We illustrate that online tutoring flow of our online tutoring system in detail. However, we alse provide a user-friendly, simple interface for users. Because we want to let users who felt confident when he/she using this online tutoring system. From the Qualitative Research, we could confirmed that our system is easy to use and they are satisfied with our online tutoring system.
46
6.2 Future work
There are some improvements for social network based online tutoring system. Through several experts’ comment, we need to add some features. First, direct log in our system by using google’s account. This could reduce the problem of duplicate login. Because users need to log in our system and log in google with their google accounts at present. Second, some expert thought that distance remote is an important feature in online tuition. Because when they are online tutoring, the student may play games or surf other websites that the tutor do not know. Add distance remote feature, tutor could catch student’s attention. Third, we want to integrate the annotated materials to the online tutoring system. The tutor uploaded an original version of material to online tutoring system. The materials they used in tuition also recorded in our online tutoring system. Fourth, we want to customize the google+ hangout. We only have one whiteboard could write and do annotation now, we want improve this for add next blank whiteboard to use.
47
Reference
[1] Allen, E. and Seaman, J. “Entering the Mainstream: The Quality and Extent of Online Education in the United States, 2003 and 2004,” Sloan Center for Online Education at Olin and Babson Colleges, Needham, MA, 2004.
[2] Olson TM, Wisher RA., “The Effectiveness of Web-Based Instruction: An Initial Inquiry,” International Review of Research in Open and Distance Learning, 3(2), 1-17, 2002.
[3] Hallinger, P. and Heck, R., “Reassessing the principal's role in school effectiveness,”
Educational Administration Quarterly, 32(1), 5–44, 1996.
[4] Mulford, B. and Silins, H., “Leadership for organisational learning and improved student outcomes,” Cambridge Journal of Education, 33(2), 175–195, 2003.
[5] Amruth N. Kumar, “Results from the evaluation of the effectiveness of an online tutor on expression evaluation,” In Proceedings of the 36th SIGCSE technical symposium on Computer science education, 216-220, 2005.
[6] Kaye, T., “Introducing computer‐mediated communication into a distance education system,” Canadian Journal of Educational Communication, 16(2), 153–166, 1987.
[7] Sellers, R. G., ”Learning to teach in a virtual environment: A case study of the Louisiana Virtual Classroom teachers,” Unpublished doctoral dissertation, Louisiana State University and Agricultural & Mechanical College, Baton Rouge, 2001.
[8] Gene I. Maeroff ,” A Classroom of One: How Online Learning Is Changing our Schools and Colleges,” 2003.
[9] Moore, M. G., & Kearsley, G. , “Distance education: A systems view,” Boston, MA:
Wadsworth Publishing, 1996
48
[10] Beaudoin, M., “The instructor's changing role in distance education,” The American Journal of Distance Education, 4(2), 21-48, 1990.
[11] O’Neil, T., “How distance education has changed teaching and the role of the instructor,” Paper presented at the E-Leader Conference, 2006
[12] 甄曉蘭, “偏遠國中課程實踐生態與教育機會現況調查研究分析,” 行政院國家科
學委員會專題研究計畫成果報, 2008.
[13] H. Cen, K. Koedinger and B. Junker, “Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement,” 8th International Conference on Intelligent Tutoring Systems, 164-175, 2006.
[14] The online tutoring project for after school’s learning (http://www.dsg.fju.edu.tw/dsg/) Retrieved in July 2011
[15] Swan, K., “Building learning communities in online courses: The importance of interaction,” Education Communication and Information, 2 (1) , 23–49, 2002
[16] Maryam Alavi, “Computer-mediated collaborative learning: An empirical evaluation,”
MIS Quarterly, 159-174, 1994.
[17] Official Site of JoinNet (http://blog.joinnet.tw/)Retrieved in July 2011 [18] Official Site of Facebook (http://www.facebook.com/) Retrieved in July 2011 [19] Official Site of Google+ (https://plus.google.com/) Retrieved in July 2011
[20] Google voice and video plugin
( http://www.google.com/tools/dlpage/res/talkvideo/hangouts/) , Retrieved in July 2011
[21] Moore, M. G.,” Three types of interaction,” The American Journal of Distance Education 3(2), 1-6, 1989
49
[22] Hillman, D. C., Willis, D. J. & Gunawardena, C. N.,” Learner-interface interaction in distance education: An extension of contemporary models and strategies for
practitioners,” The American Journal of Distance Education 8(2), 31-42, 1994 [23] Kearsley, G., “ The nature and value of interaction in distance learning,” In
Proceedings of the Invitational Research Conference in Distance Education, 366-374, 1995.
[24] Salmon, G., “E-moderating: The Key to Teaching and Learning Online,” 2nd Ed.
Taylor & Francis Books. Ltd, 2003.
[25] Dawley, L., “ The tools for successful online teaching,” Information Science Press: Hershey, PA,International Society for Technology in Education (ISTE).
(2008). National Educational Technology Standards for Teachers. 2nd Ed. ISTE:
Washington, DC, 2007
[26] Chih-Chiang Yang, “Action research on effect of math problem Solving and Learning Attitude for Junior High students through On-Line-Tutoring System,” Master thesis, University of Chiao Tung, 2005.
[27] Min-fu Su, “The Study of e-Learning after School – Take the Elementary Materials the Area of Circle as the Example,” Master thesis, University of Tainan, 2008.
[28] Chiu-Hua Wang, “A study of the Learning Behavior , Satisfaction and Performance on Internet Learning,” Master thesis, University of Da Yeh, 2001.
[29] Chen, N.-S., & Wang, Y., “Testing Principles of Language Learning in a Cyber Face-to-Face Environment,” Educational Technology & Society, 11 (3), 97-113, 2008.
[30] 呂慈涵, 林宏彥, “偏鄉中小學遠距課業輔導執行模式與策略–以輔仁大學遠距課
輔實施經驗為例,” 2008.
[31] Chao-ling Huang,” Action Research about the Teaching on Fraction Calculation Applied by Software－JoinNet,” Master thesis, University of I Shou, 2008.
[32] McPherson, M.A. and Nunes, J.M.B., “The role of tutors as an integral part of online
50
learning support,” European Journal of Open and Distance E-Learning, 35(3), 305-321, 2004.
[33] Brooke, J., “SUS: a “quick and dirty” usability scale”. In P. W. Jordan, B. Thomas, B.
A. Weerdmeester, & A. L. McClelland. Usability Evaluation in Industry. London:
Taylor and Francis, 1996.
51
Appendix A: System Usability Scale
This questionnaire is based on the System Usability Scale (SUS), which was developed by John Brooke while working at Digital Equipment Corporation. © Digital Equipment Corporation, 1986.
52
Appendix B: Questions of Qualitative Research
Please write your experiences and feelings about this online tutoring system
Please write something you thought that needs to improve in the online tutoring system Please write what features you want to add in the online tutoring system