Assignment #5
Low-fi Prototyping &
Pilot Usability Testing
Ideate - Prototype - Test
TEAM ASKSIMPLE ZHENYANG L.
PENG-HSUAN L.
LUKAS G.
MIKE T.
NICOLE S.
Introduction to
Human-Computer Interaction and Design, 2016 NATIONAL TAIWAN UNIVERSITY
Table of contents
Introduction 2
Sketches 2
2.1 Overview 2
2.2 Top Two Storyboards 4
Selected Interface Design 5
3.1 Storyboards of Tasks 6
Prototype Description 7
Testing Method 12
5.1 Participants 12
5.2 Environment 12
5.3 Tasks 13
5.4 Procedure 13
5.5 Test Measures 13
Results 13
6.1 Bugs 13
6.2 Clarity issues 14
6.3 Ideas 14
Discussion 14
Appendix 16
8.1 Consent form 16
8.2 Test scripts 16
8.3 List of critical incidents (pointed out by the interviewees) 17
1. Introduction
We discovered that students are hesitant to ask questions during class due to fear of embarrassment. However, at the same time, they want their questions to be answered as fast as possible. In addition to that, we discovered that some people seem to already know everything in class, and are bored to death during the lecture.
We thought we can use this knowledge to introduce a service into this world:
AskSimple. AskSimple is a service that makes it easier for students to ask and answer questions during class. Therefore, our value propositions is stated as follows:
AskSimple: “Asking made simple”
The earlier stated problem incentivised us to create this collaborative and anonymous tool for asking and answering questions, by implementing it in form of an application, since every student have one nowadays.
2. Sketches
2.1 Overview
Each member of our team started to sketch prototypes on their own first. We decided that we should consider other implementations than mobile application too, therefore some of our members designed prototypes for wearable, desktop, tablet.
Figure 1 shows all the prototypes we came up with.
Figure 1: Overview of Initial Prototypes
2.2 Top Two Storyboards
We chose the web interface and mobile interface sketches as the best two, because it is the most feasible to implement given our team's experience. Another reason for choosing those two is the availability of those devices among students. Most students have a laptop or a mobile phone during lectures. Figure 2 shows the two designs storyboarded, based on the rough sketches of Figure 1.
Figure 2: The Best Two Sketches Storyboarded
In the web interface (left part of Figure 2), we make use of the big amount of space available with the inherent bigger screen size. We can show multiple things on the same screen, and we can show new things, without tracing back.
In the mobile app interface (right part of Figure 2), we have less space, but we tried to make it more intuitive. Screen transition should be logical, and it should feel less busy that the web interface.
3. Selected Interface Design
We selected the mobile application interface as the best one we designed. The reasoning behind this decision is once again the availability and ease of use of a mobile device. Every student have one nowadays, and teachers have one too. On the other hand, students bringing laptops into classes are becoming rarer. We could have decided on a web interface that is usable on a mobile device. However, research shows that people prefer proper applications over a web interface accessed through there phone. According to research done by comScore, the number of global mobile users has surpassed the number of global desktop users in 2014 (see Graph 1).
Graph 1: Global desktop and mobile users 2007 - 2015
(Source: https://goo.gl/IylP9w [17.11.16])
Thus, we decided to focus on a mobile application in order to respond to global user preferences. Furthermore, we feel that our designed web interface is too busy, which is why we selected the mobile interface as the preferred design.
3.1 Storyboards of Tasks Registering (Complex):
Figure 3: Registration Storyboard
Asking/Answering (Moderate):
Figure 4: Answering & Asking Storyboard
Upvote (Simple):
Figure 5: Upvote Storyboard
4. Prototype Description
● Overview:
Our low-fi paper prototype represents our phone app. Most of our functions are registered as touch-click events, besides touch-hovering events for scrolling.
Figure 6: Paper prototype overview
● Login:
Our app starts into a screen of login/register.
Figure 7: Login screens
● Enter a classroom:
If a user wants to enter a new course, the “+” button is clicked, scan the QR code provided in class or enter an ID and then the course will be added to the course list.
Figure 8:Enter a classroom screens
● Navigation:
Users can navigate in the order: Choose a course -> choose a session -> choose a question.
Figure 9: Navigation screens
● Ask a question:
Click on the “Ask” button, enter a question in the pop-up text field and submit the question. A “Thanks for asking” message will appear and the question will appear in the question list.
Figure 10: Question asking screens
● Answer a question:
The arrow “v” below the question is clicked to open all answers to the question.
Click on answer opens a pop-up text field, Click on submit adds the answer to the answer list.
Figure 11: Question answering screens
● Upvote a question
To upvote a question, user clicks on the triangle sign. To redo the action, the triangle is clicked again.
Figure 12: Upvoting screens
5. Testing Method
5.1 Participants
We selected the three participants that we did not know, but where roaming somewhere in our dorms. We did not provide any compensation for them, since we had zero budget. We know that they are students, since they live in the National University Dorms.
5.2 Environment
We cannot test the prototype in a real setting, so we got the next best thing. We simulated a classroom session in a small room with tables and chairs. The participant will see our paper prototype in front of them (see Figure 14). The computer that controls the prototype has all the options in front of him/her (see Figure 13), and is ready to switch the prototype in from of the participant at any point.
Figure 13 : Testing environment Figure 14: The participant
5.3 Tasks
We divided the tasks according to how we learned in this course. Lukas is assigned to introduce our project and leading in the test participant. Nicole guides the test participant during the prototype test process, while Mike and Peng-Hsuan note down any observations, Zhenyang is the computer of the testing process, who switches the paper prototypes when he needs to simulate the interactions of our prototype.
5.4 Procedure
Our test procedure starts with getting the team prepared. We assume our predefined roles. We practiced our test script (see Appendix) before we actually tested it with the test participants. The test participant comes in and gets introduced to our project and he is required to sign a form of consent (see Appendix). Then we start telling him/her a scenario, afterwards we await what he/she will do to complete the task in the scenario and note down the observations. The test participant is required to speak his/her thoughts loudly. We do this for a few scenarios until the testing process is complete. Afterwards we asks some final questions about what they thought about the prototype.
5.5 Test Measures
We have qualitative measures, that consists of observations of mistakes. Mistakes are defined as pushing something with the wrong assumption. Another thing is when the test participant is wanting to do something, but could not because it was not in the prototype. We noted that down as possible major points of improvements. Our other test measure is a quantitative measure: the time to complete a task.
6. Results
Thanks to the testing of the paper prototype, we were able to find out about several bugs that our first paper prototype carries, certain issues that were unclear to the testing participants as well as several ideas that the participants had concerning the improvement of the app.
Each tasks was completed within 3 minutes, which is fast enough. They should not spend more that 5 minutes on each tasks, as our value proposition is asking made simple, so every participant met those quantitative expectations.
6.1 Bugs
i. Password in the registration could be entered twice ii. No logout button
iii. No profile page
iv. Unable to delete the answer in case of mistyping v. No back button
vi. Unable to leave the course vii. Privacy issues
viii. No registration button in login screen
ix. No sorting option for questions according to upvotes or keywords x. No password requirements shown (how many numbers or
characters)
6.2 Clarity issues
1. QR and ID in the course adding screen are a bit confusing 2. Session names and time they were created
3. No discussion possible (replies to answers)
4. Complicated notifications/answer checking (browsing) 5. Not sure on the upvote buttons
6. Course adding via QR/ID is too complex/unclear 7. Star icon unclear/treated as button
8. Participant does not like QR scan (personal opinion)
9. Inside the question + answers scroll-down, how to close the scroll down menu after answering a question
10. Upvote button unclear (not clear that this button is clickable)
6.3 Ideas
a. Upvoting of answers would be helpful (in case TA/professor cannot answer right away)
b. Upvote triangles need another color so that you can see that they are clickable
7. Discussion
We gained many learnings from the testing of our prototype. First of all, we learned that it is difficult to take all ways of usages into account as a designer. Our test participants suggested many basic functionalities which sounded logical to include.
However, some suggestions were clearly personal preferences. We have to think carefully about these suggestions, as we cannot respond to everyone’s needs and preferences in one app. However, we will try to respond to the preferences of the majority.
A clear fault in our design was a button that was clickable, but was not seen as clickable (upvote button). On the other hand, a button that was not clickable, was seen as clickable (approved sign for questions). For these cases we decided to change the buttons into something that is more obvious to their intended functions.
We did the same for any other unclear functions in our design that were pointed out.
Furthermore, we simply forgot many of the basic functionalities, like e.g. a basic logout function. Even if not everyone wants to logout the app, and rather stay logged in the app, we agreed that the option should be there. After each test, several of those basic functionalities were added.
Another improvement was in the complicated process regarding the notification of receiving an answer to a questions you asked. Clicking through the app to reach the answer took too much time, so we simplified it. Although it was not in our three tasks,we considered it still an important part of our design.
Our testing did not reveal anything regarding how the TA and teacher is using our application. Both interfaces are showing the students’ part of the app, however some functionalities need to be different. For example, teachers can make sessions, but can’t upvote questions. Both the TA and teacher have the ability to approve answers, and possibly moderate the questions/answers. In the following process, we will have to decide if we will create the TA/teacher interface and how it will be designed.
8. Appendix
8.1 Consent form
AskSimple - HCID 2016 - Consent Form
The AskSimple application is created in the context of the course “Introduction to Human-Computer Interaction and Design” at the National Taiwan University.
Participants provide data that is used to evaluate and modify the user interface of AskSimple. Data will be collected by interview and observation. Participation in this experiment is voluntary. Participants may withdraw themselves and their data at any time without fear of consequences. Participant anonymity will be provided by the separate storage of names from data. No identifying information about the participants will be available to anyone except the researchers and their supervisors.
I hereby acknowledge that I have been given an opportunity to ask questions about the nature of the experiment and my participation in it. I give my consent to have data collected on my behavior and opinions in relation to the testing of the AskSimple application. I understand I may withdraw my permission at any time.
Name ______________________________________________
Date _______________________________________________
Signature____________________________________________
Witness name ________________________________________
Witness signature_____________________________________
8.2 Test scripts
1) Participant comes in
● Welcome to AskSimple, please sit down in front of the prototype 2) Participants sits down in front of the prototype
3) Participant gets briefed about the prototype
● We of AskSimple designed a prototype of a service we wanted to provide in a classroom setting. Before we start, we need you to sign a form of consent, which allows us to use all the data gathered from this testing process for the future phases of our project.
4) Participant signs a form of consent
● Thank you
5) Participant gets a few scenarios
● You heard about the app in class from the teacher. And you decided to download it. Now you want to use this app and add a course, what will you do
● You entered a session of the course that you just added, and you see that there is questions that you also want to know, what will you do
● You have a question, but do not see the question in the list of questions, what will you do
● You know the answer to a question in the list of questions and other answers are not as good as you think, what will you do
● You submitted an answer and you see that you made a mistake. You want to edit your previously submitted answer. What do you do?
● You want to check out your profile, what do you do?
● You want to go back to your course overview, what do you do?
● You want to logout from the app, what do you do?
8.3 List of critical incidents (pointed out by the interviewees)
● Participant #1
○ Bugs
■ Password in the registration could be entered twice
■ No logout button
■ No profile page
■ Unable to delete the answer in case of mistyping
■ No back button
○ Clarity
■ QR and ID in the course adding screen are a bit confusing
■ Session names and time they were created
■ No discussion possible (replies to answers)
■ Complicated notifications/answer checking (browsing)
● Participant #2
○ Bugs
■ Unable to leave the course
■ Privacy issues
■ No registration button in login screen
○ Clarity
■ Not sure on the upvote buttons
■ Course adding via QR/ID is too complex/unclear
■ Star icon unclear/treated as button
● Participant #3
○ Bugs
■ No password requirements shown (how many numbers or characters)
■ No sorting option for questions according to upvotes or keywords
○ Clarity
■ Participant does not like QR scan (personal opinion)
■ Inside the question + answers scroll-down, how to close the scroll down menu after answering a question
■ Upvote button unclear (not clear that this button is clickable)
○ Ideas:
■ Upvoting of answers would be helpful (in case TA/professor cannot answer right away)
■ Upvote triangles need another color so that you can see that they are clickable
