An EEG-based Approach for Evaluating Graphic Icons from the Perspective of
Semantic Distance
Fu-‐Yin Cherng, Wen-‐Chieh Lin, Jung-‐Tai King, Yi-‐Chen Lee
National Chiao Tung University, Taiwan
Physical Digital
1
Graphic icons for interface Design
2
Improve Scannability
vs.
Universal
3
How to evaluate the effectiveness of icons?
Semantic Distance
5
Function Icon
Close
Far
5
Semantic Distance
Key indication of good icons
[cf. Setlur et al., 2014; Warnock et al., 2013]
Measured by behavior and self-report methods Effectiveness of conveying information
5
Close
Far
Semantic Distance
[cf. Huang et al., 2015]
Complicated cognitive states and difficult to determine semantic distance by behavioral measures alone.
Print?
6
T?
Complicated cognitive states and difficult to determine semantic distance by behavioral measures alone.
6
Print?
T?
[cf. Huang et al., 2015]
Use physiological indicators to measure
and analyze cognitive stages.
Electroencephalography (EEG) based method
7
Directly related to cognitive events and states Used in evaluation and usability testing.
[cf. Chi et al., 2014; Lee et al., 2014]
7
Electroencephalography (EEG) based method
[cf. Chi et al., 2014; Lee et al., 2014]
Directly related to cognitive events and states Used in evaluation and usability testing.
Electroencephalography (EEG) based method
7
EEG-based method is a potentially powerful
tool for evaluating icons.
Research Goal
8
Propose EEG-based method to evaluate
human perception of icons, focus on how
users perceive semantic distance of icons.
Research Question #1
9
How do users perceive semantic
distance between icon and function?
10
Research Question #2
How do semantic distance of icons affect
users in different scenarios?
Collection of Icons
6 functions: Calendar, Crop, Keyboard, Menu, Print, Setting
70 icons in gray tone
[cf. ICONFINDER; FLATICON; Google Images]
11
50 participants (24 females)
Classify
Semantic Distance
Not Closely Related
Very Strongly Related
[cf. Isherwood et al., 2007; Mcdougall et al., 1999]
12
Far icons
Close icons
19 participants (11 males), mean age: 21.1
Experiment 1
Function-icon matching
Function Name
13 Experiment 1 | Design
13 Experiment 1 | Design
Icon Match/Mismatch?
Close Far
Semantic Distance
Factors:
13
Match Mismatch
Experiment 1 | Design
Icon Match/Mismatch?
Reaction time Error Rate
EEG Signal
14 Experiment 1 | Design
Measures:
Icon Match/Mismatch?
15 Experiment 1 | Result
Potential (μV)
Time (ms) -8
-4
0
4
8
0
-100 100 200 300 400 500
Close Far Mismatch
Potential (μV)
Time (ms) -8
-4
0
4
8
0
-100 100 200 300 400 500
Close Far Mismatch
16 Experiment 1 | Result
Selective Attention
N1
Early Cognitive Stage
icon shown
Potential (μV)
Time (ms) -8
-4
0
4
8
0
-100 100 200 300 400 500
Close Far Mismatch
Close Icons attract more attention than far icons in early cognitive stage.
17 Experiment 1 | Result
Close
Selective Attention
N1
Early Cognitive Stage
Potential (μV)
Time (ms) -8
-4
0
4
8
0
-100 100 200 300 400 500
Close Far Mismatch
17 Experiment 1 | Result
0.7
Close
Reaction time (sec)
Far Mismatch
0.9 0.8
Close
Close icon can attract more attention, thereby shortening reaction time.
N1
18 Experiment 1 | Result
Potential (μV)
600
-8
-4
0
4
8
Time (ms) 0
-100 100 200 300 400 500
Close Far Mismatch
Potential (μV)
600
-8
-4
0
4
8
Time (ms) 0
-100 100 200 300 400 500
Close Far Mismatch
19 Experiment 1 | Result
Semantic Incongruence
N400
Later Cognitive Stage
Potential (μV)
600
-8
-4
0
4
8
Time (ms) 0
-100 100 200 300 400 500
Close Far Mismatch
Semantic distance level is distinguished in later cognitive stage.
20 Experiment 1 | Result
Semantic Incongruence
N400
Later Cognitive Stage
Potential (μV)
600
-8
-4
0
4
8
Time (ms) 0
-100 100 200 300 400 500
Close Far Mismatch
21 Experiment 1 | Result
N400
Close
Error Rate (%)
Mismatch
0.7
19.9
5.6
Far
Close Mismatch
Opposite groups of semantic incongruence
reduce error rate.
Potential (μV)
600
-8
-4
0
4
8
Time (ms) 0
-100 100 200 300 400 500
Close Far Mismatch
Vague semantic incongruence increases error rate.
22 Experiment 1 | Result
Close
Error Rate (%)
Far Mismatch
0.7
19.9
5.6
Far
N400
Participants’ behaviors provided basic findings, EEG results revealed causes of behaviors and performance in different cognitive stages.
http://www.userzoom.com/wp-content/uploads/2015/04/usability-lab.jpg
Experiment 2
Icon Selection Under Sliding
Selecting icon from sliding menu
23 Experiment 2 | Scenario
24 Experiment 2 | Design
Target Function
Target icon?
Target Function
24 Experiment 2 | Design
: Calendar
Target Function
24 Experiment 2 | Design
: Calendar
Non-Target
Target Function
Target
24 Experiment 2 | Design
: Calendar
Target icon Close, Far Presenting Speed Slow, Fast
24 Experiment 2 | Design
Factors:
Target Function
Target
: Calendar
Reaction time Hit Rate
EEG Signal
25 Experiment 2 | Design
Measures:
Target Function
Target
: Calendar
26 Experiment 2 | Result
Potential (μV)
600
10 5 0
-5
N1
Fast & Close Fast & Far Slow & Close Slow & Far0
-100 100 200 300 400 500
Time (ms) 15
600
Target icon shown
27
Far target icons are easily ignored in fast speed.
Experiment 2 | Result
Potential (μV)
600
10 5 0
-5
N1
Fast & Close Fast & Far Slow & Close Slow & Far0
-100 100 200 300 400 500
Time (ms) 15
600
Selective Attention
N1
Early Cognitive Stage
Fast & Far
Potential (μV)
600
10 5 0
-5
N1
Fast & Close Fast & Far Slow & Close Slow & Far0
-100 100 200 300 400 500
Time (ms) 15
600
28 Experiment 2 | Result
Novelty in a Series of Information
N2
Potential (μV)
600
10 5 0
-5
N1
Fast & Close Fast & Far Slow & Close Slow & Far0
-100 100 200 300 400 500
Time (ms) 15
600
29
Close target icons are easily recognized in fast speed.
Experiment 2 | Result
Novelty in a Series of Information
N2
Fast & ClosePotential (μV)
600
10 5 0
-5
N1
Fast & Close Fast & Far Slow & Close Slow & Far0
-100 100 200 300 400 500
Time (ms) 15
600
30
Close target icons are easily updated to working memory.
Experiment 2 | Result
Working Memory Updating
P3b
FarClose
Novelty and close semantic distance of target icons are important, especially when searching in fast speed.
http://oemsolutions.agameautotrader.com/wp-content/uploads/2015/01/185649173.jpg
Experiment 3
Icon Selection From Grid
Exp 1
EEG Exp 2
Selecting icon from icon gird
31 Experiment 2 | Scenario
32 Experiment 3 | Design
32 Experiment 3 | Design
Find and Click ‘Print’ icon
Find and Click ‘Print’ icon
32 Experiment 3 | Design
Target Surroun
ding
Surroun ding Surroun
ding
Grid Size 2x2, 3x3, 4x4
32 Experiment 3 | Design
Factors:
Target icon
Surrounding icon Close, Far
Find and Click ‘Print’ icon
Target Surroun
ding
Surroun ding Surroun
ding
Reaction time Error Rate
33 Experiment 3 | Design
Measures:
Find and Click ‘Print’ icon
Target Surroun
ding
Surroun ding Surroun
ding
4x4 Grid
Far/Far Close/Far
Far/Close Close/Close
(Target/Surrounding)
34
Close icons are good target icons.
Experiment 3 | Result
2.4 2.2
1.9
1.3
Reaction time (sec)
2.4 2.2 1.9
1.3
35
As surrounding icons, close icons distract participants.
Experiment 3 | Result
Far/Far
Close/Far
Far/Close
Close/Close
Reaction time (sec)
(Target/Surrounding)
4x4 Grid
36
Far icons always increase error rate.
Far/Far Close/Far
Far/Close Close/Close
Error Rate (%)
(Target/Surrounding) Experiment 3 | Result
Effect of surrounding icons increases with grid sizes.
37 Experiment 3 | Result
Surroun ding
Surroun ding
Surroun Target ding
Surroun ding
Surroun ding
Surroun Target ding
Surroun ding
Surroun ding
Surroun ding
Surroun ding Surroun
ding
2x2 3x3
Surroundings: Close Small Gird Size
Big Gird Size
Surroundings: Far
Make trade-offs between reaction time and
error rate based on screen size of applications.
http://www.smartwatchandroid.com/wp-content/uploads/2013/10/sony_smartwatch2-1.jpg
http://files.technobezz.com/files/uploads/2015/05/ipad.jpg
Summary
38
EEG-based evaluation complements behavioral measures and self-reports.
EEG
Self-report Behavior
Summary
39
EEG-based method is feasible and powerful tool for evaluating icons.
N1 N2 P3b
N400
Summary
Acknowledgement
104-2628-E-009-001- MY3, 102-2221-E-009-082-MY3, and 103-2911-I-009-101-.
Taiwan Ministry of Science and Technology (MOST) Anonymous Reviewers
For insightful comments
•
Identify perceptual effects of icons
•
Provide more refined method for evaluating icons
•
Demonstrate how findings from EEG enrich icon usability testing.
Fu-Yin Cherng | [email protected]
Questions?
An EEG-based Approach for Evaluating Graphic Icons from the Perspective of Semantic Distance
Wen-Chieh Lin | [email protected]
National Chiao Tung University, Taiwan