國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Table 7.3 Estimated results for learning comprehension in exam 3.
Estimate Std. Error t value Pr(>|t|)
Constant 0.947248 0.083509 11.343 < 2e-16 ***
Goal orientation (Low) -0.113797 0.061708 -1.844 0.0701 * Goal orientation (Middle) -0.013169 0.056197 -0.234 0.8155
Format (Line) 0.006314 0.058335 0.108 0.9142
Format (Radar) -0.041197 0.057088 -0.722 0.4733
Task (SearchFact) 0.034706 0.043940 0.790 0.4327
Programming-experienced
(Experienced) -0.062621 0.056753 -1.103 0.2743
Gender (Female) 0.042573 0.046107 0.923 0.3595
* Significance (Sig.) at 0.1 level, ** Sig. at 0.05 level, *** Sig. at 0.01 level.
7-2 Eye Movement and User Behavior
In order to gain a better understanding of the effect of presentation format, we included eye movement analyses in the present study. The linear regression analysis was also adopted to analyze the effects.
Fixation analysis
Goal orientation - Main effects on fixation duration time
We found main effects of goal orientation on fixation duration in regression model of All page, AOI Question and AOI Visualization. The results are summarized in Table 7.4. The results showed that the users with the middle degree of the goal orientation had a lower fixation duration time than high goal orientation users. It indicated that the users with high degree of the goal orientation tended to pay more attention on the format
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
and focus on answering the review questions with the assistance of visualizations (see Figure 7.1, 7.2, 7.3). The results were consistent with the results of learning comprehension and perceived learning, which provided a valid evidence of H1a and H3a.
We also observed that users with low degree of goal orientation tended to have longer fixation duration with bar graphs from Figure 7.1, 7.2, 7.3. However, we didn’t find the main effects in regression analysis. The possible reason was that these users were allocated with bar graphs in exam1. And they tended to spend more time in their first iteration rather than the following two iterations. This bias might influence the results of fixation analysis. As a result, we could only observe the difference between the middle goal orientation users and the high goal orientation users in the fixation analysis despite the fact that we observed the similar results between the low goal orientation users and the high goal orientation users in line and radar graphs (see Figure 7.1). It could also explain why the regression results of log analysis and eye-tracking analysis were not completely consistent.
Table 7.4 Main effects of goal orientation on fixation duration.
AOI Coefficient Estimate Pr(>|t|)
All page Goal orientation (Middle) -4.506 0.00592 ***
Visualization Goal orientation (Middle) -6.0958 0.0546 * Question Goal orientation (Middle) -8.227 0.043935 **
* Significance (Sig.) at 0.1 level, ** Sig. at 0.05 level, *** Sig. at 0.01 level.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 7.1 Main effect of goal orientation on fixation duration in All page.
* Y-axis is the average fixation duration of four AOIs.
Figure 7.2 Main effect of goal orientation on fixation duration in AOI Visualization.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 7.3 Main effect of goal orientation on fixation duration in AOI Question.
Format - Main effects on fixation duration
We found main effects of goal orientation on fixation duration in regression model of All page and AOI Visualization. The results are summarized in Table 7.5. The results showed that users had a high average of fixation duration on radar chart than bar chart in AOI Visualization (see Figure 7.4, 7.5). It indicated that the users spent more time on retrieving the information of radar chart. The possible reason was that radar chart had a more complex composition than bar chart. Hence users may be confused when doing information seeking tasks. The results were consistent with the results of understanding of visualization, which provided a valid evidence of H2b.
Same with the results of goal orientation on fixation duration, Figure 7.4 and 7.5 also showed that users with low degree of goal orientation tended to have longer fixation duration with bar graphs. However, we didn’t find the main effects in regression analysis either.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Table 7.5 Main effects of format on fixation duration.
AOI Coefficient Estimate Pr(>|t|)
All page Format (Radar) 3.948 0.001177 **
Visualization Format (Radar) 7.6618 0.0112 **
* Significance (Sig.) at 0.1 level, ** Sig. at 0.05 level, *** Sig. at 0.01 level.
Figure 7.4 Main effect of format on fixation duration in All page.
Figure 7.5 Main effect of format on fixation duration in AOI Visualization.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Goal orientation - Main effects on fixation count
Similar to the results of fixation duration, we found main effects of goal orientation on fixation count in All page regression model. The results are summarized in Table 7.6. Users with a middle degree of goal orientation had a relatively low average of fixation count than the users with a high degree of goal orientation (see Figure 7.6).
Comparing to the results of fixation duration time, though there was no significance of main effects between middle degree of goal orientation and high degree of goal orientation on fixation count in AOI Question (Pr(>|t|) = 0.1352), we still observed the similar trend between these two groups (see Figure 7.7). The possible reason that the fixation duration time result was not completely agree with the fixation count result was that we counted fixation duration time as the average of the duration for all fixations within an AOI, and the fixation count as the number of times the user fixated on an AOI. Hence the amount of fixation count was not completely correlated to the average fixation duration time due to each fixation could be just a few seconds or dozens of seconds.
Table 7.6 Main effects of format on fixation count.
AOI Coefficient Estimate Pr(>|t|)
All page Goal orientation (Middle) -14.642 0.04786 **
* Significance (Sig.) at 0.1 level, ** Sig. at 0.05 level, *** Sig. at 0.01 level.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 7.6 Goal orientation on fixation count in All page.
Figure 7.7 Goal orientation on fixation count in AOI Question.
Format - Main effects on fixation count
Similar to the results of fixation duration, we found main effects of format on fixation count in All page and AOI Visualization regression model. The results are summarized in Table 7.7. The results showed that users had a high average of fixation count on radar chart than bar chart in AOI Visualization (see Figure 7.8, 7.9). It
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
indicated that users used more fixations on retrieving the information of radar chart, which indicated a less efficient search.
Figure 7.8 and 7.9 also showed that users with low degree of goal orientation tended to have more fixation count with bar graphs. However, we didn’t find the main effects in regression analysis either.
Table 7.7 Main effects of format on fixation count.
AOI Coefficient Estimate Pr(>|t|)
All page Format (Radar) 14.155 0.04584 **
Visualization Format (Radar) 24.357 0.0813 *
* Significance (Sig.) at 0.1 level, ** Sig. at 0.05 level, *** Sig. at 0.01 level.
Figure 7.8 Main effect of format on fixation count in All page.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 7.9 Main effect of format on fixation count in AOI Visualization.
Transition analysis
To explore the dependency of the transition between areas of interest (AOIs), we focused on AOI Visualization, AOI Question and the corresponding factors because the dominated eye movement transition for both search fact and inference generation task is VQ and QV. We found the main effects of format and task on the transition rate between AOI Visualization and AOI Question. The results showed that the users performed more transitions between AOI Visualization and AOI Question during answering search fact questions than inference generation questions. It indicated that visualizations could facilitate the information search fact tasks. The results were consistent with the results of learning comprehension, which provided a valid evidence of H1c. We also found that users had a relatively less transition rate between AOI Visualization and AOI Question on radar chart than bar chart. It indicated that radar graph was not suitable for the information search task due to its complicated composition. The results were consistent with the results of understanding of visualization and fixation analysis, which provided a valid evidence of H2b.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
In sum, the results of fixation analysis and transition analysis show that learning goal orientation and format have impact on the fixation duration and fixation count, while format and type of learning task have impact on the transition rate. In terms of the learning goal orientation, participants who have a middle degree of learning goal orientation tend to have less fixation duration and fixation count than who have a high degree of learning goal orientation. This result suggest that the students with high learning goal orientation are more detailed when they review the visualization analytics.
They are willing to spend more time to explore the information embedded in the visualization for their own personal development. The results also show that participants have a relatively higher fixation duration and fixation count on radar graph then other formats, which meant that participants tend to spend more time on radar graph. The possible explanation is that radar graph contained more information than the other format, so that participants needed to spend more time on it to decode information from radar graph. It is consistent with the feature that radar graph was inferior on information retrieving. Finally, the transition analysis results show that participants have relatively higher transition rate between AOI Visualization and AOI Question on search fact task than inference generation task. As Visualization and Question AOIs are two major area of the visualization interface of TCAV, VQ and QV are two transitions which dominate user’s eye movement transition. The higher frequency of using these two transitions in the visualization interface might indicate that these processes were better supported by the design of visualizations analytics.