Chapter 5. Analysis and Results
5.2 Structural Model Analysis
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
35
Table 5.3. Correlations of Constructs
Absorption Immersion Interactivity NPM Performance
Realism Storytelling Vividness
Absorption 0.785
Immersion 0.544 0.849
Interactivity 0.515 0.5 0.791
NPM Performance 0.572 0.297 0.336 0.839
Realism 0.483 0.468 0.424 0.344 0.898
Storytelling 0.521 0.357 0.381 0.391 0.44 0.811
Vividness 0.423 0.46 0.478 0.259 0.424 0.496 0.848
5.2 Structural Model Analysis
We used three models in the analysis. Model 1 is the general model, which is applied regardless of the artifact type. Models 2 and 3 focus on specific stimuli. Model 2 is for Huai Su’s Autobiography and Model 3 is for Zhao Meng-fu’s Autumn Colors on the Que and Hua Mountains. Following Hair and associates’ (2013) suggestion, a bootstrapping approach was implemented with 5,000 samples created for estimating the path coefficient.
In the first step, path coefficients were calculated. The t-test helps us examine the statistical significance of the construct path coefficient and confidence intervals. The path coefficients and their t-values for Models 1–3 are presented in Table 5.4. In Models 1 and 2, the results indicate that most of the paths are significant, with the exception of the effect of vividness on absorption (T1= 0.064; T2= 0.01) and the effect of immersion on NPM performance (T1= 0.382; T2= 0.622). Consistent with previous research, there was one more effect, storytelling on immersion (T3= 0.89), not significant in Model 3.
Also, the moderating effect is not strong enough in Model 3.
In the second step, model validity was assessed by calculating the R2 value (Chwelos et al., 2001), a statistical measure of how close the data fit the regression
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
line. The three models each explained over 30% of the variability in response data, indicating a very satisfactory level of explanation (Table 5.6).
In the final step, we use R2 of the three models to calculate f2 statistics and pseudo F-value statistics to analyze the relationship between models (Patnayakuni et al., 2006).
This allowed us to see whether the addition of relationship value could significantly add to the explanatory power of the model. The value of f2 is derived by dividing the difference of R2 of two models by [1– (R2 of advanced model)]. Next, the pseudo F statistic is computed using the following formula: f2 * ([n–k]–1). Note that (n–k–1) is the degree of freedom, where n is the sample size and k is the number of independent variables in the basic model. The testing standard is also measured by its p-value. When the p-value is 0.1, 0.01 and 0.001, the corresponding pseudo F-value4 is 2.7, 6.7 and 11. This means that if the pseudo F-value is smaller than 2.7, there are little differences between models. Their structure and explanatory power are similar, so we can discuss the two models together. On the other hand, if the number is larger than the pseudo F-value, the model shows greater differences from the basic model. These results are summarized in Table 5.6.
Table 5.4. Testing of Hypotheses
Dependent Variable: Immersion (k=5)
Independent Variable Model 1 Model 2 Model 3
(H1a) Interactivity 0.278 (4.701***) 0.292 (3.785***) 0.258 (2.759***) (H2a) Vividness 0.205 (3.809***) 0.17 (2.26**) 0.237 (2.989***) (H3a) Realism 0.231 (4.15**) 0.209 (2.772***) 0.258 (3.152***) (H4a) Storytelling 0.376 (2.368**) 0.518 (2.023**) 0.181 (0.89) (H8a) Priming 0.535 (2.272**) 0.667 (1.897*) 0.306 (0.884) Priming * Storytelling -0.165 (2.191**) -0.202 (1.803*) -0.095 (0.867)
R Square 0.376 0.392 0.368
4 FINV() is the function in Excel, it will get the pseudo F-value by inputting the p-value, the 1st degree of freedom, and the 2nd degree of freedom. Thus, the value is computed: FINV(0.1, 1, 369)=2.7, FINV(0.01, 1, 369)=6.7 and FINV(0.001, 1, 369)=11.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
37
Dependent Variable: Absorption (k=6)
Independent Variable Model 1 Model 2 Model 3
(H1b) Interactivity 0.214 (3.635***) 0.225 (2.585**) 0.196 (2.313**)
(H2b) Vividness 0.004 (0.064) 0.001 (0.01) 0.013 (0.14)
(H3b) Realism 0.137 (2.753***) 0.121 (1.749*) 0.165 (2.225**) (H4b) Storytelling 0.572 (4.019***) 0.604 (2.996***) 0.508 (2.32**) (H5) Immersion 0.253 (4.232***) 0.26 (2.996***) 0.24 (2.738***) (H8b) Priming 0.453 (2.022**) 0.536 (1.786*) 0.306 (0.838) Priming * Storytelling -0.144 (2.003**) -0.165 (1.706*) -0.103 (0.882)
R Square 0.48 0.481 0.491
Dependent Variable: NPM Performance (k=2)
Independent Variable Model 1 Model 2 Model 3
(H6) Immersion -0.02 (0.382) -0.05 (0.622) 0.007 (0.093)
(H7) Absorption 0.583 (12.794***) 0.598 (9.152***) 0.572 (8.962***)
R Square 0.328 0.326 0.332
Note: p*<0.1; p**<0.01; p***<0.001.
‧
Table 5.5. Statistical Values of Responses
Variable Item Mean STDEV Median
‧
Table 5.6. Differences between Models
Dependent Variable: Immersion (k=5)
Model 1 vs. Model 2 Model 1 vs. Model 3 Model 2 vs. Model 3
Differenced R Square 0.016 0.008 0.024
f Square 0.026 0.013 0.038
Dependent Variable: NPM Performance (k=2)
Model 1 vs. Model 2 Model 1 vs. Model 3 Model 2 vs. Model 3
Differenced R Square 0.002 0.004 0.006
f Square 0.003 0.006 0.009
Pseudo F-value 1.104 2.228 3.359
Table 5.7. Hypothesis Inspection
Factors Hypothesis Model
1
H1a. The degree of interactivity is positively related to the extent of immersion in VR experience. O O O H2a. The degree of vividness is positively related to the extent of immersion in VR experience. O O O H3a. The degree of realism is positively related to the extent of immersion in VR experience. O O O H1b. The degree of interactivity is positively related to the extent of absorption in VR experience. O O O H2b. The degree of vividness is positively related to the extent of absorption in VR experience. X X X H3b. The degree of realism is positively related to the extent of absorption in VR experience. O O O
Storytelling
H4a. The degree of storytelling contents is positively related to the extent of immersion in VR experience.
O O X
H4b. The degree of storytelling contents is positively related to the extent of absorption in VR experience.
O O O
User Experience
H5. The higher level of immersion is positively associated with the extent of absorption in VR experience.
O O O
H6. The higher level of immersion is positively associated with NPM performance. X X X H7. The higher level of absorption is positively associated with NPM performance. O O O Moderating
Effects
H8a. The degree of priming affects effectiveness of storytelling on Immersion. O O X H8b. The degree of priming affects effectiveness of storytelling on Absorption. O O X
*Note: O denotes hypothesis supported; X denotes hypothesis rejected
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y Chapter 6. Discussion
Most of our hypotheses were fully supported, but all models were rejected for H2b and H6. For H4a, H8a, H8b, only Model 3 was rejected (Table 5.7). In the following sections, the rejected pathways will be discussed. Finally, the relationship between models will be analyzed based on the three dependent variables (Figures 6.1, 6.2, and 6.3).
Figure 6.1 Path Analysis of Model 1
Figure 6.2 Path Analysis of Model 2
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
41
Figure 6.3 Path Analysis of Model 3
6.1 Design Factors and Immersion
For the first dependent variable, immersion, design features (interactivity, vividness, realism, and storytelling) account for 37.6% of the variance in Model 1. If the artifact type is taken into consideration, 39.2% and 36.8% of the variance can be explained by design features in Models 2 and 3, respectively. According to pseudo F-value statistics, Models 2 and 3 have better explanatory ability than Model 1 (R square values are significant).
The hypotheses related to the impact of design factors on immersion are all supported in Models 1 and 2. However, in Model 3, the storytelling hypothesis (H4a) is rejected. In Model 3, there are more scenes designed in VR. Users first walk into the painting, enjoying the scenery of the Que and Hua Mountains. Next, an ancient table appears with the paintings on it. The emperor’s seals appear on the painting one by one.
In the last scene, the user travels along the river by boat. Under the moonlight, a poem by Meng-fu fades in and its words appear in the sky. Some participants reported that the “transition makes me confused . . .without instructions, I don’t know what to do next,” and “I need more instruction.” Even if we arranged abundant scenes, the scenario
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
might not connect well with users. Transitions between scenes might not be smooth enough to achieve effective storytelling for the immersing audience. We also encountered a lack of professional instruction in the use of VR. A middle-aged woman observed, “Generally, ordinary people can't read the paintings or know the original mood of the poets or painters. Instead of walking in the ancient scene, the explanation of the seals is easier to understand.” It might be better to introduce more background information rather than create an abstract atmosphere.
6.2 Design Factors and Absorption
For the second dependent variable, absorption, at most 48% of the variance could be explained by the stimulus (interactivity, vividness, realism, storytelling, and immersion) in Model 1. Also, 48.1% and 49.1% of the variance can be explained by design features in Models 2 and 3, respectively. Compared with the results for immersion, the R square values are more significant, and thus the interpretive ability is stronger. We can conclude that absorption is relatively more influential than immersion in our model.
The pseudo F-value demonstrates better explanatory power in Model 2, but the hypothesis related to vividness (H2b) is rejected in all three models. A quality VR experience requires good computer equipment with adequate processing capability.
Owing to insufficient equipment, the audio quality might sound like blurry noise. The 3-D modeling quality could not achieve the clarity of a 2-D image, so many users identified the quality issue as problematic, including comments such as, “texts are too tiny and blurry, can’t see anything,” “it would be better if the quality increased,” and
“sights are not clear for those who wear glasses.” Unfortunately, we had no budget to purchase the top-level device. Hence, the influence of vividness in the study might be affected by the low visual and audio quality of the device used.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
43
We found that storytelling was much more effective in inducing absorption than were other design factors. The t-statistic for storytelling in Model 1 was 4.019, which indicates greater influence than other design factors (Figure 6.1). Models 2 and 3 are the same as Model 1, showing a relatively higher statistic for storytelling in inducing absorption.
6.3 User Experience and NPM Performance
For the third dependent variable, NPM performance, the R square also indicates a high value of 32.8% for immersion and absorption in Model 1. In Models 2 and 3, 32.6% and 33.2% can be explained by individual user experience. According to pseudo F-value statistics, the explanatory power of the model is not significant. As a result, the type of artifact does not significantly affect the result between user experience and performance.
The hypotheses related to immersion (H6) were rejected in all three models. One user described the experience as boring, and noted: “need more interaction” in the opening answer. It’s possible that the interaction component is still not sufficient to enable someone to feel immersed. Immersion emphasizes how one feels in the present moment. Feeling immersed doesn’t necessarily lead to having an incentive to revisit the museum or refer to others after the experience.
The results reveal an important finding: absorption is more effective in improving NPM performance. In the beginning, we expected immersion to be the driver of NPM performance, but that hypothesis (H6) was not supported. Users might be impressed with the interaction but lack motivation to explore more, which would give NPM a poor performance rating. Even absorption is influenced by design factors, but that does not mean visitors’ interest would be promoted as well. In contrast, absorption clearly helps the NPM performance. VR can support NPM effectively in advertising and promoting
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
their collection, but immersion can only be effective during the time they put on the VR headset. After the experience, motivation would not be boosted. Thus, NPM should devote more attention to absorption-related design features.
In summary, H2b and H6 are rejected in Models 1 and 2; in Model 3, other than H2b and H6, H4a and the effect of moderator disclosure are rejected as well.
6.4 Moderating Effect
We expected that the priming condition would help participants gain a more favorable impression of our story. Instead, we found unexpected results. The moderating effect generally supported a negative reaction toward storytelling (Figure 6.4). It caused a greater slope on absorption than immersion. This means the moderating effect was even stronger in relation to absorption (Table 5.4). One possible reason is the artifact type. While experiencing Huai Su's Autobiography, there is only background music playing without audio commentary which directed the focus to the stormy atmosphere. In contrast, Zhao Meng-fu’s Autumn Colors on the Que and Hua Mountains has plenty of description and narration. For this reason, priming caused different results in Models 2 and 3.
In conclusion, the artifact type may influence the impact of priming. For the two artifacts in our study, priming did little to shape user experience, but it might be helpful for other kinds of objects (e.g., sculpture). We need to explore this possibility with other artifacts and cases to test our hypothesis in the future.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
45
Figure 6.4 Comparison of Moderating Effect
6.5 Findings
The results of our findings are summarized below.
6.5.1 Background story matters
Despite the influence of storytelling, the priming effect had a significant role in our model. Those who began with a glance at introduction posters could better understand what NPM tries to convey, which in turn affects how much storytelling shapes one’s experience, especially in the absence of oral guidance in the VR program (Model 2).
NPM could try to design VR programs with more preparation for users, for example, decorating the exhibition hall with clear instructions to help visitors understand its story background.
6.5.2 Storytelling is more significant than other design factors
Our study included many VR stimulus factors, but some of them did not function as expected. Despite these results, we affirm their obvious effects in UX design and remain curious about their impact. Our outcomes suggest that storytelling is relatively influential in the museum's VR experience, especially in inducing absorption. This
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
gives us confidence in the importance of storytelling. We believe storytelling holds a key to improved VR design for museums in the future.
6.5.3 Absorption could certainly help to increase NPM’s performance The results suggest that immersion is not a necessary precursor of absorption.
Although VR creates absorption, this does not mean users are immersed in the experience of the moment. The incentive to refer others and going oneself to NPM appear to not be related to the extent of immersion. NPM might consider designing a smoother transition to resolve the scattered story line and reinforce the story’s power to elicit absorption.
6.5.4 VR remains unfamiliar to the general population
In our experiment, most participants had no experience with VR. Without a doubt, local habits and customs matter. Most people in Taiwan rarely access VR applications, most likely because they view VR as used primarily for entertainment. People would rather invest in a $200NTD movie than buy a whole set of VR devices. With this in mind, offering a VR exhibition is a good way to promote and expand the museum’s reach since admission tickets are as affordable as movies.
6.5.5 Preparatory work matters
Vividness was hypothesized to have explanatory power in the VR experience.
However, the quality of the VR devices used in the study served as a limitation. This was unfortunate because we didn’t anticipate the quality problem prior to collecting user feedback. A higher-grade electronic device is needed for optimal running of VR programs.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
47
Chapter 7. Conclusions
7.1 Summary
This study investigated the impact of VR design factors on user experience and museum performance. We already knew that interactivity, vividness, and realism are important VR design factors that could enhance users’ feelings and attract more NPM visitors. We assumed the VR design would perform better after incorporating a storytelling feature. Also, based on its wide use in advertising, we included priming as a moderating variable on the effect of storytelling. The complete model is shown in Figure 3.
NPM provided everything for the exhibition, including the VR headset and handlers. We guided the volunteers to explore NPM’s VR programs. They could view and interact with two kinds of NPM artifacts in VR. After the experiment, a questionnaire was administered for user feedback. Some visitors read the background and introductions posted on the wall, and we used this exposure as a form of priming.
Furthermore, we compared the VR effects across different kinds of artifacts. For this, we created three models. Model 1 does not consider the type of artifact, Model 2 focuses on calligraphy, and Model 3 focuses on painting. Each stimulus has its own unique features, so some of the results can be attributed to these elements.
Statistical analysis was then performed using the PLS method. The data show that the effect of VR design factors on UX is positive and significant with the exception of vividness. This is probably attributable to the poor quality of the devices used, which sometimes caused blurry images and noises. Nevertheless, the strong results overall (R squares) indicated that design factors accounted for up to 49.1% of the UX variance, a very satisfactory level of explanation. Furthermore, most of our hypotheses were supported. Both storytelling and technical factors served as important VR design
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
features. Moreover, storytelling showed stronger explanatory power for user experience when compared with other design features. These findings can help NPM understand and apply storytelling more effectively in its programs. This study also demonstrated that storytelling can be used to promote the museum, which NPM could incorporate in its next exhibition.
7.2 Limitations
This study had several limitations. First, the experimental content was limited to paintings and calligraphy, so only one work represented a whole category. Secondly, since the experiment was conducted on a college campus, visitors’ backgrounds were not very diverse, and the distribution by age and profession were not typical of the general population. Third, the location of the museum may have been a factor in the results. Many participants gave feedback that distance to our exhibition was too far from downtown. Visitors had to transfer at least twice on public transportation to get to our campus and then take a shuttle bus to the final stop. Fourth, the available VR equipment was limited to NPM’s existing supply, which included only one set of VR equipment. This meant we could only serve one visitor at a time, so the number of participants in one session was limited.
7.3 Future Research
One challenge for museums in the 21st century is to provide immersive experiences through creative and innovative technology such as virtual and augmented reality. The use of storytelling has increased recently as a way to guide museum visitors, replacing traditional exhibit-focused experiences with story-centric cohesive narrations with references to exhibits and multimedia content (Pujol et al., 2012). Those artifacts and Chinese culture generally are priceless, but younger generations don’t have an appreciation of this. So, NPM has sought many ways to present those ancient
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
49
masterpieces in creative ways. We anticipate the results will help NPM to improve their VR exhibitions through better understanding of visitor preferences and how to incorporate storytelling in the design of programs.
Finally, this study offers a new approach to the design of storytelling in museum VR exhibitions. The results may guide future work in evaluating the storytelling context and motivate innovative research in this area. Our study can benefit other museums who are considering a VR exhibition to help assess whether immersive storytelling could fulfill the comprehensive explication and meaning of the artifacts.
Acknowledgement
I would like to express my deep and sincere gratitude to my research supervisor, Dr. Hsin-Lu Chang, Professor of Management Information System, National Chengchi University, Taiwan, for giving me this opportunity. It was a great honor to study and do research under such resourceful place.
Every assistance provided by National Palace Museum was strongly appreciated.
Also I want to thank those who helped or joined this project as volunteers. I would like to express my thanks to my mother for her unconditional encouragement and support. My Special thanks goes to my partner Shao, who had a great teamwork with me.
Finally, my thanks go to all the people who have supported me to complete the research work directly or indirectly.
‧
Agarwal, R., & Karahanna, E. (2000). Time Flies When You’re Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage. MIS Quarterly, 24(4), 665.
https://doi.org/10.2307/3250951
Ahmed, R. (2018). IMMERSIVE VIRTUAL REALITY ADVERTISEMENT: EXAMINING THE EFFECTS OF VIVIDNESS AND INTERACTIVITY ON CONSUMERS’ PSYCHOLOGICAL RESPONSES. Retrieved from https://shareok.org/handle/11244/301311
Ahmed, R. (2018). IMMERSIVE VIRTUAL REALITY ADVERTISEMENT: EXAMINING THE EFFECTS OF VIVIDNESS AND INTERACTIVITY ON CONSUMERS’ PSYCHOLOGICAL RESPONSES. Retrieved from https://shareok.org/handle/11244/301311