Ping-Yi Liu1, Hung-Wei Lee2, Tsai-Yen Li1, Shwu-Lih Huang3, and Shu-Wei Hsu1 1{g9527, li, [email protected]}, Computer Science Dept., Natl. Chengchi Univ.
2{[email protected]}, Dept. of Applied Psychology, Hsuan Chuang University 3{[email protected]}, Dept. of Psychology, Natl. Chengchi Univ.
Abstract. In this paper, we propose an experimental platform for the design of
interactive TV. On this platform, designers are allowed to emulate broadcasting programs, define control functions, collect user interaction data, and incorporate external application systems. We also use several examples to demonstrate that through this platform one can easily add new functions and set up psychological experiments for evaluating these new functions. Two experiments have been conducted and their preliminary results are reported.
Keywords: Interactive TV, Experimental Platform, Emulated TV, TV
Watching Behaviors.
1 Introduction
Many researches about interactive TV focus on program recommendation [5], but we believe that a better design should be based on better understanding of the interactions between the viewer and the television. Therefore, we aim to conduct research on the design of interactive TV at the behavioral level. However, it is found that most commercial interactive TV services, such as TiVo1, do not allow designers to manipulate the controls and contents of the programs to support interactive experiments. It is not easy to conduct in-depth experiments on TV-watching behaviors without these supports. The lack of a proper experimental platform has hindered the development of psychological experiments for better understanding of the effects of interactive TV. Therefore, in this paper we propose an experimental platform based on the software development kit (SDK) in Windows Media Center2. for interactive television that supports the following functions: (1) Different watching modes. We provide three TV-watching modes, including a customized video-on-demand (VOD) system, an emulated broadcasting TV service, and a subscribed cable TV service for the demands of different experiments. (2) TV control. Customized controls such as TV interface design and media presentation methods are used to serve as instrumental functions. (3) Experiment data collection. The system could automatically record the viewer’s operations on a remote controller and signals from other physiological indicators. We will describe the above three functions in more details in the next three sections.
1 http://www.tivo.com
2 Support for Different Watching Modes
In order to control experiments and observe behaviors of viewers in different scenarios, we have designed three watching modes. The first mode is a customized VOD subsystem. In this mode, the programs can be arranged into categories according to psychological states that can be aroused by these programs such that we can observe resulting behaviors and acquire instrumental measures. Second, we have designed a subsystem called SimTV that emulates broadcasting programs with multiple channels. From the user’s point of view, the programs displayed are just like in a real broadcasting TV except for that the sources come from pre-recorded videos. A common problem on TV-viewing experiments with a real TV system is that the contents in live broadcasting channels are not the same for experiments done at different times. With the SimTV system, we can precisely control the program schedules for the same experiment at different times. The last watching mode is a subscribed cable TV service, which is a real cable TV environment that can be mixed with the SimTV environment to perform evaluation of a design in a real environment.
3 Customizing TV Control
An important feature of our system is that we can customize the interface, media contents and control functions for the aforementioned watching modes. Designers are allowed to change display functions such as on-screen-display (OSD) position and the overlay of a message dialog. In addition, the functions of keys on the remote controller can be easily redefined to accommodate the needs of experimental designs. Moreover, our platform can be incorporated with external program or devices to create innovative applications that traditional TV can hardly achieve. For example, an external 3D animation system has been integrated with our platform (as shown in Fig. 1) to design an alternative user interface that can provide a more effective way to communicate with the viewer according to his/her affective and/or attentive states.
Fig. 2. The viewer uses Wii Remote to
express negative emotion.
Fig. 1. Integration of external 3D animation
Although the detection of emotional states and the development of suitable treatments are still ongoing studies, the instrumental mechanism has been made ready for this goal on the platform. Another example of application that we have implemented is similar to the work proposed in [2]. We used Nintendo Wii Remote as an input device to create various real-time visual effects on images displayed on TV. For example, the viewer can fire a destroying action to crash the figure in the video by shaking the Remote to express their negative emotions such as anger, as shown in Fig. 2.
4 Experimental Data Collection
During the session of an experiment, the platform can automatically collect responses of the viewer from the remote controller and other devices that can be triggered programmatically. Currently we collect two types of data: (1) triggered events of the remote controller such as switching channels or adjusting the volume, and (2) video contents associated with these events. By exploiting webcams that are commonly available on regular PCs, not only images that are shown on TV screen but also the viewer’s body movements and his/her facial expressions are captured in synchronization, as shown in Fig. 3. The recording is turned on and off automatically as the TV is turned on/off and the designer can use the system to examine video clips according to event triggers or markers. On this experimental platform, we plan to incorporate measurements of multiple physiological signals such as electroencephalogram (EEG), skin conductance response (SCR), heart rate, blood pressure and body temperature. We attempt to find physiological signal patterns relevant to specific TV-watching behaviors or the viewer’s psychological states [3]. The results will then be exploited to build emotion and attention detectors for the design of further appropriate multimodal interactions with the viewer [1][4].
5 Examples of Experiments
We have used the functions provided by the experimental platform described in the previous sections to design and evaluate new functions for interactive TV. In this section we will describe two psychological experiments for two functions that are found important by most users in a focus group study.
5.1 Volume Adjustment Experiment
The proper audio level for TV watching depends on programs, channels, and user preference. We aim to design new volume adjustment methods and evaluate their effectiveness through psychological experiments. Five modes for adjusting the volume of TV have been designed. (1) Traditional mode: The volume changes step by step constantly according to the number or duration that the adjustment buttons are pressed. (2) Linear mode: The adjustment scale is linearly proportional to the difference between current level and a predefined level such that it can adjust the volume more efficiently when the difference is large. (3) Degression mode: The adjustment scale for continuous presses decreases gradually for the user to fine tune the volume when the ideal level is approached. (4) Frequency-dependent mode: The amount of adjustment depends on the duration between two consecutive presses. The faster the user presses buttons, the more adjustment will be implemented. (5) Semi-automatic mode: The user can press a special button to jump directly to a predefined volume level and then fine tune it as usual.
We have taken advantage of the control and scheduling functions to facilitate the design of this experiment. First, we remap the functions of related buttons on the remote controller to corresponding methods in each of the above modes. The experiment consists of five sections for five different modes in addition to the initial instruction and warm-up practice. In each section, we ask the participant to adjust the volume to a most comfortable level for several programs with different initial settings (such as silent, extremely loud, etc.). The arrangement of TV programs in this experiment has been facilitated by using the VOD watching mode of our platform. Control events created by the 20 participants are recorded and the efficiency of each control mode is computed. The preliminary results reveal that the semi-automatic mode is most effective for adjusting volume. It is also found that the linear mode and degression mode are more efficient than the traditional mode.
5.2 Channel Selection Experiment
In this experiment, we aim to understand what kind of presentation can help the user recall the channels that he/she has visited and would like to switch back. We have
percent of time. The feedback collected through a questionnaire after the experiment also indicates that the Prior-N function is accepted as a helpful function. Among the three types of channel symbols, the typical snapshot was expected to provide more relevant link to the program. However, the experiment result shows that the participants take the longest time to switch to their target channels by using this symbol type. We think a possible explanation can be that a snapshot of a just-viewed program could be too complex and probably only exists in short-term memory while the other two types of symbols are simpler and have stronger association with the type of programs displayed on the channels. However, more experiments and analysis are needed to clarify these two possibilities.
6 Conclusions
We have built a flexible experimental platform for the design of interactive TV. The platform serves to provide innovative functions and instrumental measures to facilitate the design and experiments of interactive TV. In the future, in addition to building up more physiological measures, we will continue to conduct various experiments to understand the subjective demands of the viewers during TV watching via various observation channels and analysis methods.
References
1. Jaimes A., Sebe, N.: Multimodal Human Com-puter Interaction: A Survey. In: IEEE Int. Workshop on. Human-computer Interaction (2005)
2. Lee, C.H., Chang, C., Chung, H., Dickie C., Selker, T.: Emotionally Reactive Television. In: Proc. of Intelligent User Interface (2007)
3. Nasoz, F., Lisetti, C.L.: Emotion Recognition from Physiological Signals for User Modeling of Affect. In: 3rd Workshop on Affective and Attitude User Modeling (2003) 4. Pantic, M., Rothkrantz, L.J.M. Toward an Affect-Sensitive Multimodal Human-Computer
Interaction. In: Proc. of the IEEE, vol. 91, no. 9, pp. 1370-1390 (2003)
5. Yu, Z., Zhou, X.., Hao, Y., Gu, J.: TV Program Recommendation for Multiple Viewers Based on user Profile Merging. User Modeling and User Adapted Interaction, vol. 16, no. 1, pp.63-82. (2006)
