CHAPTER 1 Introduction
1.3 Research Questions
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
1.3 Research Questions
Through the development of this framework and exploration of related issues, we address the following challenges: 1) Designing a flexible framework for generic story content, structure, and discourse in 3D interactive story creation 2) Developing story generating mechanisms that addresses personalization and theory conformity 3) Collect user feedback through a pilot study to gain preliminary observations of how users from various backgrounds would react to and use such a framework.
The ultimate goal is to provide design principles for future development of authoring and visualization tools and drama management techniques based on this framework. By providing design concepts and frameworks for authoring in interactive narrative platforms, authoring tools and simulation can be better designed to fit users’ creative processes, in response to the rise of digital interactive storytelling.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 2
Related Work
Narratives, interactive or not, have been a constant topic of interest in the fields of literature and communications. How to tell a compelling story by combining discourse (the expression), story (the action and plotline), and narration (the organization of the events) (Lothe 2000) is crucial to literary as well as communication design. Moreover, with interactive technologies in the picture, multimedia storytelling further explores the many aspects of automation, personalization, and replay-value in storytelling.
In this chapter we attempt to approach topics in interactive storytelling from its roots in narrative theory, the role of the author and computer technology in narrative creation, and current implementations of virtual storytelling environments that have influenced the design of our system.
2.1 Interactive Narrative Theories and Structure
Since Roland Barthes’s statement in his “The Death of the Author” (1967) that releases part of the authorial control to the reader, the role of the reader in the creative text has gone from singly receptive to participative in the work and its societal impact: “a text is not a line of
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
words releasing a single 'theological' meaning (the 'message' of the Author-God) but a multi-dimensional space in which a variety of writings, none of them original, blend and clash.”
Recent attempts to understand collaborative storytelling include the Story Circle experiment carried out by a cross-national research group to encourage the mass public to film parts of their lives and stories to share with the world (Hartley & McWilliam 2009).
Though the stories created are not interactive, the project was pioneering in leading the world into the era of digital storytelling in which every reader is also a creator, challenging the forms of storytelling to change with the advancement in technologies and communications.
Murray’s Hamlet on the Holodeck also provides an optimistic viewpoint for the development of computer assisted films and novels, believing that computer technologies can transform narratives into a more immersive and expressive form through computational technologies and recombination of new mediums (Murray 1997).
With the rise of multimedia technologies, the emergence of non-linear storytelling has been explored by researchers in the field. In her book Avatars of Story, Marie-Laure Ryan examines issues in electronic and transmedia storytelling (2008). She proposes that there are certain textual architectures that are typical of interactive narratives, which affect the plotline, discourse, and story as demonstrated in Figure 1. Ryan also identifies the types of interactivity commonly seen, and how they relate to characteristics such as perspective, time, and dramatization discussed in narrative theory.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 1. Plot architectures of interactive narratives proposed by Marie-Laure Ryan
Bryan Alexander (2011) also points out how the internet, multimedia, and computer games exploit the benefits of digital storytelling to create collaborative narratives, bring together new perspectives to the story, and blend the line between fiction and reality to establish a more immersive experience.
In order to break down a narrative to fine-grained fragments for story generating, we survey literary theories on narrative structure. Existing structuralist theories provide an excellent basis in designing a story structure comprised of fragments, or analyzable units of story, and these theories also serve as an important source to designing temporal and character arrangements such as flashback, chronology, or ellipses. Vladimir Propp had provided a theorem for the breaking down of folktales into 31 functions (Propp 1928), and the functions
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
are connected into sequences of plot units called “moves.” However, Propp’s formalism has been found to be too rigid and is considered linear to be modeled for interactive narratives due to its restrictions on the order and content of story events (Tomaszewski & Binsted 2007).
Chatman proposes models in which there is a separation between form and content, narrative and discourse to break down the story content into elements that can be manipulated and recombined individually, as shown in Figure 2 (Chatman 1980). He also provides story characteristics such as hero complexion, plotline, ending, times sequence, and etc. as points of interest in narrative analysis.
Figure 2. Chatman’s formalization of elements in narrative
In the areas of generative narrative creativity and literature of drama managements systems, the concept of story fragments has also been greatly discussed. Mateas and Stern (2002) introduced the concept of beats, which are a collection of goals and actions for each story fragment. The structure allows the user to specify what happens in each beat, and whether there are goals or preconditions, but does not address the question of reusability or time sequences, and they are also platform specific. Case-Based Reasoning methods were proposed for plot generation of specific structured stories (Turner 1992; Gervás et al. 2005).
Riedl and León (2008) proposed the concept of a “vignette” which is comprised of an action,
‧
the character, and its current states. Preconditions for each vignette are checked to be fulfilled before the story progresses. This brings the story down to very concise character-action-state statements, but may result in a more loosely structured story in terms of story themes or overall plotlines. Moreover, though the concept of beats and vignettes have a basis in planning algorithms for story generation, the preconditions in the vignettes and beats is sometimes just as or even more rigid and complicated than branching formalisms. The planning formalism also requires much more effort on the authoring side to maintain the logic of the overall storyline. To deal with complicated storytelling scenarios, branching algorithms are much more intuitive for current storytelling applications.
2.2 Applications for Interactive Narrative Creation
The collaborative nature of digital narratives calls for the development of more sophisticated authoring tools that are widely accessible to the public. With the diversity of multimedia platforms, it is difficult for any person to realize their creations without having prior knowledge of the specific platform. Thus the issue of script generation is closely tied to tools and interfaces to author the script. Past projects use graph visualization techniques and pre-designed slots for authors to fill in (Spierling, Weiß & Müller 2006). Some implementations allow users to recombine existing multimedia content to author non-linear narratives (Schneider et al. 2003)
Another interesting issue on authoring tools is not only how the tool can ease the creation process, but even further enhance the creative process by providing content or structural suggestions on the narrative. Chang et al. (2011) designed a computer-aided system that assists authors to create fairytales through image and text libraries that are
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
organized based on Propp’s formalisms. The designed interface allows users to browse and borrow existing content in their own stories.
The adaptation of famous literary works into interactive narratives (Spierling & Homann 2010) brings to light the question of the suitability of various genres of fiction to be adapted into an interactive narrative. In literary theory, there is no standard to what kind of story makes a good interactive narrative. The system EmoEmma adapts Madame Bovary (Cavazza et al. 2009) taking advantage of Gustave Flaubert’s detailed notes on character thoughts and emotions and combining it with the emotion detection in their interaction design. Thespian uses a variation of “Little Red Riding Hood” (Si, Marsella & Pynadath 2009) setting out from the wolf’s perspective in the story. Still others like Façade design new stories that fit their beat-based approach to drama planning (Mateas & Stern 2003), in which each beat is a specification of a sequence of interactive goals or NPC (non-player character) actions. The genre itself is new, and what kinds of content or forms of medium are suitable for interactive narratives is a topic to be further explored.
Figure 3. Façade (2005) is a well-recognized example of interactive storytelling
‧
Finally, in narrative generation, it is also relevant to mention the applications to NLP in interactive storytelling. Especially in text-based narratives, time and consistency is a topic of constant discussion when considering the textual presentation of events. Montfort (2007) specifically discusses the issue of grammatical tense in interactive narratives using an ordered tree representation of time. Lönneker (2005) introduces architecture for “levels” of narrative in story generation technologies to deal with natural language issues such as tense and perspective, which provides a basis for designing embedded story structures.
2.3 Story Generating and Content Filtering Techniques
Story generation techniques can assist the creative process by recombining story elements into various possibilities. Si et al. (2009) also proposed a planning technique by deciding character actions at runtime. However, they approach story generation by evaluating character-based goals and not plot-based, which may sometimes result in incoherent narratives with loose plot structures. Porteous et al. have used character perspectives to generate various storylines that fit the specified point-of-view (2010). They apply a planning technique that searches and chooses story segments that serve as a suitable next-step at runtime. In a later work by Porteous et al. (2011), they also employ temporal management to ensure feasible plotlines by pre-planning character-agent actions to fulfill plot goals.
Their research demonstrates how story characteristics such as perspective and time sequences can be modeled and incorporated into virtual narratives, but we further extend the characteristic filtering mechanism in this research to incorporate real-time user interaction.
By generating and filtering story fragments to produce new narratives, we not only enhance creativity, but also allow for more personal and user-conforming storytelling
‧
experiences. The issue has been addressed by Riedl (2009), who first proposed a model to set authorial goals at which story should aim. The branching structure allows easy pruning of the story graph to find paths that generate a conforming narrative. Yu and Riedl (2012) further explored the topic, using learning techniques to simulate user preferences and score certain stories.
From the previous work on generative and interactive narratives we draw a few observations. First of all, both branching and planning formalisms adopt precedence and conditional logic to preserve the temporal structure of the narrative. Branching is often considered more restrictive in generative power than planning, but has greater control over the coherent and logical expression of the narrative at runtime, as well as efficient management of user decisions (Ryan 2006). While planning algorithms provide the capacity for dynamicity in storytelling, their benefits are often insufficiently explored due to the limitations on the scalability of existing stories, while our proposed branching formalism, combined with an intelligent filtering mechanism, provides more room to explore storylines that demand strong control of plot logic while using filtering algorithms to achieve personalization. Second, and most importantly, though a few generative architectures enforce some form of temporal logic by using precedence conditions, in most cases they only work linearly (without real-time user interaction) and cannot present temporal variations. The issue of generating large-scale dynamic stories by altering perspective or temporal sequences is still an ongoing difficulty (Gervás et al. 2006). Though past work (Porteous et al. 2010; 2011) has addressed issues concerning perspectives and timing in virtual narratives, the addition of user interactivity and real-time dynamicity is a central contribution in this work.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Finally, current research on generative interactive simulation with considerations to time and perspective often focuses on a limited aspect of presentation. Text-based systems address the grammatical accuracy in tense and focus, while 3D virtual drama platforms use temporal planning for agent and animation planning. Thus, we present a system that observes temporal issues in the plotline, with sufficient abstraction from the presentation.
2.4 Virtual Environments for Interactive Storytelling
Recent progression in development of virtual environments with autonomous character animation engines, novel interaction methods, automatic camera and lighting techniques, and intelligent drama management have brought interactive storytelling a step forward to take on a more immersive, expressive, and realistic form of presentation. In a qualitative survey of recent drama management platforms (Roberts & Isbell 2008), we note a few arising characteristics to observe and analyze interactive narratives, including replay value, the dilemma between authorial control and user autonomy, and other aspects of authoring and immersiveness.
A basic implementation of interactive narratives is in a text-based system, as demonstrated by the marlinspike interactive drama system (Tomaszewski & Binsted 2009), which can take in a specific set of typed user actions and returns pre-authored story segments.
The drama manager designed in the system records parameters in past story segments into its decision-making, in order to make decisions that maintain the logic and coherence of the overall narrative.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Swartout et al. (2001) first recognized interactive narratives as a vessel for integrating storytelling with multimedia environments that contain visual-audio content. From hence on, a wealth of research saw bloom for virtual storytelling environments with autonomous character, lighting, and camera models that adapt real-time to various scenarios (Lino et al.
2010; Chen & Li 2012). New ways of interaction such as voice and emotion detection used by Cavazza et al. (2009) and multiplayer models (Fairclough & Cunningham 2003) were built to provide even more interactive and immersive experiences. The former European project IRIS Network of Excellence—Integrating Research in Interactive Storytelling (Cavazza et al.
2008) developed drama management platforms that allow the authoring and simulating of interactive virtual stories in 3D environments. It is on this technological basis, The Theater platform developed by the MimeTIC team in IRISA/INRIA Rennes, which our research builds to establish the simulation part of the system.
Figure 4. EmoEmma uses a voice-emotion interface for user interaction
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 3
System Design
The design of the interactive narrative scenario generation and simulation involves five main parts: the Authoring Process, Linking Process, Interactive Story Generation, Discourse Representation, and the final Interactive Story Simulation. The relation between each step is demonstrated in Figure 5.
Figure 5. System Workflow
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
The details of each stage are explained in the following sections. In the first section we describe the authoring process, where the author may create and edit any number of story fragments according to their own creative process, creating a story pool which is the basis to all the following stories. The fragments are then tagged and linked together into a story graph, and through conditions and user-defined preferences, the fragments are filtered to generate a subgraph that fits all these constraints. Based on the implementation platform, the discourse of the narrative can be designed according to the content and then simulated. In this implementation, we demonstrate the abstraction of the authoring from the implementation by simulating the story both in text and in the 3D environment of The Theater.
3.1 Authoring Process
If you have had experience of writing a story, you may realize that the creative process does not always come at any particular sequence. For example: even if the events in your story are narrated in chronological order, your writing and thinking process is not necessarily so.
The authoring process of the interactive narrative could be seen as similarly spontaneous and unpredictable. Thus we hope to design an XML format to allow most flexibility and expressiveness in the interactive script. The most basic unit of the story script is what we call a “fragment” which ideally includes a small unit of action, description, or thought stream of one or a group of characters. For example, taking Gerald Prince’s example of a minimal story, “John was happy,” “John met a woman,” and “John was unhappy” could be fragments that could be arranged to create story variations (1974). The author does not need to assume any specific order or relationship between the fragments in this stage. It is similar to the
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
brainstorming stage of the creative process.
The authoring process is depicted in Figure 6 where the user can add and remove story fragments according to their own preferences. Descriptive characteristic tags can be added to the fragments for personalized filtering described in Section 3.3. The authoring fragments can be reused in various stories the author wishes to create.
Figure 6. Authoring process and the story pool
The information contained in the story fragment includes:
1. ID: A unique id for the fragment
2. Characteristics: Tags that the author can give each fragment. They may be keywords or other characteristics that help the author categorize the fragment or personalized filtering.
3. Parameters: Integer parameters that be given a value, incremented, or decremented.
4. Description: A short summary of what happens in the fragment.
5. Decisions: The fragment can embed other sub-fragments that allow decisions—a dividing of the story path or changing of story characteristics/parameters—based on
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
the user’s decisions to affect the outcome of the story.
6. Embedding Points: Certain points where the story can optionally branch out and play a specific fragment before continuing the fragment. This is a design to generate stories with varying point-of views and time sequences.
The result of the authoring process would be a pool of fragments that can be connected logically in the linking process (described in the next section) to produce a story graph for the interactive script.
3.2 Linking Process
3.2.1 Conditions
The second aspect of the linking process is setting conditions to the transitions. In a complicated branching story, there are so many paths to recombine story fragments that the author cannot keep track of which path is logical while others may be not. In this case, by evaluating the characteristics previously set in the authoring process, authors can now restrict the combinations of stories by evaluating the characteristics.
Suppose after linking the fragments, we have a collection of fragments that form a
“move” or what is similar to a chapter in a text story. Each move has entering points (initial states) and exiting points (transition states or end states). A sample move is depicted in Figure 7 below, where each node is a fragment, and the string sequence on the node is the identifying ID of each node, followed by characteristic tags, which are embraced in the curly brackets. The green node signifies an entering point into the move. The yellow nodes are
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
normal story fragments. Blue nodes indicate a transition from one move to another, while the pink “TheEnd” node ends the story.
Figure 7. A sample move with conditions on transitions
Using an example to explain conditions, in Figure 7 we see the edge between fragment FA-8 and Move B has a condition A-, which means that a path that does not go through FA-2 (which is the only fragment with the characteristic A-) would be illegal to go through that edge. For example, the path "Move A" → "FA-1" → "FA-3" → "FA-5" → "FA-7" →
"F-A-8" → "Move B" would be illegal. Thus conditions enforce logical precedence between story fragments.
With the use of conditions, we can allow the author more control over the story during the linking process and thus creating more believable and logical transitions.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
3.2.1 Moves and the Story Graph
Once we have conditions and edges to link the fragments into moves, moves can then in turn be linked together to generate complicated plotlines that involve multiple sequences of plot.
Moves can also branch to various other moves through the transitional nodes, thus creating a complicated story graph.
3.3 Story Generating and Filtering
Conditions and characteristics provide ways for the user to restrict the interactive script generation output to stories that conform to certain constraints. To do this, we use a filtering algorithm that evaluates the characteristic tags on story fragments and the conditions on the
Conditions and characteristics provide ways for the user to restrict the interactive script generation output to stories that conform to certain constraints. To do this, we use a filtering algorithm that evaluates the characteristic tags on story fragments and the conditions on the