CHAPTER 2 Related Work
2.4 Virtual Environments for Interactive Storytelling
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Finally, current research on generative interactive simulation with considerations to time and perspective often focuses on a limited aspect of presentation. Text-based systems address the grammatical accuracy in tense and focus, while 3D virtual drama platforms use temporal planning for agent and animation planning. Thus, we present a system that observes temporal issues in the plotline, with sufficient abstraction from the presentation.
2.4 Virtual Environments for Interactive Storytelling
Recent progression in development of virtual environments with autonomous character animation engines, novel interaction methods, automatic camera and lighting techniques, and intelligent drama management have brought interactive storytelling a step forward to take on a more immersive, expressive, and realistic form of presentation. In a qualitative survey of recent drama management platforms (Roberts & Isbell 2008), we note a few arising characteristics to observe and analyze interactive narratives, including replay value, the dilemma between authorial control and user autonomy, and other aspects of authoring and immersiveness.
A basic implementation of interactive narratives is in a text-based system, as demonstrated by the marlinspike interactive drama system (Tomaszewski & Binsted 2009), which can take in a specific set of typed user actions and returns pre-authored story segments.
The drama manager designed in the system records parameters in past story segments into its decision-making, in order to make decisions that maintain the logic and coherence of the overall narrative.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Swartout et al. (2001) first recognized interactive narratives as a vessel for integrating storytelling with multimedia environments that contain visual-audio content. From hence on, a wealth of research saw bloom for virtual storytelling environments with autonomous character, lighting, and camera models that adapt real-time to various scenarios (Lino et al.
2010; Chen & Li 2012). New ways of interaction such as voice and emotion detection used by Cavazza et al. (2009) and multiplayer models (Fairclough & Cunningham 2003) were built to provide even more interactive and immersive experiences. The former European project IRIS Network of Excellence—Integrating Research in Interactive Storytelling (Cavazza et al.
2008) developed drama management platforms that allow the authoring and simulating of interactive virtual stories in 3D environments. It is on this technological basis, The Theater platform developed by the MimeTIC team in IRISA/INRIA Rennes, which our research builds to establish the simulation part of the system.
Figure 4. EmoEmma uses a voice-emotion interface for user interaction
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 3
System Design
The design of the interactive narrative scenario generation and simulation involves five main parts: the Authoring Process, Linking Process, Interactive Story Generation, Discourse Representation, and the final Interactive Story Simulation. The relation between each step is demonstrated in Figure 5.
Figure 5. System Workflow
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
The details of each stage are explained in the following sections. In the first section we describe the authoring process, where the author may create and edit any number of story fragments according to their own creative process, creating a story pool which is the basis to all the following stories. The fragments are then tagged and linked together into a story graph, and through conditions and user-defined preferences, the fragments are filtered to generate a subgraph that fits all these constraints. Based on the implementation platform, the discourse of the narrative can be designed according to the content and then simulated. In this implementation, we demonstrate the abstraction of the authoring from the implementation by simulating the story both in text and in the 3D environment of The Theater.
3.1 Authoring Process
If you have had experience of writing a story, you may realize that the creative process does not always come at any particular sequence. For example: even if the events in your story are narrated in chronological order, your writing and thinking process is not necessarily so.
The authoring process of the interactive narrative could be seen as similarly spontaneous and unpredictable. Thus we hope to design an XML format to allow most flexibility and expressiveness in the interactive script. The most basic unit of the story script is what we call a “fragment” which ideally includes a small unit of action, description, or thought stream of one or a group of characters. For example, taking Gerald Prince’s example of a minimal story, “John was happy,” “John met a woman,” and “John was unhappy” could be fragments that could be arranged to create story variations (1974). The author does not need to assume any specific order or relationship between the fragments in this stage. It is similar to the
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
brainstorming stage of the creative process.
The authoring process is depicted in Figure 6 where the user can add and remove story fragments according to their own preferences. Descriptive characteristic tags can be added to the fragments for personalized filtering described in Section 3.3. The authoring fragments can be reused in various stories the author wishes to create.
Figure 6. Authoring process and the story pool
The information contained in the story fragment includes:
1. ID: A unique id for the fragment
2. Characteristics: Tags that the author can give each fragment. They may be keywords or other characteristics that help the author categorize the fragment or personalized filtering.
3. Parameters: Integer parameters that be given a value, incremented, or decremented.
4. Description: A short summary of what happens in the fragment.
5. Decisions: The fragment can embed other sub-fragments that allow decisions—a dividing of the story path or changing of story characteristics/parameters—based on
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
the user’s decisions to affect the outcome of the story.
6. Embedding Points: Certain points where the story can optionally branch out and play a specific fragment before continuing the fragment. This is a design to generate stories with varying point-of views and time sequences.
The result of the authoring process would be a pool of fragments that can be connected logically in the linking process (described in the next section) to produce a story graph for the interactive script.
3.2 Linking Process
3.2.1 Conditions
The second aspect of the linking process is setting conditions to the transitions. In a complicated branching story, there are so many paths to recombine story fragments that the author cannot keep track of which path is logical while others may be not. In this case, by evaluating the characteristics previously set in the authoring process, authors can now restrict the combinations of stories by evaluating the characteristics.
Suppose after linking the fragments, we have a collection of fragments that form a
“move” or what is similar to a chapter in a text story. Each move has entering points (initial states) and exiting points (transition states or end states). A sample move is depicted in Figure 7 below, where each node is a fragment, and the string sequence on the node is the identifying ID of each node, followed by characteristic tags, which are embraced in the curly brackets. The green node signifies an entering point into the move. The yellow nodes are
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
normal story fragments. Blue nodes indicate a transition from one move to another, while the pink “TheEnd” node ends the story.
Figure 7. A sample move with conditions on transitions
Using an example to explain conditions, in Figure 7 we see the edge between fragment FA-8 and Move B has a condition A-, which means that a path that does not go through FA-2 (which is the only fragment with the characteristic A-) would be illegal to go through that edge. For example, the path "Move A" → "FA-1" → "FA-3" → "FA-5" → "FA-7" →
"F-A-8" → "Move B" would be illegal. Thus conditions enforce logical precedence between story fragments.
With the use of conditions, we can allow the author more control over the story during the linking process and thus creating more believable and logical transitions.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
3.2.1 Moves and the Story Graph
Once we have conditions and edges to link the fragments into moves, moves can then in turn be linked together to generate complicated plotlines that involve multiple sequences of plot.
Moves can also branch to various other moves through the transitional nodes, thus creating a complicated story graph.
3.3 Story Generating and Filtering
Conditions and characteristics provide ways for the user to restrict the interactive script generation output to stories that conform to certain constraints. To do this, we use a filtering algorithm that evaluates the characteristic tags on story fragments and the conditions on the edges in order to determine if there are any constraints that are violated.
The proposed algorithm is described in Figure 8. A node means a fragment in the story graph. We first assume that the author is very thorough and subjective in adding characteristics to each fragment, tagging all sad story endings with a characteristic called tragedy T. In this case, the user of the system could set a constraint T = false, and the algorithm would perform a traversal through all the possible story paths, removing all the fragments that have the characteristic T+, ensuring that the resulting story is not sad.
Another way to set constraints is by giving values to parameters. Suppose a user would like to experience a story of limited complexity, defined by Propp (1928) as the number of moves (progression from recognition of a lack to its liquidation), with an increasing degree
‧
applied to filter paths that are too long.Figure 8. Algorithm for story generating
In detail, there are five main steps to the algorithm: (1) Setting user constraints (2) Removing contradicting nodes (3) Pasting user constraints (4) Propagating and reversing user constraints (5) Removal of dead ends.
Algorithm 1 Traverse (node N, constraints C)
1:
if C violates descriptors of node N then2: remove node N, and node’s incoming and outgoing edges
3:
end if‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
In the first step the user defines a set of constraints that they would like to see fulfilled in the ultimate outcome. Suppose the user would like a story where E = false (no evil character), M = true (with magical elements), we have an initial story graph as depicted in Figure 9 with the fragments M1F3 tagged with characteristic E, and fragment M1F5 tagged with characteristic M.
Figure 9. Example initial story graph
Once the constraints are set, the algorithm first carries out a depth-first search through the story graph to remove any contradicting nodes, i.e. any nodes tagged with the characteristic ‘E’ in this example, which represents Evil. We do not do anything with the M characteristic. We cannot determine whether a fragment is a desirable one simply by checking for the presence of M, since it may be fulfilled in other fragments of the storyline. The resulting story graph from the first step can be seen in Figure 10, where M1F3 is removed.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 10. First step: removing contradicting nodes.
Then from bottom up, the algorithm pastes the user constraints at each edge leading to TheEnd node, which ensures that the constraints are checked to be fulfilled before the story ends. This, however, does not guarantee that the constraints are fulfilled throughout the story, and can easily lead to an illogical story or dead end, since there are no constraints to limit where the story can lead throughout the story graph. Thus we need the third step: to propagate and reverse the constraints throughout the story graph. The act of propagating and reversing is done intermittently: given our current node position P, we take all the constraints on the outgoing edges of P and “reverse” them according to any changes within P. For example, if P is tagged with the characteristic ‘Magic’ and has the parameter change of
‘complexity+=1’ then the user constraints originally ‘Magic=true’ and ‘complexity>=2’ is reversed into only ‘complexity>=1’ indicating that condition of ‘Magic’ is fulfilled, and the complexity was incremented in this node. The union of all the reversed constraints for each
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
edge are propagated and pasted to the incoming edges of P to indicate that at least one of the reversed outgoing edges of P must be satisfied prior to entering P. This process of reversing and propagating ensures that all end constraints (as well as other edge constraints that are reversed along the way) are checked and validated at the beginning of the story. As demonstrated in Figure 11, we can immediately determine that the edge between fragment M1F2 and M1F6 is inappropriate due to the unfulfilled constraint M.
Figure 11. Pasting, reversing, and propagating end conditions
Though propagation and reversing already ensures that no dead ends will occur in the story, for the last step we still remove unfeasible dead ends to generate a complete subgraph (as shown in Figure 12) of the original story of all feasible story paths.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 12. Removing dead ends obtaining the final subgraph
Filtering of parameters and characteristics alone do not contribute to a suitable story.
Suppose the algorithm generates a story: Once upon a time there was a man called John. He did many evil things. Then he went to Heaven after he died.
Assuming there no unknown events in the story, even though the story is indeed simple and not of a sad ending, the reader can reasonably doubt the logic of the author. In order to preserve the logic of the story, the conditions that were set in the linking process must be strictly upheld, while ensuring that there are no dead ends.
The filtering and story generation mechanism not only provides personalization on the level of characteristics, but is also applied in our system to maintain story structure using an example of the six modes of plot proposed by Chatman (1980).
‧
Chatman’s six modes of plot:
a. An unqualifiedly good hero fails b. A villainous protagonist fails
c. A noble hero fails through miscalculation d. A villainous protagonist succeeds
e. An unqualifiedly good hero succeeds
f. A noble hero miscalculates, but then is redeemed
We are able to tag the fragments with customized characteristics as indicated below:
Tragedy (T)—Bad ending (combined with Genre) Hero (H)—A good hero
Miscalculation (M)—Hero makes a mistake
Then Chatman’s six modes of plot can be interpreted as the following logical expression of conditions: complexity>=4;‘T’=true; ‘H’=false is shown below in Figure 13.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 13. Sample story graph on the constraints complexity>=4; ‘T+’=true
3.4 Time and Perspective
The addressing of time sequences and perspectives in narratives is extremely challenging when it comes to narrative generation due to three main issues: logicality, interactivity, and presentation. First of all, if the story would require flashbacks or jumps in the sequence of events, how to preserve the logicality of the generated events is extremely important. For example: What happens in the present (such as opening a treasure chest) must be causally
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
logical with something that happened in the temporal past (found a key corresponding to the chest somewhere). Similarly interaction both in the present and flashback past most be coherent on the temporal timeline, for example, if a character chooses to open a chest in the present, in a flashback he or she must choose a path that leads them to the key (and the chest).
Finally, given that the logic of sequenced events is believable, how to present them such that the perceiver understands that there is a change in the sequence of events or show specific character perspectives (such camera movement) also requires knowledge in cinematography that cannot be addressed by the story structure alone. In this thesis we only address the first two issues and leave the last one for further investigation.
We address the issue of time sequence with the concept of “embedding,” the situation where one story event happens within the presence of another event. When embedding occurs within the story graph, we use revalidation to ensure the logicality and interactivity within and outside the embedded fragment. In the revalidation process, for every fragment the user chooses, removing any illogical branches following or preceding, while preserving all that are validated, thus preserving the interactivity. The algorithm for the revalidation process is as indicated in Figure 14.
There are three main steps to the algorithm: first, it checks the time frame of the fragment (whether it is currently an embedded node that is out of chronological order) and uses a flag to record the levels of embedding. Next, it revalidates the story graph based on the current node to ensure that the whole narrative sequence maintains its logicality, using the validation algorithm in Figure 8. Finally, it returns the next valid node calculated.
‧
Figure 14. Algorithm for real-time decision
However, due to the complexity of the revalidation process, the embedding is currently only proven on a minimal example with limited conditions (boolean conditions only) in order to demonstrate its capacity, but not included in the full demonstration or user evaluation.
The implementation of perspective filtering is much more straightforward, where we set a condition that has to be fulfilled throughout the story: the presence of a certain character from whom our perspective sets out. When the character is not present in a fragment, that fragment is still evaluated in order to preserve the logic of the events, but the presentation is repressed and hidden from the audience.
Algorithm 2 Real-Time Decision (Decision D)
1:if current node has embedding then
2:embedLevel+=1
3: push node N into levels stack L
4: next node N’ = embedding entrance point Ne 5:
embedFlag = true
6:
end if
7:
if embedFlag is false and embedLevel>0 then
8: current node N = pop L9:
end if
10: get next adjacent node N’ from current node N and decision D 11: for all incoming edges e of node N’ do
12:
C C∨conditions on e
13: end for14: Traverse(s, C) 15: return next node N’
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
An example of perspectives and time sequences can be seen in the second demonstration in chapter 4.
3.5 Discourse Representation
After generating a story, it has to be presented in some form of narrative discourse—whether in text, film, animation, or sound—to be experienced. In the system design, this research does not assume that the narrative is for any specific platform, and instead tries to model the high-level elements that apply to all forms of discourse.
3.5.1 System Representation
An XML high level story script is used to represent the story graph and its corresponding story tags, characteristics, fragments, and conditions. The story generation mentioned in the previous section is based on this XML script. The basic design of the XML script is as shown below:
<Story title="Whodunit vs. Howdhecatchem" author="Helen"/>
<move id="Investigation">
<transition num="Deny">
<descend desc="Convicted_Wrong">
<and>
<testCharacteristic variable="Murder" value="false"/>
</and>
</descend>
‧
<function id="Deny" description="The wife denies any knowledge of the murder." propp="" characteristic="Deny:wife:inspector:lie">
<embed id="Murder">
</function>
The story script includes the story title, its author, and specification on the moves and functions of the story. Conditional statements may be added on edges between transitions in the moves, while characteristic tags and parameters can be added on the fragments. In order to address time sequences, other moves and fragments may be embedded in fragments to create varied time sequences. The XML script can be flexibly edited, extended, and reorganized as fitting to the author’s creative process.
This XML script is the high level concise representation of the story, and its purpose is meant for system interpretation. Though it also represents the main concept of the story, it is not a suitable form of representation for most people. Therefore, in order to demonstrate the abstraction between content and discourse, story and form of presentation, we propose to simulate the story in two modes: first in the simple text-based narrative, and later in the 3D environment.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
3.5.2 Text Representation
The simplest form to present the interactive narrative is in text form, where each fragment has a corresponding piece of text. A sample output in text form is as follows with the conditions complexity<=1 and tragedy=true, and it simulates the user choice with a random seed every time there is a branch (such as in M1F2A there is a decision).
M1F1 There was once a poor father of twelve, just given birth to his 13th son.
M1F2 The poor father does not have enough money to feed his smallest son.
M1F2A The father decides to find a godfather for his son.
...
M1F9 The boy lives a difficult life, but in afterlife he has eternal joy.
TheEnd The End
Each fragment has a description preceded by its ID. If we output the whole story in a single paragraph, we would have something that looks like:
There was once a poor father of twelve, just given birth to his 13th son. The poor
father does not have enough money to feed his smallest son. The father decides to find
father does not have enough money to feed his smallest son. The father decides to find