CHAPTER 2 Related Work
2.4 HCI
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
2.3 Computer-aided system
Because of the rapid development of information technologies, computer has become more involved in the process of creating contents by people or by computer itself in a digital form. The applications of computer-aided systems in stimulating creativity have already been developed for several fields. For example, computer-aided design systems use decomposing/reconstituting elements and transforming/exploiting procedures to help creative process (Liu, 2000; Xu, Li, &
Su, 2009). Unlike shapes, most documents contain only texts, photographs, and figures; and, therefore, cannot be used to stimulate creativity simply by deformation. Some researchers collect various kinds of thinking from others as a source of content variation (Nakakoji, Yamamoto, &
Ohira, 1999; Huang, Li, Wang, & Chang, 2007) in order to facilitate brainstorming. Computer-assisted creative thinking has been explored in the applications of education and design, but not much in narrative yet. The majority of research in computer-aided narrative has focused on supporting the creation process. For example, computer systems are designed to let people arrange plots presented by digital photographs, digital pictures, or texts written by users (Brna, Martins, & Cooper, 1999; Balabanovi , Chu, & Wolff, 2000; Landry & Guzdial, 2006). Most work asked user to prepare the figures by themselves, but in our system, we will provide pictures and text from others to stimulate the user’s creativity.
2.4 HCI
A good user interface can allow a user to concentrate on more his/her creative process. Nakakoji, et al. (1999) designed two systems to support collective creativity in design by using visual images. The IAM-eMMa system (Nakakoji, et al. 1999) lets the user specify his requirement, and
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
conduct surveys on some adjectives of pictures defined by other users and then displays the results of surveys in 2D or 3D spaces. Nakakoji pointed out that EVIDII was more welcomed than IAM-Emma in their experiments. In addition, they think deriving a system needs to be based on the following elements: contextualized, respectable, trustful, and enabling appropriation. Shneiderman (2000) proposed four phases when designing a user interface to support innovation. They are collect, relate, create and donate. The four phases came from some beliefs: new knowledge is built on previous knowledge; powerful tools can support creativity;
refinement is a social process, and creativity work is not complete until it is disseminated. This research also provides important insights for the design of our system.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 3
System Design and Implementation
In order to help a writer find elements for story writing, this study would like to design a system to stimulate writer’s creativity. The challenges of designing such a system include: (1) how to stimulate creativity, (2) how to design a good user interface that does not bother the user. The main goal of this study is to help a writer associate/find more ideas and reduce the effort about doing so. To achieve this goal, we design the system based on the knowledge that we have collected in the literature.
3.1 The elements to stimulate creativity
It is always easier to think and associate things by concrete objects instead of thinking by nothing but ourselves. To design a system that can stimulate a user’s creativity when s/he is writing a story, we have two hypotheses of the system as listed below:
1. Pictures and other people’s thinking will stimulate the imagination field and range.
2. Attributed information is effective in creative thinking.
Our system provides two types of elements: picture and text. A picture can take more attention than a text at the first glance. And the contents of picture and text are expressed in
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
to describe detailed things. Pictures and text have their own advantages and disadvantages and are complementary with each other.
In order to make use of pictures and text related to them in our system, we have to collect these story elements first. We have chosen 50 pictures extracted from 5 different picture books as a part of elements and invited people to write attributed notes. Attributed note is a way to classify elements of a story or even a picture. When a writer is writing a story, s/he would seek different elements such as location and character, the writer would also find objects or actions from a picture. In our system, we use Character, Location, Object, Situation, Action and Theme (CLOSAT) as the attributes for the notes. To collect the CLOSAT cards, five people were invited and each of them was assigned ten pictures and wrote their thinking of CLOSAT for every picture with computer. Except colleting the CLOSAT cards of pictures, we also observed their procedure and applied it as the sub-categories of our coding system. This study will illustrate it in more detail in section 4.3.
3.2 The user interface of the PAN system
For user interfaces supporting innovation, Shneiderman (2000) has developed a framework called Genex that contains four phases: Collect, Relate, Create, Donate. Our system was designed based on the Collect and Create phases of the Genex framework. The Relate and Donate phases that are used to consult with peers and disseminate the result are temporarily ignored in this work. We expect to implement them in the system by linking to the Internet and
•‧
works stored in libraries. For our system, it means to show the user the pictures and related texts in a panel called idea exploring. The Create phase contains explore, compose and evaluatepossible solutions, and it is implemented with a data buffer panel and a story writing panel in our
system. Data buffer is a panel for a user to collect elements that s/he would like to use; the user can arrange the chosen elements to explore, compose, or evaluate his/her thinking, or use it as a memory buffer. The story writing panel is a panel for writing the story. By these panels, the main process exercised by a user in the system is called exploring-collecting-writing, and the user can go back and forth in the process. The interaction between the user and the UI are depicted in Fig.3-1.Fig.3-1: The interaction between the UI and the user.
Story writing
Read information or write down thinkings of
data
Constructure the story View the story
data Write Complete
the story
•‧
Then s/he begins to arrange data, organize his/her thinking, elaborate, and write story in the story
writing panel. When developing a story, the user may go back to the idea exploring panel if s/he
needs a new idea or to the data buffer panel to recall her previous thinking. The relationship of the three panels becomes a loop through the user’s activities.3.3 The structure of the PAN system
The structure of the PAN system contains model, listener and display view. As Fig.3-2, in the beginning, the system gets the data from database, sets the display through layout, and shows the elements in the idea exploring panel.
Model
Layout Show Save
Attribute
Panel Writing Story Panel
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
The attributes module controls the different types of elements (CLOSAT and picture) to be used in the display. When the user changes the types of elements, the shown data would change through the setting of the show attribute. The user can also write and save notes into the database as new elements. The display view module contains the three main panels mentioned above.
3.4 How to show elements in the PAN system
Fifty pictures and their related CLOSAT notes were collected for the current implementation of the PAN system. With such an amount of elements or more, how to show them in the idea
exploring panel is an important issue. Our system uses arrange and group as the two ways to
show elements. The arrange way, the default in the PAN system, is to show elements in some predefined order (such as alphabetical). Since pictures are more attractive at the first glance, the system shows only pictures in the beginning. The group way shows a single picture and its related CLOSAT notes at the same page. So the user can see the picture and read the notes at the same time.3.5 Implementation of the PAN system
In this study, we have implemented the PAN system with the ideas mentioned above. The overview of the PAN system is shown in Fig.3-3. The frame that contains many pictures is the
idea exploring panel; and the frame at lower left, called My folder, is the data buffer panel; the
lower right frame with a writing story label is the story writing panel.•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
Since we want the participants to focus on exploring data in the beginning, the idea
exploring panel is maximized when a new session starts. In the idea exploring panel (see
Fig.3-4), the top is an area for command buttons. When user clicks the second or third button, he can choose to show the data items by arranging them in one of the two ways: arrange and group.The ‘Arrange’ button is the default style. Pressing this button shows only pictures while pressing the ‘Group’ button shows one picture and its CLOSAT cards together (Fig.3-4.a). After pressing the leftmost button, named ‘Choose show content’, seven green buttons will show up on the second line. These green buttons are corresponding to pictures and CLOSAT notes. The user can choose the kind of attributes that they would like to see or hide by clicking on these green buttons (Fig.3-4.b).
The data item itself can be a button. Through double-clicking on it can show the related CLOSAT cards as in the case of pressing the ‘group’ button (Fig.3-5.a). There are also some buttons for data items. Clicking the button on the top-left can show a note panel that allows the user to write down its annotation with CLOSAT attributes (Fig.3-5.b). The system stores the inputted note after the user clicks the ‘save’ button. The user can click the second button to zoom the picture to a specific size (Fig.3-5.c). It is a simple way to make the picture larger, and the user can also scale the picture by dragging it with the right mouse button.
Fig.3-6 shows the data buffer panel (called “My Folder” in the user interface), which is a simple panel for the user to store data that they have collected for constructing a story. The story
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
pictures into the image bar to make them into a sequence as the plot of the story. They can then write down the pieces of a story and repeat the process until the completion of the story.
Most of the time the user uses only the mouse to operate the system. Through the left click one can drag, move, and choose elements. The user can also scale every element in the system by right clicks. The scale operation allows the user to take a clearer view at the details of the data. The layout of the three main panels can also be customized.
Fig.3-3: An overview of the user interface when the system begins.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b)
Fig.3-4: (a) The group style. (b) After clicking ‘choose show content’ button, seven green buttons are shown and the user can decide to hide/show different attributes.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
(c)
Fig.3-5: (a) Showing related text of a picture when user double-clicks it. (b) The note area. (c) After clicking the second button of the data item, the picture is larger than others.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 4
Experiments and Results
To verify that the PAN system can help stimulate user’s creativity and find ideas for writing a story, we have designed a pre-test and a main experiment in this study. The main goal of pre-test is to find participants whose level of divergent thinking has no significant difference, and then these participants would be invited to take part in the main experiment.
4.1 Pre-test: Test of the ability of divergent thinking
4.1.1 Procedure
We designed a pre-test to find participants whose ability of divergent thinking is about the same.
In the literature, there are several ways to evaluate a person’s creativity. Since our system uses pictures and text cards to stimulate user thinking, we choose to use Guilford’s Alternative Uses Task to assess participants’ creativity (Guilford, Christensen, Merrifield, & Wilson, 1978).
Guilford’s test asks participants to list as many ways of usage as they can for a daily object. The collected test data are analyzed from four aspects: originality, fluency, flexibility and elaboration.
There are six participants and ten daily objects in this experiment. We have invited six
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
were given the names of these objects one by one and asked to write down what they thought for each object in two minutes. We use originality, fluency, flexibility and elaboration to analyze the data and sum up the scores for each aspect; the score is then used as the creativity index of each participant.
4.1.2 Experimental Result
Table 4-1 shows the creativity indexes of the six participants for each object. To know if a pair of participants whose creativity level is about the same, we also compare all pairs of participants as show in Table4-2.
Table 4-1: The creativity index of participants measured by Guilford’s Alternative Uses Task.
Participant Objects
1 2 3 4 5 6
Brick
17 18 23 22 14 14Car tire
25 12 11 15 13 13Barrel
31 15 20 19 19 21Pencil
25 17 15 26 27 17Shoe
27 18 19 26 15 20Hook
28 16 17 21 15 10Notepaper
29 9 14 27 14 11Paper clip
29 12 15 19 12 15Racket
30 8 13 23 21 24Tissue paper
28 11 14 26 19 11We use The Wilcoxon Rank Sum Test to find the participants whose ability of divergent
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
among participant 2, participant 3, participant 5, and participant 6 (P >= 0.05, two-tailed test).
These four subjects were then invited to take part in the main experiment.
Table 4-2: The results of comparing participants’ creativity.
Participant 1 2 3 4 5 6
1
-2
4.33e-05 -3
7.58e-05 0.247* -4
0.012 0.0001 0.002 -5
0.0003 0.143* 0.853* 0.015 -6
7.58e-05 0.481* 0.684* 0.005 0.529* -4.2 Main Experiment
4.2.1 Procedure
We invited the four participants that we had found in the pre-test to participate in the main experiment. In the main experiment they were asked to write a folktale with computer in one hour, all of them are given the same set of 50 pictures for inspiration of ideas. But one of the four particiants was asked to use Microsoft Word and general software packages in MAC: Finder and Preview.app to assist the process of creating a story. The other three participnats used our PAN system during the experiment. Immediately after they finished the task, they were asked to do protocol analysis.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
4.2.2 Experiment Result
Participant 1: without the PAN system
Participant 1 spent around 1 hour and 52 minutes to complete her story without the PAN system.
Her creating style belongs to the category of telling a story from pictures; that is, her story was highly related to the pictures. In the story, she wrote 1254 words (in Chinese) and chose 11 pictures to present the underlying story. Fig.4-1 shows the snap shots of the screen in the process of creating. In the beginning of the process, she browsed all the pictures, and chose some pictures she would like to use and dragged them to Microsoft Word. After chosing pictures, she started to write her story for a while but bothered by the cumbersome of scrolling a panel in Mircrosoft Word. Thus, after a while she decided to delete all the pictures she had chosen and just wrote. She added pictures afterward when she finished the story.
Participant 2: with the PAN system
Participant 2 spent 1 hour and 44 minutes to create a story with the PAN system. Evelen pictures were chosen to present the 1375 words in Chinese. Fig.4-2 is her process of creating. Participant 2 browsed the pictures and the text cards in the beginning and then started to write her story. Her behaviors of writing and choosing pictures alternated in the process. The relationship between the pictrues and her story were more abstract compared to Participant 1. For example, she chose a picture to present the feeling of boisterousness but the character or the place was not directly related to her story.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
Fig.4-2: Participant 2’s process of creating.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
Participant 3: with the PAN system
Participant 3 spent 1 hour and 33 minutes on writing the story. The participant did not choose any picture and totally wrote 669 words in Chinese. Fig.4-3 shows the snap shots of the screen in the process of creating. Like other participants, participant 3 browsed pictures and text cards in the beginning but he wrote some notes for CLOSAT, which is different from the other participants. When he was browsing the pictures or reading a text card, he made assosication in his memory, wrote it down, and then attributed his thinking by CLOSAT. After collecting all ideas, he started to utilize and combine all these ideas to complete his story. No picture was chosen to present the story because the participant thought that it was hard to associate the pictures with his story. (The pictures’ style were cartoon while his story is more modern.)
Participant 4: with the PAN system
Participant 4 took around 1 hour to develop his story. There are 14 pictures to present his story that contains 1176 words in Chinese. Fig.4-4 shows the screen snapshots in the process of creating. Participant 4 used only pictures to help him develpe his story. In the beginning, he browsed the pictures, and chose some pictures he thought he would use, and then dragged them to the My Folder panel for further examination. Then he dragged one picture he decided to use to the Writing Story panel and scaled up the picture in My Folder panel to see more details. Like participant 1, participant 4’s creating style was telling a story from pictures. Participant 4 alternatively switch between seeing pictures and writing story during the process.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
Fig.4-3: Participant 3’s process of creating.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
Fig.4-4: Participant 4’s process of creating.
Result Summary
Table 4-3 shows the conditions of participants and the results. Participant 3 typed 1185 words in the whole creation process, which include 669 words for story and the remainder for his ideas.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
Table 4-3: The conditons and results of participants.
Gender Use PAN Pictures Num Words Num Time
Participant 1 F
11 1254 1h52minsParticipant 2 F
V 11 1375 1h44minsParticipant 3 M
V 0 669+516(ideas)=1185
1h33mins
Participant 4 M
V 14 1176 1h4.3 Establishing The Coding System
Since creativity is demonstrated in the process of problem solving, it is important to analyze the process of how a participant creates a story. It is difficult to directly compare the story contents due to the narrative abilities of the participants may be different. Therefore, we use the research method of content analysis to analyze the process of story creation.
For the encoding system, researchers have published several models of design process. Chen (2002) took the operation of the computer into consideration, and developed Purcell and Gero’s model (1998) into four categories of fields: conceptual, operational, perceptual, and evaluation, as a means to analyze the cognitive process of using computer media. To establish the sub-category, we recorded the behaviors of the people who wrote the CLOSAT cards. Through the observation of their processes, we found that the behaviors include seeing, associating, recalling, evaluating, producing, writing, revising, scaling, and dragging when they were writing with the PAN system. We adopt Chen’s structure to map out the behaviors we have observed and established our coding schemes as shown in Table 4-4.
•‧
The conceptual field represents the category of behaviors observed when the user generates new ideas or recalls an element. There are four sub-categories: Pt, Dp, Rp and Rt in this field.
The operational field is used to represent user’s operations on the system and actions. These system operations include Dr, Sca, W, and Re while the user’s actions include seeing a specific picture or text card in the system (Sp, and St). The user may be attracted by a picture or text card because the picture’s features or the text. Therefore, we use the attributes of CLOSAT as the subcategories of the perceptual field. When the user makes a description of a picture, a text card, his/her thinking, or the system, the behavior is considered as an evaluation. Therefore, we have designed four sub-categories in the evaluation field: Ep, Etxt, Et and Es.
Table 4-4: Coding system for the main experiment.
Category Sub-category Definition
Conceptual Pt Producing a thinking.
Rp Recalling a picture.
Rt Recalling a thinking.
Operational Sp Seeing a picture.
St Seeing a text card.
Dr Dragging an element to another frame.
Sca Scaling an element by moving the mouse.
W Writing the story.
Re Revise the story.
Perceptual C Associating a character.
L Associating a location.
O Associating an object.
S Associating a situation.
A Associating an action.
T Associating a theme.
Evaluation Ep Evaluating a picture.
Etxt Evaluating a text.
Et Evaluating a thinking.
Es Evaluating the system.
•‧
國
立立 政 治 大
㈻㊫學
•‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 5 Analysis and Discussion
By comparing the participants’ processes of creating a story, we found that different users use the PAN system in different ways. Table 5-1 indicates the different conditions and observed behaviors or facts in the experiments by the four participants. Participant 1 did not use the PAN
By comparing the participants’ processes of creating a story, we found that different users use the PAN system in different ways. Table 5-1 indicates the different conditions and observed behaviors or facts in the experiments by the four participants. Participant 1 did not use the PAN