Not Coded “yeah, we’ll come to that in a minute”
Issue Distribution of the Session
After arbitrating there were 1,280 segments that contained FBS issues. Ta-ble 3 summarizes the counts and percentages of each issue.
Table 3 Count and Percentage of Issues
F Be Bs S D
Count 47 275 369 512 77
Percentage 4.0 21.0 29.0 40.0 6.0
The percentages of FBS issues reflect the nature of the session – they were mainly borrowing behavior and structure of other objects. The behav-ior issues occupied half of the counts followed by structure. The low count of D was because only shared documentations were coded. Not withstand-ing the nature of the session, functional aspects were discussed.
The percentages in Table 2 only give an account of the whole session, however according to Asimow elementary model [7], design can be char-acterized by a series of cycles through analysis of the problem, synthesis of a solution, and evaluation of the solution. If Asimow's model is mapped onto this ontology, the analysis of the problem will involve function is-sues; the synthesis of solution will involve structure issues and the
evalua-J. Kan, evalua-J.S. Gero and H-H. Tang 8
of designing the number of the FBS issues needs to be counted within a sequence.
In the next sub-section, the contribution of FBS issues over the session will be presented. Results are also presented that differentiate the contribu-tion of individuals and sum them for the whole team.
Distribution and Variations of FBS Issues
In order to obtain a more fine-grained understanding of how the issues are distributed, a window of 128 segments is taken and moved segment by segment from the beginning to the end of the protocol. This produces an averaging of that issue over those segments. With this 128 segments slid-ing window the number of FBS issues produced by an individual is counted and presented in Figures 6 to 10 for issues F, Be, Bs, S and D re-spectively. The horizontal axes show segment numbers and the vertical axes show issue counts. In each of the graphs the top surface of the graph shows the overall behavior of the team, while the shading maps onto indi-vidual team members.
Fig. 6
F issue distribution of individuals with a 128 segment moving window.Fig. 7
Be issue distribution of individuals with a 128 segment moving window.9
Measuring Cognitive Design Activity Changes 9
Fig. 8
Bs issue distribution of individuals with a 128 segment moving window.Fig. 9
S issue distribution of individuals with a 128 segment moving window.Fig. 10
D, issue, distribution of individuals with a 128 segment moving window.As expected in a brainstorming session the structure issue, S, is the domi-nant issue. Again as expected in a brainstorming session the issues of ex-pected behavior, Be, and behavior from structure, Bs, are relatively low as team members had been advised to suspend judgement. One unexpected result is that the F issues concentrate towards the middle rather than at the beginning of the session as has been observed elsewhere [8].
From Figures 6 to 10, qualitatively it appears that there are changes in
J. Kan, J.S. Gero and H-H. Tang 10
tested by counting the first, middle and last 300 segments of each session, giving us a total sample of 900 issues spread across all members of the team, Table 4 and Figure 11.
Fig. 11
The distribution of design issues of individuals within the first, middle and last 300 segments of the session.Table 4. FBS issue variations of individual team members
F Be Bs S D Sum
Allan First 4 21 11 31 12 79
Middle 6 24 20 28 9 87 Last 0 15 11 27 1 54
Chad First 0 5 11 15 4 35
Middle 1 2 4 9 2 18 Last 0 5 19 21 1 46
Todd First 3 7 8 20 6 44
Middle 6 8 26 23 5 68 Last 0 17 14 35 1 67
Rodney First 0 1 3 6 0 10
Middle 0 1 3 3 0 7
Last 0 1 2 4 0 7
Jack First 3 12 6 20 5 46
Middle 2 9 11 17 2 41 Last 2 8 16 27 2 55
Tommy First 1 9 24 26 5 65
Middle 3 17 19 22 0 61 Last 0 14 22 35 0 71
Sandra First 1 2 6 12 0 21
Middle 1 4 8 4 0 17
Last 0 0 0 0 0 0
11
Measuring Cognitive Design Activity Changes 11
Table 5 shows the results, for each individual member of the team, of two tailed paired-t tests carried out for testing the following:
• FBS issues distribution of the first 300 segments against the same issues in the middle 300 segments,
• FBS issues distribution of the first 300 segments against the same issues in the last 300 segments, and
• FBS issues distribution of the middle 300 segments against the same issues in the last 300 segments.
Table 5. Probabilities of paired-t test of individual's FBS issue count First and Middle
(p) Middle and
With paired-t test, the test statistic is t with n-1 degrees of freedom. If the p-value associated with t is low (< 0.05), there is evidence to reject the null hypothesis that the difference between the two observations is zero.
Row 1 of Table 4 suggests, with a high probability, that Allens’ distribu-tion of FBS issues at the end of the session is different from the middle and start of the session. Other than for Allan, the variations of the distribution of FBS issues in the others are not conclusive.
Producing Design Processes from a Linkograph
Linkography was first introduced to protocol analysis by Goldschmidt [9]
to assess the design productivity of designers. The design protocol is de-composed into small units Goldschmidt called “design moves”. Gold-schmidt defined a design move as “a step, an act, an operation, which transforms the design situation relative to the state in which it was prior to
J. Kan, J.S. Gero and H-H. Tang 12
She states that the links are established by discerning, using domain knowledge and common sense, whether a move is connected to the previ-ous moves.
Using the FBS coding scheme design processes are an automatic conse-quence of the generation of a linkograph as the two ends of a link have an FBS issue and the transition between those issues defines a design process.
The 8 design processes in Figure 1 map onto the start and end issues of a link in the linkograph, Table 6.
Table 6. FBS Processes
Figure 12 shows a linkograph connecting the design issues. Column 1 in this study is the participant, column 2 is the segment number and column 3 is the design issue of the protocol. The dots represent the segments and the links and the gray arrow lines represent the derived design processes. The four links represent four design processes.
Fig. 12
Exemplar case showing the production of the design processes from the construction of the linkograph.Segment 3 has two links, which indicate it spawns two processes: analy-sis (S > Bs) and documentation (S > D). Considering the participant who responded as the one who contributed to that design process, the link from segment 1 to segment 2 is a formulation process by participant B.
Design Process Link Formulation F>Be Synthesis Be>S Analysis S>Bs Documentation S>D Evaluation Be<>Bs Reformulation I S>S Reformulation II S>Be Reformulation II S>F
13
Measuring Cognitive Design Activity Changes 13