This article was downloaded by: [National Chiao Tung University 國立交通大學]
On: 28 April 2014, At: 04:37
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office:
Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Behaviour & Information Technology
Publication details, including instructions for authors and subscription
information:
http://www.tandfonline.com/loi/tbit20
The effect of time pressure on expert system
based training for emergency management
Dyi-Yih M. Lin & Yuan-Liang Su
Published online: 08 Nov 2010.
To cite this article: Dyi-Yih M. Lin & Yuan-Liang Su (1998) The effect of time pressure on expert system
based training for emergency management, Behaviour & Information Technology, 17:4, 195-202, DOI:
10.1080/014492998119409
To link to this article:
http://dx.doi.org/10.1080/014492998119409
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)
contained in the publications on our platform. However, Taylor & Francis, our agents, and our
licensors make no representations or warranties whatsoever as to the accuracy, completeness, or
suitability for any purpose of the Content. Any opinions and views expressed in this publication are
the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis.
The accuracy of the Content should not be relied upon and should be independently verified with
primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever
caused arising directly or indirectly in connection with, in relation to or arising out of the use of the
Content.
This article may be used for research, teaching, and private study purposes. Any substantial
or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or
distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can
be found at
http://www.tandfonline.com/page/terms-and-conditions
The eŒect of time pressure on expert system
based training for emergency management
DYI-YIH M. LIN and YUAN-LIANG SU
Institute of Industrial Engineering, National Chiao Tung University, H sinchu 300, Taiwan, ROC; em ail: u8233801@ cc.nctu.edu.tw
Abstract. In many emergency situations, hum an operators are required to derive countermeasures based on contingency rules whilst under time pressure. In order to contribute to the hum an success in playing such a role, the present study intends to examine the eŒectiveness of using expert systems to train for the time-constrained decision domain. Emergency management of chem ical spills was selected to exemplify the rule-based decision task. An Expert System in this dom ain was developed to serve as the training tool. Forty subjects participated in an experiment in which a computerized information board was used to capture subjects’ rule-based performance under the manipulation of time pressure and training. The experiment results indicate that people adapt to time pressure by accelerating their processing of rules where the heuristic of cognitive availability was employed. The simplifying strategy was found to be the source of hum an error that resulted in undesired decision performance. The results also show that the decision behaviour of individuals who undergo the expert system training is directed to a normative and expeditious pattern, which leads to an improved level of decision accuracy. Implications of these ® ndings are examined in the present study.
1. Introduction
Expert Systems (ESs) are a class of computers that emulate the reasoning process of human experts in the area of their specialty (Hayes-Roth et al. 1983). The role of ESs has been in acting as decision aids in order to support humans in the cognitive requirem ents of a speci® c task (Pew 1988). The expert level assistance provided by ESs has improved decision-making perfor-mance in a wide variety of application areas, such as industrial supervisory control, medical diagnosis, and computer con® guration, etc (Turban 1993).
However, there are a number of decision environ-ments where the utilization of ESs as a decision aid would not be viable. These application scenarios, the interest of the present study, mainly involve emergency managem ent of risks where hum an operators are required to make decisions under extreme time pressure.
M oray (1988) noted that `in many error situations . . . the tim e constants are such that there will be no chance to invoke the aid.’ In light of the small opportunity to access ESs for advice in limited time situations, the m erits of ESs under such circumstances thus clearly shift to reside in their utilization as a training aid (M cFarland and Parker 1990, Sharit et al. 1993).
The attractiveness of using ESs to train for tim e-constrained emergency m anagement stems primarily from the following rationale. First, in situations that permit no utilization of ESs as decision aids, the responsibility for decision making relies totally on the hum an operators themselves. Unfortunately, successful handling of emergency disturbances is likely to be hampered by the human’ s tendency to make errors in such situations (Rasmussen 1982, Reason 1990). There-fore, it is required that humans be adequately trained in order to develop eŒective programmes for em ergency m anagement.
Second, human decision making in emergency man-agement is often characterized by the need to reason extensively over knowledge expressed in rule-based form (e.g. contingency procedures). Speci® cally, the rule-based decision task is carried out by recognizing system and
/
or environm ent symptoms and associating rules with those sym ptoms (Rasmussen 1986). This rule-based level of performance is consistent with the production system paradigm (Brownston et al. 1985) that underlies m ost ESs. Possibilities thus exist that ESs can poten-tially serve as a training aid to improve human rule-based reasoning in emergency managem ent.Third, ESs are distinct in being able to m ake the process of rule-based reasoning transparent to users through the so called explanatory facility (Luger and Stubble® eld 1989). This feature enables a human ± ES interactive environm ent where, from the explanatory feedback, people can learn how ESs conduct rule-based searches.
0144-929X/98 $12.00Ó 1998 Taylor & Francis Ltd.
In their exploratory work, Sharit et al. (1993) found supportive results for the use of ESs to train for emergency managem ent. However, there is a fundam en-tal limitation of the study that m ust be addressed. This inadequacy relates primarily to the lack of consideration of time constraints, which is the crux that gives rise to the viability of ES-based training in the emergency domain. It has been suggested that the transfer of training from ordinary conditions to those of time pressure situations might be poor (Zakay and W ooler 1984). In order to investigate further training methods with real-world implications, the present study is aimed at examining the eŒectiveness of ES-based training by taking into account the eŒect of time pressure.
Time pressure has been interpreted as one among a num ber of task characteristics that determine the costs and bene® ts of using particular strategies (M aule and Hockey 1993, Payne et al. 1988, 1990). In this conceptualization, costs refer to the mental resource implication of implem enting a particular strategy, while bene® ts are considered as the values accruing to decisions made when using that strategy (M aule and Hockey 1993). The dominant theme has been that people appear to adapt to diŒerent strategies as a function of tim e pressure with a minimum of cognitive eŒorts in cost
/
bene® t calculation (Beach and M itchell 1978, Edland and Svenson 1993, Payne et al. 1988).The hum an’ s adaptive decision behaviour in reaction to tim e pressure can be demonstrated by various aspects of adjustm ent (M aule and Hockey 1993, Payne et al. 1988). For example, under time pressure, people tend to accelerate their decision process by increasing the rate of information search (Ben Zur and Breznitz 1981, M aule and M ackie 1990, Payne et al. 1988). W hen tim e pressure is so pressing that acceleration alone fails to satisfy task dem ands, people adapt to time pressure by resorting to an increased use of heuristics-based decision rules. Payne et al. (1988) found that, under tim e pressure, intuitive strategies (e.g. the elimination-by-aspects rule) were adopted by subjects in place of normative strategies (e.g. the weighted additive rule). The employment of sim plifying strategies under tim e constraints was also re¯ ected in the human’ s adoption of ® ltering, in which selective processing of information occurs (Ben Zur and Breznitz 1981, Edland 1993, Svenson et al. 1990, W right 1974).
Given this contingent nature, it is likely that the way people perform rule-based emergency managem ent tasks will be in line with the adaptive mechanisms underlying time-constrained human information pro-cessing. Speci® cally, we predict that people will tend to rely on heuristics-based manipulation of rules, in an accelerating manner, when deriving rule-based counter-measures under time pressure.
The prevalent use of heuristics, however, often results in decision biases that eventually lead to severe human errors (Kahneman et al. 1982). In order to prevent potentially catastrophic susceptibility in decision mak-ing, several prescriptions, emphasizing training with norm ative decision procedures have been attempted and som e of them were at least partially successful (Bell et al. 1988). For example, Fong et al. (1988) found that subjects who received formal statistical training en-hanced their capability for averting reasoning errors that resulted from the use of statistical heuristics. In Zakay and W ooler’ s study (1984), the use of non-compensatory multi-attribute utility procedures led to improvements in the eŒectiveness of decisions made.
Considering these promising eŒorts, successful man-agement of emergencies seems feasible if hum ans are trained with ESs which can derive solutions that are always norm atively correct by employing formal Arti-® cial Intelligence (AI) techniques (Luger and Stubble-® eld 1989). The ES-based normative search mechanism is expected to provide a resource to which humans can revert when the tendency towards heuristic processing of rule-based information is taking place. Therefore, we hypothesize that individuals who undergo ES-based training would be better equipped to make accurate and consistent rule-based decisions when emergency inci-dents have to be resolved under extreme time pressure.
2. M ethodology
2.1. Independent variables
Given the purpose of the present study, time pressure and training were manipulated as independent variables. Tim e pressure was operationalized by two deadline conditions and was designed as a within-subjects factor in order to provide a strong test of the hypothesized adaptivity. The subjects in the `no time pressure’ condition were allowed to complete a task at whatever pace they wished, whereas the subjects in the `time pressure’ condition were required to ® nish a task within 90 seconds (the 90 second constraint was found to constitute time pressure in a pilot study). Training was designed as a between-subjects factor and de® ned by two treatment levels. The ES group received lines of reasoning generated by an ES. The control group, however, received no such information.
2.2. Subjects
Forty undergraduate students in industrial engineer-ing at a m ajor university served as subjects. Participa-D.-Y.M . Lin and Y.-L. Su
196
tion in the experiment earned credit toward ful® llm ent of a course requirement. N one of the forty subjects had taken ES
/
AI related courses prior to participating in the experiment. The subjects were randomly assigned to one of the two training conditions, with each group receiving twenty participants. All subjects completed the experi-ment successfully.2.3. Expert system development
An ES in em ergency management of chemical spills (Johnson and Jordan 1983) was developed to serve as the training tool. The knowledge base of the ES consisted of ® fty-two domain-speci® c rules, in the form of `IF symptoms TH EN action’ , that dealt with various aspects of spill managem ent (see Appendix A for som e sample rules). A `HOW ’ explanatory facility (Luger and Stubble® eld 1989) was program med to demonstrate how the ES employs a normative search to chain relevant rules to prove a query. This facility is particularly signi® cant because it is the reasoning lines generated by the justi® er that serve as the prescription for debiasing intuitive processing. The ES also included an interface in which subject interaction with the ES could be conducted in a friendly, natural language environment.
2.4. Stimulus material
/
query systemsThe query system represented a set of scenarios made up of queries and associated facts that simulated spill incidents (see Appendix B for a sample scenario). There were two sets of query systems. One was for training and consisted of four scenarios. The second was for experimental tests and included ten scenarios. Both sets of query systems were manipulated to chain the sam e num ber of rules (eleven) so that the fourteen queries were identical in terms of processing di culty.
2.5. Performance measures
In order to test the aforementioned hypotheses, three aspects of rule-based performance were measured. The ® rst measure was the average tim e spent in acquiring each item of rule-based information (i.e. each symptom value). This response measure was to test the hypothesis of acceleration of rule-based processing under tim e pressure.
The second m easure exam ined rule-based processing in terms of the search pattern employed by the subject. The hypothesized rule-based decision behaviour con-cerned whether it was an ES-based search (Type E) or a
heuristics-based search (Type H). The measure was thus de® ned by an index that calculated the num ber of Type E m ovements, minus the number of Type H movements, divided by the total movements of both types. This performance m easure ranged from + 1.0 to Ð 1.0, with + 1.0 indicating 100% employment of ES-based norma-tive strategies, and Ð 1.0, 100% employment of heur-istics-based strategies.
The third measure related to rule-based perform ance in terms of the level of accuracy achieved. Accuracy was operationalized as the percentage of the rule-based queries that were solved correctly.
2.6. Information acquisition methodology: the CRIB Information board methodology is a process tracing technique that oŒers great potential to capture human tim e pressure decision-making processes (M aule and Svenson 1993). A Computerized Rule-based Informa-tion Board (CRIB) was developed in the present study in order to collect data for examining the perform ance m easures.
The CRIB system was programmed to run in a M icrosoft W indows environment on an IBM PC. The PC was equipped with a m ouse for performing a variety of functions. The system included a scenario window that displayed the test query system. Directly beneath the window was a panel that consisted of ten item s in a 5
´
2 matrix. Each item included an attribute (symptom ) and alternative values associated with that attribute. The item at the bottom of the screen provided choices for the decision outcom e. Rule-based search activities performed by subjects were carried out through a sequence of information selection by clicking the m ouse on appropriate symptom values and on a proper decision choice.Each state value and the outcome choice could be repeatedly selected or deselected (e.g. to change pre-viously entered values) until the `STOP’ button was clicked. The `NEX T’ button allowed the system to proceed to a new session (i.e. a new test query). The CR IB recorded, for each test query, the outcome choice, response time spent for processing each sym ptom value, and the order in which those values were selected.
Time constraint conditions were also controlled by the system through the deadline window. The window signaled a 90-second countdown as soon as a tim e-constrained test query was presented. W hen the count-down entered the last 15 seconds a beep sounded each second as a warning. The window displayed `NULL’ when a no time pressure session began. Figure 1 shows a completed session of a tim e-constrained test query on the CRIB.
2.7. Procedures
The entire experiment consisted of the following stages:
1. M em orization session: All subjects were required to memorize the ® fty-two domain rules, with an emphasis on being able to recognize the associa-tion between the symptom values in the `IF’ section and the action in the `THEN’ section. This task was performed as a take-home assign-m ent.
2. Pre-training mem ory test: All subjects were required to take a test in this session to demonstrate their knowledge of the rules. The test included twenty-two blank-® lling questions in which the subjects were asked to provide asso-ciated `IF’ symptom values, given a `THEN’ section, or associated `THEN’ actions, given an `IF’ section. Only subjects who answered all the questions correctly quali® ed for the next training session. The adoption of such a strict measure was to exclude a possible extraneous situation where the failure in rule-based reasoning resulted from forgetting of the rules. Subjects who scored unsatisfactorily were instructed to review the rules and rescheduled for a make-up test.
3. Training session: This training session was con-ducted on a SUN computer workstation. Each subject in both training groups was instructed to open a scenario window that displayed the training query system. The ES subjects were then required to interact with the ES and the associated `HOW ’ explanatory justi® er to observe the
nor-mative reasoning process derived by the ES. However, instead of having access to the ES, the control subjects were required to derive solutions for the training queries by searching among the ® fty-two dom ain rules, which appeared randomly in a rule window.
4. Test session: In this session, all forty subjects were required to solve the ten test queries on the CRIB. Three warm-up exercises and a lea¯ et of opera-tional instructions were provided to allow the subject to become familiar with the system. The ten test queries were separated, by the time pressure variable, into two categories. One half of the queries were designated for the 90-second deadline condition and the other half for the no tim e pressure condition. The order in which these ten replicates (queries) were presented to the subject was randomized independently for each subject. All subjects were asked to enter the answer to the CRIB immediately on ® nishing each of the ten test queries.
3. Results and analysis
The m eans and standard deviations for the three response measures are summarized in table 1. Separate Analyses of Variance (AN OVA ), with one within-subjects factor (time pressure) and one between-within-subjects factor (training), were performed on the three measures. Interactions were examined where appropriate. In-depth analysis of the interactions was conducted by the method of sim ple main-eŒects (K irk 1995). Interpreta-tion of simple main-eŒects results concerned, ® rst, how subjects adapted to time pressure in symptomatic search of the rules, followed by an examination regarding the eŒectiveness of ES-based training.
The ANOVA results on accuracy showed that the main eŒect of time pressure was signi® cant (F[1,38]= 71.06, p
<
0.0001) but there was no signi® canteŒect of training (F[1,38]= 2.21, ns). However,
explana-tion of the two main eŒects must be quali® ed since there was a signi® cant time pressure by training interaction (F[1,38]= 4.44, p
<
0.05). The results of sim plemain-eŒects analysis revealed that, clearly, time pressure had a destructive eŒect on the perform ance of rule-based reasoning in terms of the level of accuracy. This was evidenced by the fact that the subjects in both training conditions suŒered a signi® cant decrease in accuracy when confronting time pressure (M = 68% vs. M = 38% , F[1,38]= 55.52, p
<
0.0001 for the control group, andM = 70% vs. M = 52% , F[1,38]= 19.99, p
<
0.0001 for theES group). However, under the presence of a time constraint, the normative manipulation of rules ob-D.-Y.M . Lin and Y.-L. Su
198
Figure 1. A completed session of a time-constrained test query on the CRIB.
served in ES training did exhibit its competence in sustaining a reasonable level of rule-based performance (M = 52% vs. M = 38% , F[1,76]= 5.29, p
<
0.05).W ith regard to the measure of search time, there was a signi® cant eŒect of time pressure (F[1,38]= 288.24,
p
<
0.0001) but the main eŒect of training was insig-ni® cant (F[1,38]= 3.86, ns). However, the existence of asigni® cant interaction (F[1,38]= 4.86, p
<
0.05) called forfurther investigation of the main eŒects. The sim ple main-eŒects analysis indicated that acceleration of rule-based searching was obvious with the presence of tim e pressure. This was con® rmed by the evidence that the imposition of a deadline forced subjects to speed up their rule-based information processing, regardless of which training condition they were in (M = 9.38 s vs. M = 7.52 s, F[1,38]= 109.13, p
<
0.0001 for the controlcondition, and M = 9.07 s vs. M = 6.66 s, F[1,38]= 183.97,
p
<
0.0001 for the ES condition).On the other hand, expeditious processing of rule-based information was found to bene® t from the experience learned from the ES navigation of rule searching, but only when time pressure was present. Under time pressure, the subjects who had received ES training performed rule-based reasoning more rapidly than those in the control group (M = 6.66 s vs. M = 7.52 s, F[1,76]= 7.13, p
<
0.01). The diŒerence inprocessing speed between the tw o training groups in the no time pressure condition was, however, not signi® cant (F[1,76]= 0.9, ns).
W ith respect to the measure of the search pattern, the ANOVA results indicated that time pressure did have a signi® cant impact on subjects’ rule-based decision making where their search behaviour was dom inated by heuristic-based processing (F[1,38]= 38.31, p
<
0.0001).On the other hand, training also demonstrated a signi® cant eŒect on subjects in facilitating the use of norm ative rule-based reasoning procedures (F[1,38]= 75.06, p
<
0.0001). The results, however,re-quired in-depth examination due to the presence of a signi® cant interaction (F[1,38]= 8.26, p
<
0.01). Theana-lysis of simple main-eŒects con® rmed the hum an’ s adoption of heuristics-based strategy selection when
under time pressure. It was found that subjects exhibited a natural tendency to use intuitive strategies to reason with rules, and this tendency became signi® cantly stronger as a result of the subject’s adaptation to time constraints (M = Ð 0.36 vs. M = Ð 0.45, F[1,38]= 5.49, p
<
0.05). Similarly, the adjustment towards increased use of heuristic rule processing under time pressure also occurred in subjects who received ES training (M = 0.35 vs. M = 0.11, F[1,38]= 41.08, p<
0.0001).The simple main-eŒects results indicated, however, that the tendency towards the reliance on heuristic searching of rules could be deterred to a considerable extent by normative ES training. The subjects in the ES group displayed a relatively norm ative type of rule-based reasoning behaviour when no tim e constraint was imposed, as compared to those in the control group (M = 0.35 vs. M = Ð 0.36, F[1,76]= 83.24, p
<
0.0001). This decision pattern was also sustained when perfor-m ance was exaperfor-mined under tiperfor-m e pressure (M = 0.11 vs. M = Ð 0.45, F[1,76]= 51.06, p<
0.0001).4. Discussion
Overall, the experiment results support our hypoth-eses. One of the prim ary hypotheses predicted an accelerating, yet intuitive decision-making pattern in the rule-based emergency domain and this prediction was con® rmed. It appears this acceleration is a m anifested strategy people adopt to complete a task before an imposed deadline. However, the subject’s rule-based performance does not seem to capitalize on the increased rate of inform ation processing. This is probably because the subject’ s rule processing is dom inantly biased by simplifying heuristics. It has been suggested that some decisions are so important that exclusive use of intuitive strategies is highly inadvisable (Nisbett and Ross 1980). This is particularly true for emergency management of risks as this application area inherently bears safety-related consequences. In fact, the disadvantageous situations facing human operators who supervise emergency incidents are evidenced by the Table 1. Means and (standard deviations) of performance measures for each training condition as
a function of time pressure.
No time pressure Time pressure
Control ES Control ES
Accuracy (% ) Search time (second) Search pattern 68 (18) 9.38 (1.22) Ð 0.36 (0.19) 70 (17) 9.07 (0.72) 0.35 (0.27) 38 (23) 7.52 (1.13) Ð 0.45 (0.20) 52 (20) 6.66 (0.93) 0.11 (0.28)
con® rmed hypothesis that predicted a signi® cant de-crease in decision accuracy due to the detrimental impact of tim e pressure.
An analysis of the subjects’ search movem ents on an individual basis will give us a more explicit under-standing of the mechanisms that underlie the biased decision-making behaviour. W e found that, under no time pressure conditions, 67% of the subjects in the control group chose to process those rules whose symptom values can be directly sampled from the scenarios, prior to considering any rules where the values of the antecedents need to be deduced. W hen tim e pressure was present, 87% of the control subjects resorted to the sam e approach in the m anipulation of rules. This was also true for the subjects who received ES training. An increase of 23% (from 31% to 54% ) of the ES subjects employed the strategy as a result of the imposition of time pressure.
These results appear to suggest that, due to the limited capacity of human short-term memory (STM ) (W ickens 1992), people tend to em ploy the heuristic of availability (Tversky and Kahneman 1982) in the processing of rule-based information. This is done by assigning unwarranted weight to the rules that are prominent in terms of their availability to STM . The ease with which these rules can be brought into the hum an’ s cognitive resources make the mental shortcut an appealing strategy in adapting to time pressure, which explains the much stronger phenomenon of heuristics-based rule processing in the presence of tim e constraints.
Given this speculation, any contingency procedure
/
rule that is readily available to the decision-m aker’ s mental resources can be the origin of biased decision-making behavior and therefore of possible human error. These rules may include, for exam ple, rules whose symptom values are directly provided in an incident, and rules associated with high-frequency events.Chances are that, biased by availability heuristics, people would get lost during their search process and never ® nd a way out (i.e. fail to derive a correct solution), particularly when the rules selected on the intuitive basis disperse in the state space. This is very likely as real-life emergency tasks often demand a much larger scale of domain knowledge, and on-scene human operators are usually not so well prepared in rule memorization as the subjects are in an empirical setting. These ® ndings undoubtedly demonstrate the vulner-ability people are prone to in managing rule-based emergencies and justify the need for training in the time-constrained decision making.
Note that, in addition to the major implications, the results of the present study may also serve as a guideline for inform ation display design. W e suggest that
rule-based information for emergency management should be presented to decision-makers with equal salience so that rules carrying cognitive availability will not be processed with unwarranted precedence.
The hypothesis on the eŒectiveness of ES-based training was also con® rmed. In contrast to the intuitive selection of rules based on cognitive availability, the ES search mechanism paves a solution path along which rules are systematically chained in approaching the goal state (i.e. the answer to a query). Apparently, normative navigation proves to be an e cient strategy in proces-sing rule-based information, especially when the answer has to be derived within a restrictive deadline. M ean-while, ES-based reasoning makes the subject realize that the use of cognitive availability would eventually lead to a search dead-end. This interpretation appears to explain the signi® cant eŒect of ES training in sustaining a normative pattern of rule-based decision making where intuitive processing would dictate otherwise. It appears that it is also the systematic navigation of rules that facilitates expeditious and fairly accurate perfor-mance under the im position of deadline constraints. These results obviously exhibit the potential of ESs as an eŒective training prescription for rule-based reasoning under time pressure.
However, it should be noted that the supportive results do not im ply that the use of normative algorithm s to training can be applied universally. For example, Zakay and W ooler (1984) found that, for multi-attribute evaluations, the positive eŒect of norma-tive training in ordinary conditions did not transfer to those of time pressure. This inconsistency seems to suggest that the advantages of decision training with norm ative models do not necessarily generalize across task domains. Therefore, it is necessary to investigate the nature of a speci® c task in order to de® ne the strategy to be used under time pressure before we can postulate the training eŒect.
The justi® ed paradigm of ESs as training devices does have a role to play in the development of training programmes for emergency managem ent. Em ergency training eŒorts have typically been focused on building simulators that simulate real-world emergency situa-tions with various aspects of ® delity (G ovindaraj et al. 1996). Simulator training has been employed for a num ber of applications such as marine power plants (Su and Govindaraj 1986), and com mercial aviation (Flex-man and Stark 1987). This methodology assumes that the scenarios humans are trained for can be identi® ed in advance. In contrast, the primary bene® ts of ES-based training methodology lie in the concepts and strategies employed in the AI framework (e.g. forward
/
backward chaining). This ES/
AI knowledge enables people to resolve em ergency problem s arising from a broad D.-Y.M . Lin and Y.-L. Su200
spectrum of situations that can not be classi® ed before-hand. Therefore, ES-based training can be viewed as a complem ent to the conventional simulator training method.
It is also interesting to note that in the present study, the signi® cant eŒects of ES training were achieved through a very small amount of training (i.e. only four training trials). These training trials, however, consisted of speci® c feedback regarding the exhaustive tracing of norm ative rule-based reasoning. This ® nding renders ESs as an appealing training option when the imple-mentation of costly simulator training is not a viable alternative. Another im plication of this ® nding is that feedback with a higher degree of speci® city may represent a more powerful de-biasing technique for time-constrained decision making. It is suggested that in practice, more informative feedback should be given higher priority when a substantial amount of training is not possible.
5. Conclusions
M anagement of em ergency situations often requires hum an operators to make prom pt and accurate decisions under stringent time constraints. The present study intends to contribute to the human’ s success in playing such a role by examining the eŒectiveness of ES-based training under time pressure.
The results of this study imply detrimental eŒects of time pressure on successful managem ent of emergency abnormalities. People are found to accelerate their decision process by putting unwarranted weight on rules that carry salience of cognitive availability. Decision quality suŒers from such an intuitive decision pattern. H owever, this tendency towards heuristic searching of rules can be deterred by normative training where the formal deductive mechanisms ESs employ in rule processing are provided. People undergoing ES-based inference strategies are immune to the availability bias and are capable of deriving more expeditious and successful solutions.
W hile the present study provides some of the ® rst ® ndings regarding human rule-based reasoning under time pressure, and the eŒectiveness of ES training in improving such performance, there are limitations to this research which must be addressed. First, the knowledge base for the selected application domain in the experim ent was kept monotonic and reliable. Although this simplicity was needed to control the scale of the tasks appropriate for the subject, the problem s arising in real-world emergencies often bear data that are uncertain or incomplete. Future research calls for the need to incorporate uncertainty in dom ain
knowl-edge in order to understand better the real-life implica-tions of the issues examined in this study. This can be done by training people with an ES built with models that handle uncertainty, such as fuzzy logic (Zadeh and K acprzyk 1992), and the certainty factor theory (Buchanan and ShortliŒe 1984). Secondly, most emer-gency problems diŒer in the degree of processing di culty. Task di culty may play an important role in in¯ uencing hum ans’ rule-based performance under tim e pressure (Sharit et al. 1993). Therefore, it is necessary to diŒerentiate the number of rule chaining to investigate the hard-easy eŒect in future research.
Acknowledgement
This study received support from the National Science Foundation Council of Taiw an under grant N SC-83-0415-E-009-006.
Appendix A: Some sample rules of the ES knowledge base
Rule 4: IF (victims are contam inated) AND (victims have blood circulation
problems)
TH EN (perform treatment S on the victims) Rule 7: IF (the spill area is
>
10 mm) AND(the spill is classi® ed as type A) TH EN (establish command post C2)
Rule 24: IF (the spill substance is chlorine) AND (the spill density is
>
5 ppm)TH EN (classify the spill as type A) Rule 37: IF (perform treatment S on the
victims) AN D
(take evacuation route X)
TH EN (assign the victims to the RED ® rst aid zone)
Appendix B: A sample spill scenario
/
query system [Query]: Given the following facts, please identify theemergency level of the spill incident . . . A, B, C, D, or E?
[Facts]: the spill is taking place in the chip production zone; spill substance is chlorine; night working shift is on duty; spill area is
>
10 mm ; victims are contaminated; there is on-scene explosion; spill density is>
5 ppm; victims have breath-ing problems.References
BEACH, L. R. and MITCH ELL, T. R. 1978, A contingency model for the selection of decision strategies, Academy of
Manage-ment Review, 3, 439 ± 449.
BELL, D. E., RAIFFA, H. and TVER SK Y, A. 1988, Decision Making: Descriptive, Normative, and Prescriptive Interac-tions, (Cambridge, Cambridge University Press).
BENZUR, H. and BREZNIT Z, S. J. 1981, The eŒect of time pressure on risky choice behavior, Acta Psychologica, 47, 89 ± 104.
BROW NSTO N, L., FARR ELL, R., KAN T, E. and MARTIN, N. 1985,
Programming Expert Systems in OPS5: An Introduction to Rule-Based Programming (Reading, Addison-W esley).
BUCHA NAN, B. G. and SHORTLIFFE, E. H. 1984, Rule-Based
Expert Systems: The MYCIN Experiment of the Stanford Heuristics Programming Project (Reading, Addison-W esley).
EDLAND, A. 1993, The eŒects of time pressure on choices and
judgment of candidates to a university program, in O. Svenson and A. J. Maule (eds), Time Pressure and Stress in
Human Judgment and Decision Making, (New York, Plenum
Press) 145 ± 156.
EDLAND, A. and SVENSON, O. 1993, Judgm ent and decision
making under time pressure: studies and ® ndings, in O. Svenson and A. J. Maule (eds), Time Pressure and Stress in
Human Judgment and Decision Making (New York, Plenum
Press) 27 ± 40.
FLEXM AN, R. E. and STA RK, E. A. 1987, Training simulators, in
G. Salvendy (ed.), Handbook of Human Factors (New York, John Wiley), 1012 ± 1038.
FONG, G. T., KRAN TZ, D. H. and NISBETT, R. E. 1988, The eŒect of statistical training on thinking about everyday problems, in D. E. Bell, H. RaiŒa and A. Tversky (eds), Decision
Making: Descriptive, Normative, and Prescriptive Interac-tions (Cambridge, Cambridge University Press, Cambridge),
299 ± 340.
GOVIND ARAJ, T., SU, Y.-L., VA SA ND ANI, V. and RECKER, M. M.
1996, Training for diagnostic problem solving in complex engineered systems, in W. B. Rouse (ed.), Human/ Technol-ogy Interaction in Complex Systems, Volume 8, (London,
JAI Press) 1 ± 66.
HAY ES-ROTH, F., WATERM AN, D. A. and LENAT, D. B. 1983,
Building Expert Systems (Reading, Addison-W esley).
JOH NSON, C. K. and JORDA N, S. R. 1983, Emergency
manage-ment of island oil and hazardous chem ical spills: a case study in knowledge engineering, in F. Hayes-Roth, D. A. Waterman, and D. B. Lenat (eds), Building Expert Systems (Reading, Addison-Wesley), 349 ± 375.
KAH NEM AN, D., SLOVIC, P. and TVERSKY, A. 1982, Judgment
under Uncertainty: Heuristics and Biases (New York,
Cam-bridge University Press).
KIRK, R. E. 1995, Experimental Design: Procedures for the Behavioral Sciences (Paci® c Grove, Brooks/Cole).
LUGER, G. F. and STU BBLEFIE LD, W. A. 1989, Arti® cial Intelligence and The Design of Expert Systems (Redwood
City, Benjam in/Cummings).
MAULE, A. J. and HOCKEY, G. R. J. 1993, State, stress, and time pressure, in O. Svenson and A. J. Maule (eds), Time Pressure
and Stress in Human Judgment and Decision Making (New
York, Plenum Press), 83 ± 101.
MAULE, A. J. and MACKIE, P. 1990, A componential investiga-tion of the eŒects of deadlines on individual decision making, in K. Borchering, O. I. Larichev and D. M. Messick (eds), Contemporary Issues in Decision Making (Amsterdam, North-Holland), 449 ± 461.
MA ULE, A. J. and SVEN SO N, O. 1993, Theoretical and empirical
approaches to behavioral decision making and their relation to time constraints, in O. Svenson and A. J. Maule (eds),
Time Pressure and Stress in Human Judgment and Decision Making (New York, Plenum Press), 3 ± 25.
MCFARLA ND, T. D. and PARKER, O. R. 1990, Expert Systems in
Education and Training (Englewood CliŒs, Educational
Technology Publications).
MO RAY, N. 1988, Can decision aids help to reduce risk and
hum an error? A program for research on the management of risk, Report EPR L-88-20, Engineering Psychology Research Laboratory, Department of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign.
NISBETT, R. and ROSS, L. 1980, Human Inference: Strategies
and Shortcomings of Social Judgment (Englewood CliŒs,
Prentice Hall).
PAYN E, J. W., BETTM AN, J. R. and JOH NSON, E. J. 1988, Adaptive
strategy selection in decision making, Journal of
Experi-mental Psychology: Learning, Memory, and Cognition, 14,
534 ± 552.
PAYN E, J. W., JOH NSON, E. J., BETTMAN, J. R. and COU PEY, E.
1990, Understanding contingent decision making: a compu-ter simulation approach, IEEE Transactions on Man,
Systems, and Cybernetics, 20, 296 ± 309.
PEW, R. W. 1988, Human factors issues in expert systems, in M. Helander (ed.), Handbook of Human ± Computer
Inter-action (North-Holland, Elsevier Science Publishers B.V.),
931 ± 940.
RASM USSEN, J. 1982, Human errors: a taxonomy for describing
hum an malfunction in industrial installations, Journal of
Occupational Accidents, 4, 311 ± 333.
RASM USSEN, J. 1986, Information Processing and Human ±
Machine Interactions: An Approach to Cognitive Engineering
(New York, North-Holland).
REASON, J. 1990, Human Error (Cambridge, Cambridge
University Press).
SH ARIT, J., CHEN, S. and LIN, D.-Y. M. 1993, Expert system
based training for emergency management, Journal of
Computing in Civil Engineering, 7, 6 ± 22.
SU, Y.-L. and GOVIND ARAY, T. 1986, Fault diagnosis in a large dynamic system: Experim ents on a training simulator, IEEE
Transactions on Systems, Man, and Cybernetics, SMC-16(1), 129 ± 141.
SVENSON, O., ED LAN D, A. and SLOVIC, P. 1990, Choices between
incompletely described alternatives under time stress, Acta
Psychologica, 75, 153 ± 169.
TU RBAN, E. 1993, Decision Support and Expert Systems. (New
York, Macmillian).
TVERSKY, A. and KAHN EM AN, D. 1982, Availability: A heuristic
for judging frequency and probability, in D. Kahneman, P. Slovic, and A. Tversky (eds), Judgment under uncertainty:
Heuristics and Biases (New York, Cambridge University
Press), 163 ± 178.
WICKENS, C. D. 1992, Engineering Psychology and Human
Performance (New York, Harper Collins).
WR IG HT, P. 1974, The harassed decision maker: time pressure,
distraction and the use of evidence, Journal of Applied
Psychology, 59, 555 ± 561.
ZA DEH, L. and KA CP RYZK, J. 1992, Fuzzy Logic for the
Management of Uncertainty, (New York, Wiley).
ZA KAY, D. and WOOLER, S. 1984, Time pressure, training, and
decision eŒectiveness, Ergonomics, 27, 273 ± 284. D.-Y.M . Lin and Y.-L. Su
202
